Training Quality


I was hired to start up a QA department from scratch for a website agency. The company had been getting along without QA but saw the need to improve their product(s). People had been used to doing things their way, the way that “works”. To get the company in-line  would be a challenge.

The Learning Curve

In past positions, I had just walked into a QA department and hit the ground running (they had established processes). This time, though, I had to figure it all out. The benefit was that I got to do it my way. I have lessons learned from previous companies, so I knew what to keep an eye out for. First, I needed to get on the same page as management. I kept hearing buzzwords from the VP and decided that it would be wise to investigate. This consisted of buying a handful of books (I’ll list them off at the end on this post).

Lessons Learned

I had heard “Scrum” and “Agile” before and was part of half-assed implementations. This half-assery led to abandoning it and going back to “Waterfall“. So I knew I needed to dive in completely and commit. Through that experience, I found that being on the bottom of the totem pole sucks. Management, it felt, came out of he 80s school of business where many upper managers adopted strategies that involved setting vague goals and then delegating vague goals.

… many upper managers adopted strategies that involved setting vague goals and then delegating … the responsibility for meeting those vague goals. – Juran on Quality by Design (p. 24)

To fill you in further, my QA Director at the time had suggested that the team go out and subscribe to Scrum and Agile newsletters. This aggravates me now. If you want to adopt commit the entire department and train them. I think the gap there was that the director didn’t understand the concepts fully. My goal is to understand and train.

Learning for Myself

Years ago, I was offered to have an IIST certification paid for by work. I took the class and hated it. As a QA tester, it was hard to look past such a shitty site that only worked in IE8. I would later read reviews and find that most people felt the same. But I digress. I did learn something. Through out the course, Effective Test Design, I kept hearing the narrator reference a book. Reflecting on this I decided that I could read/pay for other people’s interpretations OR I could read those volumes myself and come to my own conclusions. At another previous job, I had killed time when I was slow by watching whatever I could on YouTube (related to my job). I came across one of James Bach‘s keynotes. In it, he talked about the importance of being an autodidact.

Where to Start?

As I mentioned in The Learning Curve (above), I need to get on the same page as management. So, I looked up this word “kaizen” that I kept hearing. Turns out kaizen comes from Taiichi Ohno’s Workplace Management. I bought the book and read it in a few sittings at home. It is a short read and I highly recommend you read it, no matter your position. From this book, I gained an understanding of “Lean“. The company that hired me to create a QA department is big on Lean.

My next book was one that I think was referenced in the IIST course I took called, “The Art of Software Testing“. That book seems to have been the QA Bible for the past 25+ years. Reading through, my mind was on the look out for things we could try. I tell my team to “be all kaizen and shit”; meaning “let us try, committing fully, to continuously improve.” Some lessons learned from the book didn’t apply to just QA. I was able to establish guidelines for Code Reviews (The Art of Software Testing, p. 22) and give our sales team a quote or two so they could sell our QA service as a product (The Art of Software Testing, p. 141).

Next up, “Juran on Quality Design“. This one was recommended to one of my employees by a family friend. A the tag line states, it is about planning for Quality. From this I learned that it is best to train everyone to be experts on quality, rather than to rely on a couple “experts”.

There are some many lessons learned from each book, that I could write a post on each. I’ll keep it short and sweet right now, since I am focused on the subject title, “Training Quality”.

Lessons Learned

  • Commit fully
  • Continuously Improve
  • Always look for “waste”
  • Implement new processes that you believe will have benefit
  • Be prepared to admit when things do not work
  • Management needs to be with and around their team
  • Quality needs to be understood by everyone
  • Companies automated processes to save time, but end up automating bad processes

Implementing Changes

Some changes are easier than others to implement. This is mostly due to the perceived interruptions. I had to tell the sales and project management teams that billable hours were going to go up. This shouldn’t have been a surprise but I had set a dangerous precedent. In my efforts to hit the ground running, I did my best to be frictionless as possible and just hammer through tickets until I could judge the work load and get he appropriate number of people working under me. This meant that requirements were lax and no test plans or automation was being done.

First Things First

We needed to get our tickets (for “Kanban“) under control. No one ticket looked like another. We needed a standard. I worked with the VP and PM Director to hammer out a template that would fit all user stories. It ended up being based off of Kent Beck‘s “user stories“.

Without standards, there can be no Kaizen – Taiichi Ohno

It was an uphill battle as people used “what worked” before. I got the VP involved and we sat down with the PM/PO team to explain why the change was needed. Next, we needed to police it. If you tell someone to change the process and they revert, or just as worse “half-ass” it, then no real benefits can be sussed out. I asked the Dev team to help by pushing back tickets (moving it left on the Kanban board) until it was right. This resulted in a lot of side convos of “Oh, I meant this…”, which was never put into the ticket. I had adopted the mantra, “If it wasn’t written, it wasn’t said”. QA passes or fails tickets based off of the documented Acceptance Criteria(s). So then when QA got the ticket in our queue, we would have to send it left.

One suggestion we ran with is “moved-left” tracking. We, in QA, would note on a shared spreadsheet who was fucking up and why. This included anything that made a ticket “move left” on the board by anyone (PMs, Designers, Developers, QA). This was met with some resistance and I had to clarify that we are not on a witch hunt. We are just looking for common issues among people in the same departments or even a single person making the same mistakes, repeatedly. To alleviate the dark nature, we added a “kudos” tab to call out people who are improving. I jokingly hand out a gold star each week. With this arrow in our proverbial quiver, we are able to address issue with standards and metrics!

Lead by Example

We in QA also needed to try out a few things and improve the process while it was still in its infancy. Here is a short list of things we tried and their results:

  • Extreme QA: Taking a page from Extreme Programming by having two testers on a single ticket
    • Result: Abandoned. We felt like more issues were being caught but our backlog quickly grew.
  • 4x10s: We were light on work on Monday and Friday so I asked my team to pull 4x10s one sprint.
    • Result: Optional. We saw that our billable hours evened out but at the cost of energy.
  • Automating Regression: Automate a ticket after it has been manually tested. A ticket does not move right (to UAT) until it has been automated.
    • Result: Partial. We had to find a balance. Working at an agency makes it so that a project in constantly in a state of flux until it is done. We now automate everything on the backend and use our best judgement for front-end.
  • Test Plans
    • Result: Kept. We went through a few iterations of Test Plan templates and I think our current iteration will suffice. The long story short here is that the “old way” of doing it was too granular. Now it is essentially a collection of user stories.

By trying these and communicating our efforts, we show the company that it is okay to try and fail. I want to take a moment and point out something I picked up from Ohno. A manager does not need reports on processes, they should be on the floor (the “Gemba“) with the team and are able to see it for themselves.

“Go see, ask why, show respect” – The words of Toyota Chairman Fujio Cho

Books Mentioned