Training Quality

Background

I was hired to start up a QA department from scratch for a website agency. The company had been getting along without QA but saw the need to improve their product(s). People had been used to doing things their way, the way that “works”. To get the company in-line  would be a challenge.

The Learning Curve

In past positions, I had just walked into a QA department and hit the ground running (they had established processes). This time, though, I had to figure it all out. The benefit was that I got to do it my way. I have lessons learned from previous companies, so I knew what to keep an eye out for. First, I needed to get on the same page as management. I kept hearing buzzwords from the VP and decided that it would be wise to investigate. This consisted of buying a handful of books (I’ll list them off at the end on this post).

Lessons Learned

I had heard “Scrum” and “Agile” before and was part of half-assed implementations. This half-assery led to abandoning it and going back to “Waterfall“. So I knew I needed to dive in completely and commit. Through that experience, I found that being on the bottom of the totem pole sucks. Management, it felt, came out of he 80s school of business where many upper managers adopted strategies that involved setting vague goals and then delegating vague goals.

… many upper managers adopted strategies that involved setting vague goals and then delegating … the responsibility for meeting those vague goals. – Juran on Quality by Design (p. 24)

To fill you in further, my QA Director at the time had suggested that the team go out and subscribe to Scrum and Agile newsletters. This aggravates me now. If you want to adopt commit the entire department and train them. I think the gap there was that the director didn’t understand the concepts fully. My goal is to understand and train.

Learning for Myself

Years ago, I was offered to have an IIST certification paid for by work. I took the class and hated it. As a QA tester, it was hard to look past such a shitty site that only worked in IE8. I would later read reviews and find that most people felt the same. But I digress. I did learn something. Through out the course, Effective Test Design, I kept hearing the narrator reference a book. Reflecting on this I decided that I could read/pay for other people’s interpretations OR I could read those volumes myself and come to my own conclusions. At another previous job, I had killed time when I was slow by watching whatever I could on YouTube (related to my job). I came across one of James Bach‘s keynotes. In it, he talked about the importance of being an autodidact.

Where to Start?

As I mentioned in The Learning Curve (above), I need to get on the same page as management. So, I looked up this word “kaizen” that I kept hearing. Turns out kaizen comes from Taiichi Ohno’s Workplace Management. I bought the book and read it in a few sittings at home. It is a short read and I highly recommend you read it, no matter your position. From this book, I gained an understanding of “Lean“. The company that hired me to create a QA department is big on Lean.

My next book was one that I think was referenced in the IIST course I took called, “The Art of Software Testing“. That book seems to have been the QA Bible for the past 25+ years. Reading through, my mind was on the look out for things we could try. I tell my team to “be all kaizen and shit”; meaning “let us try, committing fully, to continuously improve.” Some lessons learned from the book didn’t apply to just QA. I was able to establish guidelines for Code Reviews (The Art of Software Testing, p. 22) and give our sales team a quote or two so they could sell our QA service as a product (The Art of Software Testing, p. 141).

Next up, “Juran on Quality Design“. This one was recommended to one of my employees by a family friend. A the tag line states, it is about planning for Quality. From this I learned that it is best to train everyone to be experts on quality, rather than to rely on a couple “experts”.

There are some many lessons learned from each book, that I could write a post on each. I’ll keep it short and sweet right now, since I am focused on the subject title, “Training Quality”.

Lessons Learned

  • Commit fully
  • Continuously Improve
  • Always look for “waste”
  • Implement new processes that you believe will have benefit
  • Be prepared to admit when things do not work
  • Management needs to be with and around their team
  • Quality needs to be understood by everyone
  • Companies automated processes to save time, but end up automating bad processes

Implementing Changes

Some changes are easier than others to implement. This is mostly due to the perceived interruptions. I had to tell the sales and project management teams that billable hours were going to go up. This shouldn’t have been a surprise but I had set a dangerous precedent. In my efforts to hit the ground running, I did my best to be frictionless as possible and just hammer through tickets until I could judge the work load and get he appropriate number of people working under me. This meant that requirements were lax and no test plans or automation was being done.

First Things First

We needed to get our tickets (for “Kanban“) under control. No one ticket looked like another. We needed a standard. I worked with the VP and PM Director to hammer out a template that would fit all user stories. It ended up being based off of Kent Beck‘s “user stories“.

Without standards, there can be no Kaizen – Taiichi Ohno

It was an uphill battle as people used “what worked” before. I got the VP involved and we sat down with the PM/PO team to explain why the change was needed. Next, we needed to police it. If you tell someone to change the process and they revert, or just as worse “half-ass” it, then no real benefits can be sussed out. I asked the Dev team to help by pushing back tickets (moving it left on the Kanban board) until it was right. This resulted in a lot of side convos of “Oh, I meant this…”, which was never put into the ticket. I had adopted the mantra, “If it wasn’t written, it wasn’t said”. QA passes or fails tickets based off of the documented Acceptance Criteria(s). So then when QA got the ticket in our queue, we would have to send it left.

One suggestion we ran with is “moved-left” tracking. We, in QA, would note on a shared spreadsheet who was fucking up and why. This included anything that made a ticket “move left” on the board by anyone (PMs, Designers, Developers, QA). This was met with some resistance and I had to clarify that we are not on a witch hunt. We are just looking for common issues among people in the same departments or even a single person making the same mistakes, repeatedly. To alleviate the dark nature, we added a “kudos” tab to call out people who are improving. I jokingly hand out a gold star each week. With this arrow in our proverbial quiver, we are able to address issue with standards and metrics!

Lead by Example

We in QA also needed to try out a few things and improve the process while it was still in its infancy. Here is a short list of things we tried and their results:

  • Extreme QA: Taking a page from Extreme Programming by having two testers on a single ticket
    • Result: Abandoned. We felt like more issues were being caught but our backlog quickly grew.
  • 4x10s: We were light on work on Monday and Friday so I asked my team to pull 4x10s one sprint.
    • Result: Optional. We saw that our billable hours evened out but at the cost of energy.
  • Automating Regression: Automate a ticket after it has been manually tested. A ticket does not move right (to UAT) until it has been automated.
    • Result: Partial. We had to find a balance. Working at an agency makes it so that a project in constantly in a state of flux until it is done. We now automate everything on the backend and use our best judgement for front-end.
  • Test Plans
    • Result: Kept. We went through a few iterations of Test Plan templates and I think our current iteration will suffice. The long story short here is that the “old way” of doing it was too granular. Now it is essentially a collection of user stories.

By trying these and communicating our efforts, we show the company that it is okay to try and fail. I want to take a moment and point out something I picked up from Ohno. A manager does not need reports on processes, they should be on the floor (the “Gemba“) with the team and are able to see it for themselves.

“Go see, ask why, show respect” – The words of Toyota Chairman Fujio Cho

Books Mentioned

QA’s Role in Systems Development Life Cycle (SDLC)

What is Systems Development Life Cycle

The systems development life cycle (SDLC), also referred to as the application development life-cycle, is a term used in systems engineering, information systems and software engineering to describe a process for planning, creating, testing, and deploying an information system. The systems development life-cycle concept applies to a range of hardware and software configurations, as a system can be composed of hardware only, software only, or a combination of both.
http://en.wikipedia.org/wiki/Systems_development_life_cycle

QA Involvement

It is imperative that QA or a representative of QA be present as early in the SDLC as possible. This will allow the QA member or team to begin their process. This includes, but is not limited to:

  • Resource Planning
    • Who the tester will be
    • What the tester will need
    • What existing test plans can be executed (to include automation)
  • Release Planning
    • Given past projects, what is the likelihood of meeting the proposed release date
    • Can smaller chunks of the User Story be passed to QA sooner
    • Deployment planning
      • Smoke tests should be executed on production environments

Resources for Managing SDLC

Gone are the days of being told there is something to test and reporting pass or fail w/ bugs via email or TPS reports. Many systems can be used in conjunction as well. There are benefits to using a mixture of separate systems, mainly cost, but a fragmented system can lead to a waste of man-hours from managing tasks. The big appeal of a unified system, aside from time savings, is tracking. In a application such as TFS or Atlassian, all User Stories, tasks, issues, and hours are logged in a central location.

A QA member can log into these systems to access the work assigned to them. When isssues are found they can be logged and assigned back the the developer and with non-showstopping bugs work can continue. The developer can see the issue assigned to them and submit a fix.

I have worked in a QA department that used FogBugz for the entire life cycle where a single ticket was passed from the Stake Holder to Dev to QA and then back to the Stake Holder. This system was difficult to see where a project was at a given point. We then made a transition to TFS. Tasks were split up and a User Story would be closed after UAT. The sys admin set up a few TVs around the office that would display a KanBan board. This board (also available online) gave an at a glance report of where a project was in development.

At another company, we used a custom WinForms application for story pointing, sprint planning, and hours tracking. We also used BugTracker.net to track issues. In lieu of a KanBan board they used a whiteboard with post it notes. This fragmented system would mean that there were hours wasted in updating a project in multiple systems as well as having to get up and physically move tasks on the white board. Sometimes a developer would set something ‘Ready for QA’ in the WinForms app but not move the post-it note, or vice-versa. Thankfully, this anecdote ends with the transition to TFS.

After each sprint, an analysis can be done using these systems (TFS, Atlassian) to review the development process. Charts and graphs can be populated to show the sucesses and pitfalls of that last sprint.

Incorporating QA into SDLC

Using a system like KanBan it can be easy to flesh out a sprint or release cycle and visualize the work to be done. In KanBan you would have vertical silos for each stage of the development process. You can also add swim lanes to delineate the work needing to be done at each step of the process or split responsibilities by department.

QA will own the QA column and any work “Ready for QA” should have its resources already planned. If a bug is found an issue is attached to the task and either sent back to Dev or work is continued after Dev is notified of the issue. When all tasks are completed and passed acceptance, the task can be moved to the Deployment column. A build should be kicked off and the project deployed to a production environment.

In production, another round of QA is needed as things like a bad code merge could corrupt the project. The QA team should have well established test plans and automation by this point so the time this step takes ought to be minimal.

QA Columns in KanBan

QA testing consist of many subsets, such as; Unit, Path, Security, Integration, Regression, Automation, and UAT. To have a column for each would render the KanBan board too large and require constant moving of tasks by the QA team. Instead, I recommend only three columns:

  1. QA Testing
    1. Consisting of; Path, Security, Automation Development/Testing, System, Integration
  2. UAT
    1. The stakeholder needs to spend sometime looking at the completed project to ensure it meets the requirements and their expectations of implementation/design
  3. Production Testing
    1. Smoke and Automated test execution

 Further Reading

Professional Application Lifecycle Management with Visual Studio

  • WROX published book written by the Visual Studio evangelist

Succeeding with Agile: Software Development Using Scrum

  • The most popular and highest rated book on Amazon for this topic

 

 

Q&A with a Software Tester

What is a QA software analyst?

Develops, publishes, and implements test plans. Writes and maintains test automation. Develops quality assurance standards. Defines and tracks quality assurance metrics such as defect densities and open defect counts. Requires a bachelor’s degree and 0-2 years of experience coding in C, C++, Java. Must have a working knowledge of quality assurance methodologies. Familiar with NT, UNIX and/or Solaris environments. Relies on experience and judgment to plan and accomplish goals. Performs a variety of tasks. Works under general supervision; typically reports to a manager. A certain degree of creativity and latitude is required.
Salary.com

What is a typical day in QA like?

Typically, a QA member will:

What is the career path?

Note: There is no formal system or naming convention.

  1. Software Tester –  Entry level testing positions (testers) that do black-box testing.
    1. Requires an aptitude for PC troubleshooting
    2. Executes Test Plans
  2. Analyst – More experienced testing positions do white-box testing.
    1. Requires knowledge of programming logic and design
    2. May also require security testing proficiency
    3. Writes and Executes Test Plans
  3. Automation Specialist  – Uses white-box testing to design for test automation.
    1. Requires knowledge of Computer Science, generally a Bachelor of Science
    2. Converts Test Plans to automated tests
  4. Team Lead – Provides peers guidance.
    1. Can be a Tester, Analyst, or Automation Specialist
  5. Manager – Guides the efforts of the team and makes process adjustments

What are the differences between a developer and an automation specialist?

Both require programming knowledge, however a developer has more. A developer is responsible for writing clean and efficient code to run in production environments. The automation specialist’s code is often only seen and used by QA. That is like how the unit tests are only seen and used by developers. QA’s code is written in large nets of error catching and handling. The program might just try to get record 13 but the tests would have to make sure 13 exists, the call to delete was made, the record 13 was deleted, and no errors were returned.

How do I become a tester?

Like most jobs the education requirements are to scare off people with no confidence. I have seen people hired with no prior QA experience. As long as you show yourself capable of critical thinking you will be fine.

Should I get certifications?

If you want, sure. Will it help? It shouldn’t. You can learn a great deal about testing and even show you retained some of that information for a short while. The problem is that you can take these tests and pass by simply remembering the vocabulary. This does not equate to proficiency.

Instead, I recommend doing some reading on testing. Also, it never hurts to get an online Master of Science in Computer Science for $6,600 from Georgia Tech.

How do I hire a good tester?

There are tests on ProveIt, but those kinds of tests mark you off for clicking a button instead of going to File->Open.

I recommend making your own test:

  • Use a VM snapshot that you can revert to with each interview
  • Give them the tools they need – Internet access and/or pre-installed software that is used by your team
  • Give them 30 minutes to take the test
  • Have a number of bugs ranging in difficulty
    • Easy Bugs – Ones found during Happy Path Testing
      • Example: Clicking ‘submit’ on login form causes 404 error
    • Avg Bugs – Ones found during Negative Testing
      • Example: Entered a letter in a number field
    • Hard Bugs – Ones found using tools
      • Example: Javascript errors in Developer Tools