Manual Software Testing Primer

About Software Testing

Software testing is an investigation conducted to provide stakeholders with information about the quality of the product or service under test. Software testing can also provide an objective, independent view of the software to allow the business to appreciate and understand the risks of software implementation. Test techniques include, but are not limited to the process of executing a program or application with the intent of finding software bugs (errors or other defects).
http://en.wikipedia.org/wiki/Software_testing

Test Environment

The environment in which you will test is likely predetermined by the QA team, Development team, or the Infrastructure team. Use of virtual machines (vm) is standard practice. Virtual machines allow you to pre-configure an environment (OS/Software combo) and save as a snapshot. A tester can fire up the vm and perform his/her testing and revert the vm when finished. Use of a VM is preferred over your workstation, because you want to limit potential risk factors. VMs should contain just your software (and any dependencies) and the OS. You may also want to include some testing tools, like Fiddler.

Common VM Technologies

Testing the Application

The VM is fired up (or you have logged onto your remote testing device) and you have installed the latest build. Now what?

Verify the Build and Deployment

Here is a check list of things to check:

  • Does the application start?
    • For web application deployments, a warm-up test is recommended
      • Warm-up Test: Requests every publicly accessible page and asserts the response is 200 (OK)
        • Should be part of any deployment script
  • If the application requires a SQL connection, then verify the connection settings
    • Usually authentication is tied to a database/server, so try logging in
      • If there is an error:
        • Check the *.config file for your application in a text editor, like Notepad++
          • Inspect the connection string, verify
            • Server
            • Database
            • Authentication method
            • Authentication credentials
        • Ensure the database is online and can be connected to, by
          • (Windows) Run cmd.exe and type ping {serverName}
          • Connect to the SQL server using a browser like SQL Server Management Studio (include in SQL Server Express)
            • Drill down to {serverName}\Databases\{databaseName}\{tableName}
              • Where {tableName} is your user table
            • Right-click a table and click Select Top 1000 Rows
  • Release Builds only, also check:
    • Version Numbers: Right-click the executable, select Properties, and then select Details
    • Digital Certificates: Right-click the executable, select Properties, and then select Digital Certificates

Read more on Installation Testing

Smoke Test

Verify that the current build is worth the effort of a full test. Sometimes a build breaks so bad that you cannot perform simple tasks. These should be done first in each environment and requires co-ordination in large teams. A build could be broken in one particular OS/Software combination and the fix will require a re-factoring. In this case you wouldn’t want to continue testing in the other environments as it will change soon.

Read more on Sanity Testing and Smoke Testing

Verify functionality

This will differ widely depending on the application. Hopefully there is already some documentation for you to start.

Testing from here is split into two categories:

  • Functional: Tests focus on the immediate task
    • Usually reserved for updating existing code
    • Example: A new login page was made
      • Test the login functionality
  • Regression: A full run of all tests
    • Usually reserved for major versions, release builds, and when code modifies a shared resource.
    • Example: A new login page was made using new session management
      • Most of your tests require an active session

Read more on Regression Testing

Exploratory Testing

When are given new code area to cover, you should try:

  1. Happy Path Testing
  2. Negative Testing
  3. Fuzz Testing
  4. Security Testing
  5. Conformance Testing (if applicable)
  6. Accessibility Testing (if applicable)
  7. Language localization (if applicable)

In all cases you should also:

Other Testing

A/B Testing: Usually done by the marketing team. QA will have tested all pages and the mechanism diverts users to one page or the other.

Acceptance Testing: The stakeholder will do some light testing after QA is finished to ensure the final product satisfactorily meets the requirements.

Performance Testing: Tests the application under various loads. Usually reserved for test automation.

Concurrent Testing

Concurrent Testing is the process of running automated tests while manual testers do exploratory testing. Any company that uses test automation, will have some sort of concurrent testing. No one should rely solely on automated tests.

Read more on Concurrent Testing

Further Reading

Software Testing

  • Wikipedia parent article of most of the links in this article

Udacity CS258

  • Software Testing Methodologies Class Online