Are you wasting your time using unstructured and unplanned testing?
There is a very big difference between informal testing and unstructured or unplanned testing. Informal testing is testing that is not completely anchored by low level planning of the steps you want to run, but it is very far from being unplanned or unstructured.
Actually if you ask most testing professionals, they will explain that their testing approach involves a balanced blend of formal and informal testing, mixed together in order to achieve optimal coverage of their application under tests.
Knowledgeable testers will also tell you that Informal testing alongside formal testing requires a plan that will make them work together like a couple of ice skaters performing a routine.
What is a test plan?
The Test Plan is the instructions manual for your testing process. It describes the intentions of your project (why are you testing?), together with the high and sometimes low level schedule of the activities you want to perform (how are you testing?).
Test plans can sometimes list the risks you foresee in the project (what are you testing for?), and they may also include things such as resources required (what are you going to be testing with?), and any other aspects that you may think are interesting or important to share about your testing approach.
Have an organized approach
So how do you create a test plan that includes both formal and informal testing activities into a testing schedule. Here is a simplified and generic example of how to do it:
#1. Review your requirements
Verify that the requirements for the application are clearly defined either by the customer or whoever took his place in the project. Look for completeness and correctness in them and list any issues you find during your review.
Informally play “Devil’s Advocate” and go though the requirements to make sure the all parts of the application are covered. Formally review the requirements and assure that the information provided is accurate and consistent.
#2 Analyze the Software Design
Verify that the software design covers all the necessary aspects of the application such as functional consistency, security, backwards compatibility, scalability, testability, etc.
An added objective here can also be to get a deeper understanding of the application, in order write the subsequent test plans.
Informally, schedule meetings with the software architects and development teams, to review both high and low level design documents for every part of the system.
Feel free to ask questions whenever something “does not sound right”, regardless if you have or lack the technical knowledge, many times it takes a pair of fresh eyes asking “naive” questions in order to find the big flaws in the software that were hiding under the noses of the technical experts…
#3 Run a cycle of Initial or Preliminary Testing
Pass on preliminary feedback to development regarding core elements of the application such as stability, usability, consistency and other aspects that could mandate significant re-coding work. Don’t wait to run all your tests and start by those that will give quick and important feedback to the development team!
Informally conduct exploratory testing of the application, in order to familiarize yourself with all the different parts of the application so you can quickly detect simple to spot bugs in the system.
Formally conduct an acceptance test cycle. This should cover any major functional and infrastructure parts of the application to verify that there aren’t any significant bugs that might prevent testing continuation before getting the whole team on the task.
#4 Provide Periodical Deliveries from the QA
Continuously send out feedback and commentary to the development team about the progress of the testing on the application, while providing information about any bugs found along the way, as well as relevant information regarding project constraints such as: timelines, regression status, priority etc.
Formally build acceptance tests, to ensure that the delivered application is solid enough to continue into the testing cycle. Then move over to the formal testing cycle that should cover all the application, both what was mentioned as new functionality in the requirements as well as all the regression functionality that should still work as expected by your users.
Informally conduct “bug hunts”, by assuming the role of the application’s user in order to catch on to bugs or scenarios not covered by formal testing. Make sure such informal testing is timed alongside the progression of the project to remain effective.
#5 Create User Acceptance Tests
Assure that the final result meets all the customer’s/ end user’s requirements based on their testing scenarios.
Formally test based on the pre-defined scenarios from the customer while Informally test customer scenarios when possible in the customer’s environment and with their duplicated data.
#6 Post Testing Analysis
Time for high level review of the development and QA process. Gather all feedback from any stakeholders in the project: development, QA, customers etc. in order to create a knowledge base to learn and improve from for following projects.
Formally, analyze statistics from throughout the QA process. Informally, distribute and collect questionnaires, or group debated about separate stages of the project to gather “take away” points for future reference.
While mistakenly disregarded by some, the right balance of informal testing techniques alongside formal testing protocols contributes the most QA coverage for any application development from design to release.
This article is based on a blog post that was published on our QAblog