Adding Exploratory Testing for Optimum QA Coverage
Are you using Session based/ Exploratory Testing in your QA efforts?. Most test teams today are struggling to find better ways to handle their testing. With the advent of Agile in our software development processes, teams are perennially under pressure to provide faster releases without lowering their standards of quality. This, in turn, adds load on the in-house test teams to ensure finding more and crucial issues and to prevent defect leakage. For this reason, testers look at strategies and practices that can help them achieve their goals and add more value to the product’s quality.
In my opinion as a hands-on agile tester, there is no single silver bullet to quality, but a combination of different types and approaches to testing that can help us get closer to our quality goals. Test teams need to strategize and plan the usage of a combination of scripted tests, automated tests as well as exploratory tests for achieving an optimum coverage and best quality software.
When we look at the typical test approach, it begins with test scripting and designing tests as per software functionality. These are created using requirement analysis and test design techniques and also using common sense and skills by our skilled testers. These scripted tests form the starting point of testing a new feature, change or addition in the software.
In addition to running the scripted tests manually, testers also rely on automated tests. These tests are scripted using various test automation tools and test automation, i.e. ability to write these automated test scripts is, thus, a much-wanted skill nowadays for all test professionals. The ability to run some tests using automated scripts helps repeatability and saves a lot of time and effort on part of the test teams. But most importantly, by automating the drudgery away, it saves the tester from repeated manual laborious tests and frees up their time for more creative thinking and exploration around the application.
Exploration of software is basically looking at the feature/functionality/change and overall behavior from a learning as well as a critical standpoint. Exploratory Testing is a crucial aspect of software testing, which almost every tester performs knowingly or subconsciously.
Cem Kaner coined the term Exploratory Testing in his book “Testing Computer Software” and described it as:
“Simultaneous test design, test execution and learning with an emphasis on learning”
By definition, exploratory testing is a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project.
Exploratory testing relies on the tester’s experience and thought process, where the tester is involved in minimum planning and maximum execution, and that is why it is not heavily documented or scripted.
As an experienced tester myself, I cannot stress enough upon the benefits of exploratory testing enough. Working in an agile team we faced the struggles of time-crunch and release-time pressure. Though we did have scripted tests and automated regressions too, my best defects were found during exploration!
Session Based Test Management
Some test managers or project managers are wary of exploratory tests because of lack of ways to manage or gather evidence of work performed or achieved by testers. This is where Session Based Test Management comes in. Session Based Test Management aims to combine accountability and exploratory testing to provide rapid defect discovery, creative on-the-fly test design, management control and metrics reporting.
For a novice team, the best way to start with exploratory tests would be to set aside an uninterrupted time box for exploration with a certain objective in mind. Set the goal and have a block of 1-2 hours set aside to look around that feature/ component/ functionality/ behavior or use case of the application, with the focus of finding defects. During this time keep a note of your observations and report them at the end. This is called a Test Session and the pre-defined goal you set for your session is called a Test Charter. A test manager may then keep track of the number of test sessions allotted to a tester, their time spent and their observations of value which may include defects, questions on design or doubts about the usage of features.
Here is an example of a test charter created by a tester, who will later add his observations and findings into it when executing and share it with his team-
For teams that are larger in size, more mature in their exploratory test skills, or for teams in need of more structure around their exploration, there are many tools that support the exploratory testing by providing standardized ways of creating, assigning and fulfilling these test charters.
Exploratory Testing Tools
Test Management tools like PractiTest, that support exploratory and session based testing practices can give your QA coverage that extra advantage, by enabling all various testing efforts managed under “one roof”.
Exploratory tests tools allow you to define Charters for your Exploratory Test Sessions, document your notes, defects and observations as annotations as you are running your tests, report bugs directly from your runs, and finally to review your exploratory test sessions with colleagues in order to gather feedback. They also help time your session and keep a track of the history and logs of past runs of your test sessions.
These features of an automated exploratory test tool will help create as well as manage exploratory test sessions and also help easily combine them with scripted as well as automated test runs.
A Test Set can contain all types of tests at once: automated, manual and exploratory. This ability to combine various tests to improve QA coverage and traceability. This will be beneficial in analyzing the overall effectiveness of the complete test effort, gauging the time and effort spent on different types of testing and also gather useful metrics.
To know more about how this works, see this useful video:
So, if you are looking to improve your testing practices, add Exploratory Tests to the mix of Scripted and Automated tests to push up your software quality! And if you are looking for better and easier management around exploration, exploratory test tools (like PractiTest) can help and support you further in your quality endeavors.