What is Exploratory Testing?
By definition, exploratory testing is a software testing style that emphasizes the personal freedom and responsibility of the individual tester, to continually optimize the quality of his/her work, by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities, that run in parallel throughout the project.
Exploratory testing relies on the tester’s experience and thought process, where the tester is involved in minimum planning and maximum execution, and that is why it is not heavily documented or scripted. But that is not to say that it cannot be documented on some level and become an integrated type of testing to provide excellent QA coverage.
Exploration of software is basically looking at a certain feature, functionality, change and the overall behavior from learning as well as from a critical standpoint. Exploratory Testing is a crucial aspect of software testing, which almost every tester performs knowingly or subconsciously. That to say, all testing is exploratory to some extent. Even when performing a test based on a written script, a good tester will do so with some degree of exploration.
Exploratory testing is also known as ad hoc testing, and was known solely by this term until the early 1990’s, when a group now called the “Context-Driven School” started using the term exploratory. Cem Kaner was the person that coined the term Exploratory Testing in his book “Testing Computer Software” and described it as:
“Simultaneous test design, test execution and learning with an emphasis on learning”.
As an experienced tester myself, I cannot stress enough upon the benefits of exploratory testing. Working in an agile team, we are facing the struggles of time-crunch and release-time pressure. Though we did have scripted tests and automated regressions too, my best defects were found during exploration!
Session Based Test Management
Some test managers or project managers are wary of exploratory tests because of the lack of ways to manage or gather evidence of work performed or achieved by testers. This is where Session Based Test Management comes in. Session Based Test Management aims to combine accountability and exploratory testing in order to provide rapid defect discovery, creative on-the-fly test design, management control and metrics reporting.
For a novice team, the best way to start with exploratory tests would be to set aside an uninterrupted time box for exploration with a certain objective in mind. Set the goal and have a block of 1-2 hours set aside for looking around that feature/ component/ functionality/ behavior or use case of the application, with the focus of finding defects. During this time, keep a note of your observations and report them at the end. This is called a Test Session and the pre-defined goal you set for your session is called a Test Charter. A test manager may then keep track of the number of test sessions allotted to a tester, their time spent and their observations of value which may include defects, questions on design or doubts about the usage of features.
For teams that are larger in size, more mature in their exploratory test skills, or for teams in need of more structure around their exploration, there are many tools that support exploratory testing by providing standardized ways of creating, assigning and fulfilling these test charters.
Have questions regarding Exploratory and Session-Based testing?
Check out this recent LIVE Q&A webinar we hosted with James Bach, Consulting Software Tester and Buccaneer-Scholar.
In this unique Webinar, Both James Bach and Joel Montvelisky discuss questions regarding Exploratory and Session-Based Testing, from Theory to Methodology.
Exploratory Testing Tools
Test management tools like PractiTest, that support exploratory and session based testing practices, can give your QA coverage that extra advantage, by enabling all various testing efforts managed under “one roof”.
Exploratory tests tools allow you to define Charters for your Exploratory Test Sessions, document your notes, defects and observations as annotations as you are running your tests, report bugs directly from your runs, and finally to review your exploratory test sessions with colleagues in order to gather feedback. They also help to time your session and keep a track of the history and logs of past runs of your test sessions. These exploratory testing tools’ features will help you create as well as manage exploratory test sessions, and also help easily combine them with scripted as well as automated test runs.
Exploratory Testing in PractiTest
The PractiTest exploratory testing module allows you to define test charters & guide points for your exploratory sessions and save those guidelines as test cases for future reusability. Additionally, the usage of custom fields can help perform the session for different environments.
When running the ET test case, you can document important points that arises in the annotation section as well as report new issues or links existing issues. Finally, you can create reports based on your sessions and later review them with your colleagues and your team lead.
A Test Set can include all types of tests at once: automated, manual and exploratory. This ability can help you combine all various test types and to improve QA coverage and traceability.
Exploratory Testing Examples
Most exploratory tests will start with a test charter. The test charter will usually indicate the goal of the session. The charter can be chosen either by the executing tester or by the test lead/test manager. Test charter may include some guiding points that will further guide the tester while conducting the session.
The following example, presents an exploratory test case that will serve the tester as a plan for his/her exploratory session.
As you can see in this example, the session’s plan includes a test charter and guide points for the tester to follow when performing the session, as well as the environment to be used for the session (Operating system and browser in this case).
When the test charter is established (and guide points in some cases), and the time frame for the session is set, the tester can start conducting the session. During the session, the tester will take notes for further reference and future test case design, as well as report the issues and defects he/she discovered. Following is an example from an exploratory session with tester’s notes and reported issues:
Adding Exploratory Testing for Optimum QA Coverage
It is no secret that software development is shifting more and more towards Agile and DevOps practices. These practices are pushing organizations that applying them to provide faster releases, without lowering their quality standards. This process has had some important effects on software testing practices as well.
The clearest and most visible of those effects is the obscureness of boundaries between developers and testers. From separate, and even at times opposing teams, to one joint force on a single team, with the common goal of releasing a quality product.
The increasing popularity of test automation on agile teams is also contributing to the blurriness between developer and tester roles. Testers are now expected to understand code better and on most cases to know how to write code, while developers are to have a role in testing their own code
The demands from QA teams are constantly getting more intense, and it is a struggle to keep up with the ever-changing testing world. Finding new testing strategies and follow “best practices” that can add extra value to the product’s quality is an unceasing effort.
Foreseeing this trend, PractiTest - a leading SaaS test management tool, has continuously made the effort to develop its platform to accommodate the various types of testing used in today’s software development market, with the understanding that there is no one solution for software quality. A combination, however, of different testing types and approaches can help meeting those quality goals and add this extra value.
So what should we mix into our agile testing strategy for maximum coverage?
Scripted testing is a step-by-step testing approach for testing a specific functionality. This approach confides on the testers’ skills and common sense. Even though it is considered to be a precise testing method, it does has some disadvantages when it comes to agile development. The main shortcoming is the time factor. Executing scripted tests can take a lot of the testing teams’ valuable time. Additionally, while the scripted tests method does offer great coverage for planned requirements, it does not offer coverage for the unexpected and the for the unplanned.
As said earlier, the automation testing method is continuously expanding its reach, and it is now almost mandatory to include automation in the testing process in order to keep up with market demands. The use for automated testing (scripting) has overwhelmingly increased over the past five years, and for good reason. The process of automating some of the formerly manually executed scripted tests has allowed QA teams to speed up the process and save some valuable time. Automated tests can also be scripted using one of the test automation tools out there, and the skill of writing such scripts has become much needed for QA professionals.
By automating the repetitive manual tests, testers can free up more time for conducting more creative exploratory and session based tests.
Manual Exploratory Testing and Session Based Testing
Exploratory testing is more intuitive based and requires testers to be professional and responsible to continually optimize their work. It allows testers to gain a deeper knowledge of the system, and to get the user’s perspective on its functionalities, combined with their inside business knowledge. By performing exploratory sessions, the test team can better identify weak spots and design new test cases for better coverage of those weaknesses. Additionally, testers are more likely to find new defects in those ET sessions than while running regression tests that are executed over and over again as they tend to lead to new directions.
By adding exploratory and session based testing to the testing process, the QA team can save a lot of valuable time. When performed by skilled and experienced testers, the focus of the session can be altered on the fly, directing the testing team to find vulnerabilities and bugs faster and to address them more efficiently.
How to combine Exploratory practices into your testing routine?
The two main ways for a test lead to manage exploratory tests are delegation and participation.
When delegating, the test lead will usually dictate a test charter, leaving the execution for a member (or few members) of his team, while the team reports back the results of the session. Participation of the lead in the sessions, means the test lead will have an active role in the conducted session. Participation will allow the test lead to shift the session’s strategy in real time and serve as a live example for what is expected from the testers while performing the sessions.
For smaller and more novice teams, the best way for blending exploratory testing in the mix will be dedicating a designated time slot for sessions, while keeping in mind a certain functionality, feature, component or behavior to focus on (session based tests).
More experienced and mature teams, and teams that are in need for a more consistent way for creating, assigning and fulfilling test charters can use a session based test management tool to ensure clearer results and better structure.
So, if you are looking to improve your testing practices, add Exploratory Tests to the mix of Scripted and Automated tests to push up your software quality! And if you are looking for better and easier management around exploration, exploratory test tools (like PractiTest) can help and support you further in your quality endeavors.