Breaking Down The Key Differences: Smoke vs Sanity Testing

What is Smoke testing and Sanity testing?

Smoke testing in the practice of software development and software testing has become a commonly used technique for the early and continuous detection of software defects. There are many ways that smoke testing can be applied in software applications and systems.

Smoke testing, also known as build verification testing or confidence testing, is shorter than the Sanity test and includes only quick scenarios that point at major areas of the product. It is performed on initial builds before they are released for extensive testing. Smoke testing is one of the important functional testing types.

The main idea of this test is to probe these areas in search of smoke that will signal there is a problem hiding below the surface.

Figure 1 – Smoke Testing

On the other hand, sanity testing is a type of software test that is performed on a stable build after minor changes to code or functionality. The goal is to determine if the proposed functionality works as expected. Sanity testing is similar to regression testing in the fact that you choose specific scenarios that cover all the AUT or application area but shorter and based on the highest risk factors. It is executed at an end in the process of a software development Lifecycle.

Figure 2 – Sanity Testing

What is Software Testing Life Cycle (STLC)?

Figure 3 - Software Testing Lifecycle (STLC) Phases

The Software Testing Life Cycle (STLC) is a sequence of specific actions performed during the testing process to help certify the software product. A STLC helps teams increase consistency and efficiency, defines goals and expectations for every aspect of the project, ensures that each feature is tested and approved before adding additional features, and verifies that project requirements are met.

Phases of the STLC:

• Requirements phase
• Planning Phase
• Analysis phase
• Design Phase
• Implementation Phase
• Execution Phase
• Conclusion Phase
• Closure Phase

Pros of Smoke testing

• Smoke tests are very useful when you either don't have time or don't know where to start and need a quick test to show you the level of your product.
• The results of this test help decide if a build is stable enough to continue with further testing.
• It helps to find issues in the early phase of testing.
• It helps to find issues introduced in integration of modules.
• Improves the effectiveness of the QA team. Many of the problems could be discovered with a Smoke test, thus helping to save time and resources.
• Faster troubleshooting of new and regression bugs. This is due to the high coverage, shallow depth nature of Smoke testing suites.
• Very limited number of test cases is required to do the Smoke testing.
• Smoke testing is quite easy to perform as no special efforts of the testing team are required.
• Improves the overall quality of the system.

Cons of Smoke testing

• Smoke testing does not cover detailed testing.
• Sometimes even after Smoke testing the whole application, critical issues arise in integration and system testing.
• Smoke testing sometimes causes the wastage of time if the software build is not stable.
• This type of test is more suitable if you can automate; otherwise, a lot of time is spent manually executing the test cases.
• This testing is not equal to or a substitute for complete functional testing.

How to automate Smoke testing?

Smoke tests can be either manual or automated, but the best way to do a Smoke test is by using an automation tool and programming the smoke suite to run when a new build is created.

Automation testing is an efficient method that allows to automate most of the testing efforts and there are many tools available to perform this type of test.

One place where you can find all the tools you need to automate your smoke test is PractiTest. It is an end-to-end test management tool that gives you control of the entire testing process - from manual testing to automated testing and CI.

Designed for testers by testers, PractiTest can be customized to your team's ever-changing needs.

With fast professional and methodological support, you can make the most of your time and release products quickly and successfully to meet your user’s needs.

Some tools that can be used to perform the Smoke Automation test process are:

• Selenium
• Appium
• Jenkins (CI tool)
• Robotium
• Cucumber
• Calabash
• Test Complete
• Watir

Pros of Sanity testing

• It saves lots of time and effort because it is focused on one or few areas of functionality.
• It is used to verify that a small functionality of the application is still working fine after a minor change.
• Since no documentation is required, these tests can be carried out in a lesser time as compared to other formal tests.
• In case issues are found during Sanity testing, the build is rejected. This saves a lot of time and resources.
• It is the simplest way to assure the quality of the product before it is developed.
• Sanity testing is simple to understand and to carry out and very effective in detecting bugs.
• It can reduce the effort during Smoke and Regression testing.
• It helps to find related and missing objects.

Cons of Sanity testing

• Its scope is relatively narrow, compared to the other types of testing.
• Sanity testing focuses only on the commands and functions of the software.
• Most of the time Sanity testing is not at all scripted, which may be a problem for some testers.
• Sanity testing goes nowhere near the design structure level, so it’s very difficult for the developers to understand how to fix the issues found during the Sanity testing.

How to automate Sanity testing?

Sanity tests can be either manual or automated. But beyond that, Sanity tests are more often scripted and automated because they are derived from existing Regression tests. Automation saves time and provides faster results for the development team to take action on.

One place where you can find all the tools you need to automate your Sanity testing is PractiTest.

Important differences: Smoke vs Sanity testing

In the next table we show the main differences between these two types of tests:

Now let's see their similarities:

Which testing is performed first, Smoke or Sanity?

Figure 4 – Which is done first - Smoke or Sanity?"

Smoke tests are executed first, followed by Sanity testing and then based on time availability Regression testing is planned. As mentioned above, the Smoke test is done in a new build, then if it passes the Smoke testing, more tests and Regression or Sanity tests are done.