This topic may seem simple to many, but despite of hundreds of web articles – Smoke, Sanity, Retesting & Regression are the most misunderstood topics in Software Testing. There is enormous amount of literature on the subject, but most of them are confusing. The following article makes an attempt to address the confusion. Before understanding these terminologies, first & foremost you need to understand the concept of Software Build.
Table of Contents
Software Build
What do you think it takes to build a software? Yeah! The code. But it isn’t just one single code file, generally there are multiple source code files. Now these source code files need to be compiled & combined into a single executable file which can then be deployed by the release team. This process of combining multiple source code files into a single executable file is known as ‘Software Build’. As you might have guessed, before being delivered to the client a software undergoes multiple changes (defect fixes), i.e. multiple ‘Builds’!
Smoke Testing (General Health check-up)
What if developers are too reckless? There is a defect at the very first step – User is unable to login itself. Yeah! You will say what about Unit testing, but usually developers don’t follow the rules every time J Smoke testing is the preliminary testing to reveal simple failures severe enough to reject a prospective software release. In other terms you can call it as ‘Confidence testing’ or the ‘Build verification testing’. Smoke tests cover the MOST CRITICAL functionalities. The purpose is to reject a ‘critical defect’ build before the Test team starts detailed functional tests. Before starting the day, a daily build and smoke test is among industry best practices.
Note: The term “smoke test” refers to powering on a device simply to make sure it does not start smoking (indicating a major problem). It originated in the testing of electronic hardware.
Sanity Testing (Specialized Health check-up)
First of all irrespective of Testing, Sanity check is a basic concept – a simple check to see if the produced output is rational (that the product’s creator was thinking rationally, applying sanity). Extending the concept to software, after every change in a build Sanity testing is performed to ascertain that the particular changes are working as expected post which detailed tests are performed. If sanity test fails, the build is rejected to save the time and costs involved in a more rigorous testing.
Note: Sanity in general refers to the soundness, rationality and healthiness of the human mind. A person is not considered sane anymore just if he/she is irrational.
Retesting (Recovery Health check-up, post medicine)
This is the simplest to understand. What do you do in testing? Obviously find & log defects. After that? Yeah! Developer will fix the defect. As a Test team you need to verify that the defect fix is working fine, in other words you need to ‘retest’ the defect based on its steps to reproduce. Simple, right?
Regression Testing (No side-effects)
The job is not yet over J the code is developed >> Build is deployed >> Sanity is performed >> Full testing is done >> Defects are logged, fixed & retested. What else? Yeah! Truth is stranger than fiction, and so is the Software. You never know a change in one function (defect fix or enhancement or change request) can impact multiple areas of the software. It’s our duty as a Test team to ensure everything (apart from the change) impacted is working as expected. In other words, to ensure there are no new defects introduced. As you might have guessed, knowing the impact (Impact analysis) is a must-have to perform effective regression tests!
Key pointers
There is an inherent problem in the Testing Community. Not only do we call the same process by multiple names, but sometimes some of us also use the same name to call different processes. This is the reason for much of the confusion. But you must clear all your doubts before being a great tester!
- You might not have observed, but the sequence of these tests is the same as ordered in this article ;-) Usually regression testing is the last test cycle before signing-off the software.
- Both Smoke & Sanity tests are performed to verify critical functionalities & avoid any wasted effort (functional testing) in case of critical issues.
- Regression testing is where the ‘impact analysis’ comes in handy, to gauge the impacted areas due to any software change.
- Sanity & Regression testing can be performed either manually or using automation. Regression testing is the best-fit for automation tests using effective tools such a Selenium, HPE UFT, etc.
Different organizations & people have different understanding of Smoke & Sanity tests hence these are often used interchangeably. Regression tests are mostly neglected or the team relies completely on automation but if done diligently using the impact analysis, it is one of the most important test cycle.
