What determines the Run Status of your Tests and Test Sets?
Test Run Status
- By default, before any steps are run, the status of a Test Run is set to NO RUN.
- If there’s at least one FAILED step, then Test Status is Failed
- If there’s at least one step which is BLOCKED (no FAILED steps), then status is BLOCKED
- Once the Test Run has begun, as long as all the steps you have run up to now are either Passed or N/A, and if you still have steps that you need to run, the status will appear as NOT COMPLETED.
- Only if all steps of the test are defined as PASSED (or some can be also be defined as N/A and then will be disregarded) then the test status will be PASSED as well.
- If all steps are set to N/A, the status of the Test Run will appear as N/A as well.
The Test Run Status that appears in the Test Library reflects the status of the last time that test was run on any test set.
Sometimes it’s much easier to look at the code to understand how it works:
Test Set Run Status
The Run Status of a TestSet is calculated similarly to Tests Run Status:
- By default, the status of a TestSet is NO RUN.
- Once the TestSet has begun running but not finished, and as long as no Test Instances have been set to Blocked or Failed, the TestSet’s status will be NOT COMPLETE.
- If one of the Test Instances is BLOCKED (and there are no instances with status Failed) – the entire TestSet’s Run Status will be BLOCKED as well.
- If one of the Test Instances of a TestSet is FAILED it overrides any previous status (including BLOCKED) and the entire TestSet Status will be set to FAILED.
- Only if all Test Instances of the TestSet are PASSED (or some can be also be defined as N/A and then will be disregarded) then the TestSet status will be PASSED as well.
- If all Test Instances of the TestSet are marked as N/A, the TestSet status will appear as N/A as well.