Test Sets & Runs

Once you created your tests in the Test Library, you can run these tests in the Test Sets & Runs module.

Test Sets allow you to re-use your tests as many times as you need, in order to make your process more efficient.

You can watch this video for more information.

Test Sets let you organize your tests in the same way that you organize your work into cycles or tasks.

A Test Set is a group of tests that belong to specific tasks, features, or have some other reason to be run together.

Good candidates for Test Sets can be, for example:

Test Sets are organized in PractiTest using Filters, in the same way that you organize other entities (e.g., Issues).

The heart of your Test Sets are the Test Instances you assign to them.

A Test Instance is a link that allows you to reuse the Tests you created in the Test Library. Instances allow you to reuse Tests by assigning them and running them under multiple Test Sets (or even multiple times to the same Test Set).

In order to add a Test Instance, press the Add Test Instance button on the top left side of the Test Instance grid. This will display the list of Filters and Tests in the Test Library. Simply choose the tests you want to add to your Test Set using the check-boxes, then click “add selected tests”.

In addition, you can create a new test directly from a Test Set by pressing New Test. After you create the new test and press ‘Save Changes’, the new test will be automatically added to the Test Set as a Test Instance.

You can also add test instances to an existing Test Set or create a Test Set based on chosen tests directly from the Test Library. Read more about it here.

Please Note:
Test Sets are limited to having up to 2K test instances per set.
Users who are not project admins will be able to add up to 100 instances at a time.

Many times you want to access the information you defined as fields in your Test Library within the Test Instance grid (e.g., the Platforms for which the test is relevant, tested component, test level, etc). Or, you will like to see the information from Runs related to the instances (e.g., Build number of your last run). This is easily done in PractiTest.

Start by pressing the “Columns” button on the top of the Test Instance grid. In the window that pops up, you will see a list of all available fields to display. Each field has a toggle next to it - Choose the fields you want to display or remove fields from the display by pressing the toggle. You can also change the order of the fields by dragging and dropping them in the columns window.

Notice that part of these fields are displayed as “Test:<Field-Name>”, or 'LastRun: Field-name'. These are the fields from the Test Library or from runs that you can display in the Test Instance grid.

Now you are ready to run your test.
Each test instance can be run multiple times. You can view the results of previous runs by clicking on an instance’s name in the grid. The status displayed on the Test Set grid will be the status of the last run of each instance.

1. To run a test, click the “Run” link at the left of each instance in the grid.

   

2. During your run, you should go over each step of your Test and verify that the desired action can be executed. If a step is successful, click the “pass” link to pass the step.
3. In case this action cannot be executed or if there is a bug in your application, you should set the corresponding step to FAIL. By pressing the Fail & Issue link you will be able to automatically report a defect based on the steps already executed in your run, as well as to link the newly reported issue to the run.
4. You can also link between a step and an existing issue by typing its ID (number) on the box and pressing the “link” button.
5. Note: when one of the steps in the run is marked as Fail then the Run Status will automatically be failed as well. However, if you mark a step N/A (Non Applicable) it will allow you to complete the test run without failing the whole run.
6. If you already have previous runs of the Instance, you can click on the Instance name to see the past Runs results including Run Date, Run Status, Tester information, and Run level custom fields.
   

Step Status bar
As part of the Test Instance grid, you can find a bar that graphically displays the status of your steps within the last test run of each of your instances. Notice that when hovering your mouse over the bar it will also show you in numbers how many steps are in each status.

Test instances and Steps’ Status Count Graphs
Your test set displays a pie chart showing how many test instances have passed, failed, blocked, etc.

You can view this same information but in the Steps level (as an aggregation of all the steps in the test instances last run of your Test Set) by clicking the arrows next to the test pie chart.

You can also create Graphs for Steps Status as part of your PractiTest Dashboard. Step Status Pie Charts are available when you select to create dashboard graphs for Instances.

The Test Set contains test instances to be run but is also an entity on its own. You can add comments and attachments to it using the Comments & Attachments tab (next to the Instances tab) or view its History in the History tab.