Test Sets & Runs
Organizing your testing work into Test Sets
- Tests that focus on a certain part of the system, such as GUI or Database
- Tests that belong to a certain task, such as Regression or Sanity
- A set of tests that need to be run by a single tester during a day or calendar week.
Adding Test Instances to Test Sets
In order to add a Test Instance, press the Add Test Instance button on the right side of your Test Set window. This will display the list of Filters and Tests in the Test Library. Simply choose the tests you want to add to your Test Set using the check-boxes, then click “add selected tests”.
In addition, you can create a new test directly from a Test Set by pressing New Test. After you create the new test and press ‘Save Changes’, the new test will be automatically added to the Test Set as a Test Instance.
You can also add test instances to an existing Test Set or create a Test Set based on chosen tests directly from the Test Library. Read more about it here.
Test Sets are limited to have up to 2K test instances per set.
Users who are not project admins, will be able to add up to 100 instances at a time.
Displaying Test Library or Last Run fields in the Test Instance Grid
Many times you want to access information you defined as fields in your Test Library within the Test Instance grid (e.g. the Platforms for which the test is relevant, tested component, test level, etc). Or, you will like to see the information from Runs related to the instances (e.g. Build number of your last run). This is easily done in PractiTest.
Start by pressing the “Columns” link on top of the Test Instance grid. In the window that pops up you will see on the left side the fields been currently displayed in your Test Instance grid, and on the right column all the additional fields you can add to the grid.
Add or remove fields by dragging and dropping, or by pressing on the “+” and “-” signs.
Notice that part of these fields are displayed as “Test:<Field-Name>”, or 'LastRun: Field-name'. These are the fields from the Test Library or from runs that you can display in the Test Instance grid.
Running your Tests
Now you are ready to execute, or run, your test.
Each test-instance can be run multiple times. You can view the results of previous runs by clicking on an instance’s name in the grid. The status displayed on the Test Set grid will be the status of the last run of each instance.
- To run a test, click the “Run” link at the left of each instance in the grid.
- During your run you should go over each step of your Test and verify that the desired action can be executed. If a step is successful, click the “pass” link to pass the step.
- In case this action cannot be executed or if there is a bug in your application, you should set the corresponding step to FAIL. By pressing the Fail & Issue link you will be able to automatically report a defect based on the steps already executed in your run, as well as to link the newly reported issue to the run.
- You can also link between a step and an existing issue by typing its ID (number) on the box and pressing the “link” button.
- Note: when one of the steps in the run is marked as Fail then the Run Status will automatically be failed as well. However, if you mark a step N/A (Non Applicable) it will allow you complete the test run without failing the whole run.
- If you already have previous runs of the Instance, you can click on the Instance name to see the past Runs results including Run Date, Run Status, Tester information, and Run level custom fields.
Viewing step information from the Test Set level
Step Status bar
As part of the Test Instance grid you can find a bar that graphically displays the status of your steps within the last test run of each of you instances. Notice that when hovering your mouse over the bar it will also show you in numbers how many steps are in each status.
Steps’ Status Count Graph
In the same way that your test set will display a pie chart showing how many test have passed or failed or blocked, you can have this same information but in the Steps level (as an aggregation of all the steps in the tests last run of your Test Set).
You can launch this graph from the link underneath the test pie chart, this will open the Steps’ Status Count Graph in a pop up window.
You can also create Graphs for Steps Status as part of your PractiTest Dashboard. Step Status Pie Charts are available when you select to create dashboard graphs for Instances.