Test Sets & Runs
Organizing your testing work into Test Sets
- Tests that focus on a certain part of the system, such as GUI or Database
- Tests that belong to a certain task, such as Regression or Sanity
- A set of tests that need to be run by a single tester during a day or calendar week.
Adding Test Instances to Test Sets
In order to add a Test Instance, press the Add Test Instance button on the left side of your Test Set window. This will display the list of Filters and Tests in the Test Library. Simply choose the tests you want to add to your Test Set using the check-boxes, then click “add selected tests”.
In addition, you can create a new test directly from a Test Set by pressing New Test. After you create the new test and press ‘Save Changes’, the new test will be automatically added to the Test Set as a Test Instance.
You can also add test instances to an existing Test Set or create a Test Set based on chosen tests directly from the Test Library. Read more about it here.
Test Sets are limited to have up to 2K test instances per set.
Users who are not project admins, will be able to add up to 100 instances at a time.
Displaying Test Library or Last Run fields in the Test Instance Grid
Many times you want to access information you defined as fields in your Test Library within the Test Instance grid (e.g. the Platforms for which the test is relevant, tested component, test level, etc). Or, you will like to see the information from Runs related to the instances (e.g. Build number of your last run). This is easily done in PractiTest.
Start by pressing the “Columns” button on the top of the Test Instance grid. In the window that pops up you will see a list of all available fields to display. Each field has a toggle next to it - Choose the fields you want to display or remove fields from the display by pressing the toggle. You can also change the order of the fields by dragging and dropping them in the columns window.
Notice that part of these fields are displayed as “Test:<Field-Name>”, or 'LastRun: Field-name'. These are the fields from the Test Library or from runs that you can display in the Test Instance grid.
Running your Tests
Now you are ready to execute, or run, your test.
Each test-instance can be run multiple times. You can view the results of previous runs by clicking on an instance’s name in the grid. The status displayed on the Test Set grid will be the status of the last run of each instance.
- To run a test, click the “Run” link at the left of each instance in the grid.
- During your run you should go over each step of your Test and verify that the desired action can be executed. If a step is successful, click the “pass” link to pass the step.
- In case this action cannot be executed or if there is a bug in your application, you should set the corresponding step to FAIL. By pressing the Fail & Issue link you will be able to automatically report a defect based on the steps already executed in your run, as well as to link the newly reported issue to the run.
- You can also link between a step and an existing issue by typing its ID (number) on the box and pressing the “link” button.
- Note: when one of the steps in the run is marked as Fail then the Run Status will automatically be failed as well. However, if you mark a step N/A (Non Applicable) it will allow you complete the test run without failing the whole run.
- If you already have previous runs of the Instance, you can click on the Instance name to see the past Runs results including Run Date, Run Status, Tester information, and Run level custom fields.
Viewing step information from the Test Set level
As part of the Test Instance grid you can find a bar that graphically displays the status of your steps within the last test run of each of you instances. Notice that when hovering your mouse over the bar it will also show you in numbers how many steps are in each status.
Your test set displays a pie chart showing how many test instances have passed, failed, blocked, etc.
You can view this same information but in the Steps level (as an aggregation of all the steps in the test instances last run of your Test Set) by clicking the arrows next to the test pie chart.
You can also create Graphs for Steps Status as part of your PractiTest Dashboard. Step Status Pie Charts are available when you select to create dashboard graphs for Instances.
Comments, Attachments and History