The Auto Create a Run API is a smart endpoint that helps you report automated test results to PractiTest without manually creating every test, instance, and run in advance.
When automated tests are executed outside PractiTest, for example in Playwright, Cypress, Selenium, Postman/Newman, Robot Framework, Ghost Inspector, Jenkins, GitHub Actions, GitLab CI, Azure DevOps, Bamboo, or other tools, this endpoint can handle the reporting flow in one API request.
Based on the request you send, PractiTest can find or create the relevant test, find or create the relevant instance in a specific Test Set, and create the automated run result. This allows your automation framework to stay focused on execution, while PractiTest centralizes the results, traceability, dashboards, reports, and release visibility.
- When to Use Auto Create a Run
- How It Works
- Prerequisites
- General Implementation Pattern
- How to Map Automated Tests to PractiTest Tests
- Recommended Naming Approach
- Reporting Passed and Failed Results
- Reporting Steps
- Optional Run Data
- Recommended First Implementation
- Best Practices
When to Use Auto Create a Run
Use this endpoint when you want to:
- Send automated test results from an external automation framework into PractiTest
- Report test-level pass/fail results
- Automatically create tests in PractiTest when they do not already exist
- Automatically create test instances in a specific Test Set
- Add execution output, duration, custom fields, steps, or attachments to automated runs
This is especially useful when your automation framework should remain focused on test execution, while PractiTest stores the results, traceability, dashboards, reports, and release visibility.
How It Works
When the Auto Create a Run endpoint receives a request, PractiTest performs the following actions:
- Looks for the relevant test
- Creates the test if needed
- Looks for an instance of that test in the selected Test Set
- Creates the instance if needed
- Creates the run result on that instance
PractiTest can identify the test in one of two ways:
- By test ID
- By the test name provided in test-attributes/name
If a test-id is provided, PractiTest uses the existing test.
If no test-id is provided, PractiTest looks for a test with the provided name and creates one if no matching test is found.
Prerequisites
Before creating your integration, make sure you have:
- A PractiTest project
- A PractiTest Test Set where automated results should be reported
- The Test Set system ID, not the display ID
- A PractiTest API token
- The PractiTest project ID
- A decision about whether to map by test-id or by test name
Note: Run permission is required. Additional permissions are required when the endpoint needs to create new tests or instances.
General Implementation Pattern
Most automation integrations follow the same pattern:
- Run the automated test
- Capture the result
- Convert the framework result into a PractiTest payload
- Send the payload to the Auto Create a Run endpoint
The integration logic can live in different places depending on the framework.
| Tool or Framework | Common Integration Point |
| Playwright | Custom reporter, usually onTestEnd |
| Cypress | Node event such as after:spec |
| Postman/Newman | JSON reporter output after collection run |
| Robot Framework | Listener or parsed output XML |
| Selenium | Test framework hook, such as after test or after method |
| Ghost Inspector | Webhook JSON or exported result report |
| CI/CD tools | Post-test reporting step in the pipeline |
The main idea is the same: your automation tool runs the tests, and a small reporting layer sends the result to PractiTest.
How to Map Automated Tests to PractiTest Tests
There are two common ways to connect your automated tests to PractiTest.
Option 1: Use an Existing PractiTest Test ID
Use this option when the tests already exist in PractiTest and you want full control over the mapping.
This is usually best when:
- Your team already manages test cases in PractiTest
- Each automated test should map to a specific existing PractiTest test
- You want to avoid creating new tests automatically
- You prefer explicit mapping over name-based matching
Example:
"data": {
"type": "instances",
"attributes": {
"set-id": 1234,
"test-id": 5678,
"exit-code": 0,
"run-duration": "00:00:12",
"automated-execution-output": "All assertions passed"
}
}
}In this example:
- set-id tells PractiTest which Test Set should contain the instance
- test-id tells PractiTest which existing test should be used
- exit-code: 0 reports the run as Passed
- automated-execution-output adds a short execution summary
Option 2: Use the Automated Test Name
Use this option when you want PractiTest to find or create the test based on the automated test name.
This is usually best when:
- You are starting a new automation integration
- You do not want to manually map every automated test to a PractiTest test ID
- Your automation framework has clear and stable test names
- You want PractiTest to create missing tests automatically
Example:
{
"data": {
"type": "instances",
"attributes": {
"set-id": 1234,
"exit-code": 1,
"run-duration": "00:00:08",
"automated-execution-output": "1 assertion failed: expected status 200, received 500"
},
"test-attributes": {
"name": "API Login - Invalid Password",
"test-type": "ApiTest",
"description": "Verifies that login fails when an invalid password is provided"
}
}
}
In this example, PractiTest looks for a test named API Login – Invalid Password. If it does not exist, PractiTest creates it using the details provided in test-attributes.
Recommended Naming Approach
When using the test name option, make sure the name is stable and unique. If multiple tests share the same name, PractiTest will use the most recently updated test.
Good examples:
- Login API – Invalid Password
- Checkout Flow – Apply Discount Code
- User Management – Create User – Missing Email
For frameworks that support nested test titles, you can build the PractiTest test name from the full title path.
For example, in Playwright:
Authentication > Login validation > rejects invalid password
This helps avoid duplicate tests and makes the test easier to understand in PractiTest.
Reporting Passed and Failed Results
PractiTest uses exit-code to determine the automated run status:
- 0 = Passed
- Any other value = Failed
You can also send run duration, execution output, custom fields, run steps, and file attachments.
If run steps are included, PractiTest calculates the run status according to the step statuses instead of the exit code.
Example failed run:
{
"data": {
"type": "instances",
"attributes": {
"set-id": 1234,
"exit-code": 1,
"run-duration": "00:00:21",
"automated-execution-output": "Failed on checkout confirmation assertion"
},
"test-attributes": {
"name": "Checkout Flow / Complete Purchase"
}
}
}Reporting Steps
You can report automated results at the test level or include run steps. For many automation integrations, test-level reporting is enough:
Automated test → PractiTest run
This keeps the integration simple and maintainable.
Use run steps when you want PractiTest to show more detailed execution information inside the run.
Example with steps:
{
"data": {
"type": "instances",
"attributes": {
"set-id": 1234
},
"test-attributes": {
"name": "Login Flow / Valid User Login"
},
"steps": {
"data": [
{
"name": "Open login page",
"expected-results": "Login page is displayed",
"actual-results": "Login page loaded successfully",
"status": "PASSED"
},
{
"name": "Submit valid credentials",
"expected-results": "User is redirected to dashboard",
"actual-results": "Dashboard was displayed",
"status": "PASSED"
}
]
}
}
}
Supported run step statuses:
- PASSED
- FAILED
- BLOCKED
- NO RUN
- N/A
Optional Run Data
Adding Execution Output
Use automated-execution-output for a short summary of the automated execution.
xamples:
- All assertions passed
- 3 assertions passed, 1 failed
- Failed on step: Verify dashboard title
- API response validation failed: expected 200, received 500
This field is designed for short output text in automated runs and supports up to 255 characters.
For longer logs, screenshots, videos, or full reports, use file attachments or store the external report URL in a custom field.
Adding Attachments
You can attach files to automated runs, such as:
- Failure screenshots
- HTML reports
- JSON result files
- Console logs
- Trace files
Files are sent as Base64 encoded content, please note that the maximum file size is 50 MB.
The API supports attachments on the automated run and also supports attaching files to specific steps.
This is especially useful for UI automation frameworks where screenshots, traces, or videos may be created when a test fails.
Recommended First Implementation
For a first version, we recommend keeping the integration simple:
- Report one PractiTest run per automated test
- Use exit-code for pass/fail
- Use run-duration for execution time
- Use automated-execution-output for a short summary
- Attach screenshots or logs only for failed tests
- Use stable test names, unless you already have PractiTest test IDs available
This creates a lean integration that is easy to maintain and can be expanded later.
Example first payload:
{
"data": {
"type": "instances",
"attributes": {
"set-id": 1234,
"exit-code": 0,
"run-duration": "00:00:15",
"automated-execution-output": "All assertions passed"
},
"test-attributes": {
"name": "Checkout Flow / Apply Discount Code",
"description": "Automated test reported from external automation framework"
}
}
}Best Practices
- Use test-id when you want strict control over which PractiTest test receives the result
- Use test-attributes/name when you want a faster setup and automatic test creation
- Keep automated test names stable, because name changes can create new tests instead of updating the intended existing test
- Use full title paths for frameworks with nested test names, such as Playwright describe blocks
- Send short summaries in automated-execution-output, and use attachments or links for full logs
- Start with test-level reporting before moving to step-level reporting
- Store API tokens securely using environment variables or CI/CD secrets
