FireCracker

There are a few ways to run automation with PractiTest.
You can read about them here

FireCracker is PractiTest’s automation integration tool. It takes XML report files from your CI/CD or automation framework and uploads them into PractiTest automatically.

With FireCracker, you can:

• Consolidate automated and manual testing in one platform.
• Use existing automated tests and upload their results to PractiTest without changing any of them, or writing a single line of code.
• Upload results after every build, including all relevant fields and parameters.
• See the full picture of your software development and deployment processes in PractiTest.

If you are integrating Cucumber, SpecFlow, or Behave BDD test results, please see the dedicated guide: FireCracker Integration for BDD Tests.

Table of Contents

• Before You Start
• JSON File Configuration
  • JSON Configuration File Template
  • Example 1 – Simple Json
  • JSON Fields Explanation
  • Example 2 – With System & Custom Fields
• Modifying the JSON File
• Run FireCracker
• Automate the Process
  • Jenkins Example (Linux)
• Advanced Functions
  • Example – JSON with Advanced Functions
  • Full Functions Table

Before You Start

Prepare the following before running FireCracker in your automation flow:

1. FireCracker JAR file – Download the latest Download FireCracker JAR
2. Install Java on your machine
3. XML results file(s) –  From your CI/CD tool or automation framework.
4. PractiTest API token – Personal API token enabled by an account admin, generated in your personal settings
5. Configuration JSON file – Defines how your XML results map to PractiTest fields and which project/Test Set to send results to. Please see examples below (you can also get help from PractiTest Support).

JSON File Configuration 

The configuration JSON file tells FireCracker:

• Which PractiTest project to send results to.
• How to map XML fields from your automation tool to fields in PractiTest. For example: if you have a field called ‘class_name’ in your automated tests, map it with a PractiTest test custom field. This way you will be able to see the equivalents of your automated tests, in PractiTest’s reports and dashboard graphs.
• Any transformations to apply to field values before they are sent to PractiTest.

Note: You don’t need to create the configuration JSON from scratch. Use one of the examples below as a starting point and modify it to match your project. If you need help, please contact our Support team.

JSON Configuration File Template

The following examples show ready-to-use JSON configurations for FireCracker. You can use them as templates and adjust the values and field mappings to fit your own PractiTest project and XML structure.

Example 1 – Simple JSON

{
  "additional-test-fields": {
    "custom-fields": {},
    "system-fields": {}
  },
  "additional-testset-fields": {
    "custom-fields": {},
    "system-fields": {}
  },
  "pt-instance-params": "",
  "pt-test-step-description": "?failure-message",
  "project-id": "add-your-project-id",
  "email": "add-your-email",
  "testset-name": "Firecracker Test Case",
  "test-case-as-pt-test-step": true,
  "multitestset": false,
  "match-step-by": "name",
  "pt-test-name": "?pt-test-name",
  "use-test-step": false,
  "pt-test-step-name": "?pt-test-step-name",
  "api-token": "add-your-api-key",
  "only-failed-steps": false
}

JSON Fields Explanation

Field	Description
additional-test-fields	Placeholder for mapping extra fields to tests (e.g., linking to PractiTest custom/system fields). Empty here.
additional-testset-fields	Placeholder for mapping extra fields to Test Sets.
pt-instance-params	This is required to import the instance parameters into PractiTest in BDD Tests. For scripted tests, leave blank.
pt-test-step-description	Input a name for the Test step description in PractiTest.
project-id	Project ID in PractiTest. This is the project the tests and test sets will be created in. Enter your project number here.
email	The PractiTest account email associated with the API token. Enter your email here.
testset-name	The Test Set name where uploaded results will be stored.
test-case-as-pt-test-step	Values: False/True  False: <testcase> in the XML will be mapped to Tests in PractiTest.  True: <testcase> in the XML will be mapped to steps in PractiTest.
multitestset	When true, results are split into multiple Test Sets based on the suite name, each <testsuite> in the XML will become a Test Set in PractiTest. If true, do not add a value in the testset-name row . When false, all results go into a single Test Set.
match-step-by	Determines how steps in XML are matched to PractiTest steps (by name or description).
pt-test-name	Maps the XML test name to the PractiTest Test Name field.
use-test-step	If true, FireCracker tries to match reported steps to existing steps in PractiTest. If false, it only uses steps reported in XML.
pt-test-step-name	Maps the XML step name to the PractiTest Test Step Name field.
api-token	Add your PractiTest personal or general API token.
only-failed-steps	When true, FireCracker only imports failed steps from the XML. When false, it imports all steps.

Example 2 – With System & Custom Fields

Sometimes your XML results include extra attributes you want to bring into PractiTest as custom or system fields. For example, the XML below contains classname, time, hostname, and failure information.

<testsuites id="" name="Regression Run" tests="6" failures="1" skipped="0" errors="0" time="7.246564">
 <testsuite name="pt_demo.spec.ts" timestamp="2024-04-15T12:11:47.697Z" hostname="chromium" tests="2" failures="1" skipped="0" time="7.569" errors="0">
   <testcase name="should navigate to the product page and find header" classname="pt_demo.spec.ts" time="4.236">
     <failure message="Header not found">Step 2 failed – expected header text not found</failure>
   </testcase>
   <testcase name="should have free trial button" classname="pt_demo.spec.ts" time="3.333"/>
   </testcase>
 </testsuite>


 <testsuite name="pt_demo.spec.ts" timestamp="2024-04-15T12:11:47.697Z" hostname="firefox" tests="2" failures="0" skipped="0" time="8.852" errors="0">
   <testcase name="should navigate to the product page and find header" classname="pt_demo.spec.ts" time="5.071"/>
   </testcase>
   <testcase name="should have free trial button" classname="pt_demo.spec.ts" time="3.781"/>
   </testcase>
 </testsuite>


 <testsuite name="pt_demo.spec.ts" timestamp="2024-04-15T12:11:47.697Z" hostname="webkit" tests="2" failures="0" skipped="0" time="6.454" errors="0">
   <testcase name="should navigate to the product page and find header" classname="pt_demo.spec.ts" time="3.576"/>
   </testcase>
   <testcase name="should have free trial button" classname="pt_demo.spec.ts" time="2.878"/>
   </testcase>
 </testsuite>
</testsuites>

To bring these XML values into PractiTest, the JSON file should contain the desired field attributes, as well as the field ID in PractiTest (can be taken from the system). The JSON below shows how those fields are mapped into PractiTest with the field IDs (—f-123456, —f-234567, etc.), and XML attributes.

{
  "additional-test-fields": {
    "custom-fields": {
      "---f-123456": "?time",
      "---f-234567": "?classname"
    },
    "system-fields": {}
  },
  "additional-testset-fields": {
    "custom-fields": {
      "---f-345678": "?hostname"
    },
    "system-fields": {}
  },
  "pt-instance-params": "",
  "pt-test-step-description": "?failure-message",
  "project-id": "add-your-project-id",
  "email": "add-your-email",
  "testset-name": "",
  "test-case-as-pt-test-step": true,
  "multitestset": true,
  "match-step-by": "name",
  "pt-test-name": "?pt-test-name",
  "use-test-step": false,
  "pt-test-step-name": "?pt-test-step-name",
  "api-token": "add-your-api-key",
  "only-failed-steps": false
}

Modifying the JSON File

To adjust the JSON file:

1. Open it in any text editor.
2. Update project-id, email, and api-token, to match your PractiTest setup.
3. Edit fields or use different field functions. See the Advanced Functions section below for examples of how to split, join, or otherwise transform values.

Run FireCracker

Create a folder that will store your XML files, the configuration JSON file you downloaded and the JAR file of FireCracker that can be downloaded from here in this structure:

Folder:

• xml_reports
  • xml1.xml
  • xml2.xml
  • …
• CONFIGURATION_FILE.json
• practitest-firecracker-standalone.jar

Note: The File & folder names can change based on your selected names.

Now you need to run the command below, in any Shell, in order to send results into PractiTest (Please make sure you specify the name of your relevant folder):

java -jar practitest-firecracker-standalone.jar --reports-path="xml_reports" --author-id=USER_ID --config-path="CONFIGURATION_FILE.json" create-and-populate-testset

Additional Notes:

• In case your account is located in EU environment, please add this parameter to the command:
  --api-uri=https://eu1-prod-api.practitest.app
• USER_ID should be a user who belongs to a PractiTest account. you can retrieve user IDs by sending this GET request.
• To update existing runs, the test names in PractiTest must match those in the XML.

Now you are able to see all the results in PractiTest!

Automate the Process

To make this uploading process automatic, you will need to write a script that will download new XML report files into the ‘xml_reports’ folder, remove old files and run the command. This way every time you get a new build and result files, they will automatically be uploaded into PractiTest.

If you are using CircleCI, please use the FireCracker Orbs as described here

Jenkins Example (Linux)

In order to make sure the script will continue to update PractiTest via FireCracker, we are using the Pipeline plugin for Jenkins (that makes sure Jenkins continues to update PractiTest even when a test fails). We are also using the Parameters plugin for Jenkins to allow multiple projects in Jenkins to update PractiTest.
Minimum requirement for this example is Jenkins version 2.13.

1. Inside Jenkins go to Manage Jenkins –> Manage plugins –> sub tab Available –> look for Pipeline plugin and install.
2. For this example, we want multiple Jenkins projects to to activate the pipeline that update PractiTest. So, inside Jenkins go to Manage Jenkins –> Manage plugins –> sub tab Available –> look for Parameters plugin and install.
3. In Jenkins, press Create a new item – choose Pipeline. Name your pipeline.
   Paste the below script, inside your new pipeline under Script field, then press save. For now, leave the defaultValues empty, as we are going to define them using the parameters:
   
   

   
         pipeline {
            agent any
            parameters {
         	    string(name: 'TESTSET_NAME', defaultValue: '', description: 'testset name')
                string(name: 'PROJECT', defaultValue: '', description: 'jenkins project folder and name')
                string(name: 'RELATIVE_TEST_LOCATION', defaultValue: '', description: 'folder where test results where located in jenkins project')
                string(name: 'JAR_VERSION', defaultValue: '', description: 'firecracker jar version')
                string(name: 'AUTHOR_ID', defaultValue: '', description: 'author id')
                string(name: 'API_URI', defaultValue: '', description: 'api uri')
                string(name: 'CONFIG_FILE', defaultValue: '', description: 'firecracker config file content- downloaded from Firacracker')
                booleanParam(name: 'RUN_BUILD', defaultValue: true, description: 'run build or don\'t for parameterized call')
            }
            stages {
                stage('run project') {
                    steps{
                        script{
                            if ("${params.RUN_BUILD}"  == "true") {
                                build "${params.PROJECT}"
                            }
                        }
                    }
                }
            }
            post {
                always {
                    script{
                        if (!fileExists("practitest-firecracker-standalone.jar")) {
                            sh "wget https://github.com/PractiTest/practitest-firecracker/releases/download/v${params.JAR_VERSION}/practitest-firecracker-standalone.jar"
                        }
                        if (!fileExists("surefire-reports")) {
                            sh 'mkdir surefire-reports'
                        }
                        if (fileExists("surefire-reports/*")) {
                            sh 'rm surefire-reports/*'
                        }
                    }
                    sh "echo \'${params.CONFIG_FILE}\' > config.json"
                    sh "cp -a ../${params.PROJECT}/${params.RELATIVE_TEST_LOCATION}/. surefire-reports"
                    sh "java -jar practitest-firecracker-standalone.jar --api-uri=${params.API_URI} --reports-path=surefire-reports --config-path=config.json --testset-name=${params.TESTSET_NAME} --author-id=${params.AUTHOR_ID} create-and-populate-testset"
                }
            }
         }
         

   
   
4. Since we want to see multiple projects in Jenkins updating PractiTest, go to the project you want to build and trigger the Pipeline:
   1. Press configure.
   2. Scroll to Post-build actions. On the bottom left side you will see a button “Add post-build action”. Press the arrow next to it and choose “Trigger parameterized build on other projects.
   3. Fill the “Projects to build” field with the name of the Pipeline you created.
   4. For the “Trigger when build is” field, choose “Complete (always trigger)” – or something else that better suits your process.
   5. Press Add parameters –> Predefined Parameters. A Predefined parameter window will open. Inside this window, define your parameters using the text below. Add the relevant values for each parameter according to the explenation underneath the text box: TESTSET_NAME=   PROJECT=   RELATIVE_TEST_LOCATION=   JAR_VERSION=   AUTHOR_ID=   API_URI=   CONFIG_FILE=   RUN_BUILD=false  
      Please add the relevant value as follows:
      • TESTSET_NAME= If you use an existing name – the FireCracker will update an existing testSet with a new run. If you give the test set a new name – the firecracker will create a new test set in PractiTest with the name you gave it.
      • PROJECT=The project name is the name of Jenkins project we are currently in and want to run.
      • RELATIVE_TEST_LOCATION = is the location of the test report XMLs that can be found under the Post build actions, Publish Junit test result report. This can be configured using the ‘Add post build action’ button
      • JAR_VERSION= is the version of the Jar file we downloaded from the help page. Can be found here
      • AUTHOR_ID= fill in the user id of the person running the firecracker.
      • API URI= for the US users: https://api.www.practitest.com, and for the EU users: https://eu1-prod-api.practitest.app
      • CONFIG_FILE= paste the content of the json config file downloaded from the FireCracker.
      • RUN_BUILD= Always keep false in order to prevent Pipeline from building the current project again and again in a loop
   6. Press Save.

From now on, each time this project will build, it will automatically update Pipeline which will update FireCracker and will send the build results into PractiTest automatically.

Advanced Functions

If you need more control on your XML output, you can use several advanced functions in the JSON file such as splitting values, trimming text, or extracting only part of a string.

Example – JSON with Advanced Functions

Here’s an example configuration showing how to use a few of the advanced functions inside your JSON mappings

{
  "additional-test-fields": {
    "custom-fields": {},
    "system-fields": {}
  },
  "additional-testset-fields": {
    "custom-fields": {},
    "system-fields": {}
  },
  "pt-instance-params": "(trim (join (drop-last 1 (get 2 (split \"outline example :\" ?name)))))",
  "pt-test-step-description": "(join (drop-last 4 (get 3 (split \"`\" ?failure-detail))))",
  "project-id": "add-your-project-id",
  "email": "add-your-email",
  "test-case-as-pt-test-step": false,
  "multitestset": true,
  "match-step-by": "description",
  "pt-test-name": "(trim (join (drop-last 2 (get 1 (split \"outline example : \" ?name))))))",
  "use-test-step": true,
  "pt-test-step-name": "",
  "api-token": "add-your-api-key",
  "only-failed-steps": true

JSON Functions Explanation

1. pt-instance-params →  (trim (join (drop-last 1 (get 2 (split “outline example :” ?name))))) – Splits the test name by “outline example :”, takes the 3rd element, removes the last character, rejoins it, and trims spaces.
2. pt-test-step-description → (join (drop-last 4 (get 3 (split “” ?failure-detail))))` – Splits the failure detail by backticks, takes the 4th element, removes the last 4 characters, rejoins it.
3. pt-test-name → (trim (join (drop-last 2 (get 1 (split “outline example : ” ?name)))))) – Splits the test name by “outline example :”, takes the 2nd element, removes the last 2 characters, rejoins, and trims spaces.

Full Functions Table

Text (?field)	Function	Result
alpha beta gamma	(split ” ” ?field)	[“alpha”,”beta”,”gamma”]
com.example.my.app	(tokenize-package ?field)	[“com”,”example”,”my”,”app”]
MyXMLParser	(tokenize-class-name ?field)	[“My”,”XML”,”Parser”]
red,green,blue,black	(get 2 (split “,” ?field))	blue
one two three four six	(take 3 (split ” ” ?field))	[“one”,”two”,”three”]
one two three four five	(drop 2 (split ” ” ?field))	[“three”,”four”,”five”]
foo.bar.baz.qux	(drop-last [2] (tokenize-package ?field))	[“foo”,”bar”]
a-b-c	(concat (split “-” ?field))	abc
john doe	(capitalize ?field)	JOHN DOE
a b c	(join (split ” ” ?field))	abc
hello world	(trim ?field)	hello world
raw-value-123	?field	raw-value-123
ABCDEFGH	(join (drop-last 2 (drop 4 ?field)))	EF