PractiTest supports the execution of both Manual and Automated Tests, allowing customers to run tests from any commercial or home-grown tools.

There are a few ways to run automation with PractiTest.
You can read about them here

Firecracker allows you to take XML report files from any CI/CD tool and upload them into PractiTest easily and automatically. From now on, every time you have new build results, they will be sent to PractiTest, including all fields and parameters, and you will be able to see the results of your entire software development and deployment processes in one solution.

 

A few things to prepare before starting the Firecracker’s configuration:
  1. Create an API token. We recommend using a personal API token that should be enabled by account admin and then generated in personal settings.
  2. Create test custom fields that will match your fields in the test results you get from your CI/CD tool. For example: if you have a field called ‘class_name’ in your automated tests, map it with a PractiTest test custom field. This way you will be able to see the equivalents of your automated tests, in PractiTest’s reports and dashboard graphs.
  3. Make sure you have Java installed on your machine

To start the configuration, please access Firecracker here using your PractiTest credentials and follow the steps below:

Step 1 - Upload the XML result files

firecracker upload xml

Step 2 - Modify and download the configuration

Paste your API token. From the project dropdown list select the project you would like to send results to.

firecracker insert tokens

If you want to modify the results that will appear in PractiTest, use the function column. For Example: If the function is ‘join (take 3 ?field)’ the result that will appear in PractiTest are the first 3 letters of the value in the chosen field of the xml.

Click the ‘Download Configuration’ button. You will receive a JSON file.

firecracker configuration results

In case you will need to map more fields or to change the project you are sending results to, you can always upload the Configuration file to the Firecracker UI mode, modify it and download an updated version. No need to remap existing fields, only to update the information when needed.

firecracker update configuration

Step 3 - Check the configuration

Create a folder that will store your XML files, the configuration JSON file you downloaded and the JAR file of Firecracker that can be downloaded from here in this structure:

Folder:

  • xml_reports
    • xml1.xml
    • xml2.xml
    • ...
  • CONFIGURATION_FILE.json
  • practitest-firecracker-standalone.jar

Note: The File & folder names can change based on your selected names.

Now you need to run the command below, in any Shell, in order to send results into PractiTest:

*. Make sure you specify the name of your relevant folder.

java -jar practitest-firecracker-standalone.jar --reports-path="xml_reports" --testset-name="TEST_SET_NAME" --author-id=USER_ID --config-path="CONFIGURATION_FILE.json" \create-and-populate-testset

USER_ID should be a user who belongs to a PractiTest account. you can retrieve user IDs by sending this GET request.

In case you created a firecracker test manually and you want it to be updated with the new runs, please make sure the name you give it matches the name of the test in the xml file.

Now you are able to see all the results in PractiTest.

Step 4 - Automate the process

To make this uploading process automatic, you will need to write a script that will download new XML report files into the ‘xml_reports’ folder, remove old files and run the command. This way every time you get a new build and result files, they will automatically be uploaded into PractiTest.

Here is an example script for Jenkins:
In order to make sure the script will continue to update PractiTest via FireCracker, we are using the Pipeline plugin for Jenkins (that makes sure Jenkins continues to update PractiTest even when a test fails). We are also using the Parameters plugin for Jenkins to allow multiple projects in Jenkins to update PractiTest.
Minimum requirement for this example is Jenkins version 2.13.

  1. Inside Jenkins go to Manage Jenkins --> Manage plugins --> sub tab Available --> look for Pipeline plugin and install.
  2. For this example, we want multiple Jenkins projects to to activate the pipeline that update PractiTest. So, inside Jenkins go to Manage Jenkins --> Manage plugins --> sub tab Available --> look for Parameters plugin and install.
  3. In Jenkins, press Create a new item - choose Pipeline. Name your pipeline.
    Paste the below script, inside your new pipeline under Script field, then press save. For now, leave the defaultValues empty, as we are going to define them using the parameters:
    
          pipeline {
             agent any
             parameters {
          	    string(name: 'TESTSET_NAME', defaultValue: '', description: 'testset name')
                 string(name: 'PROJECT', defaultValue: '', description: 'jenkins project folder and name')
                 string(name: 'RELATIVE_TEST_LOCATION', defaultValue: '', description: 'folder where test results where located in jenkins project')
                 string(name: 'JAR_VERSION', defaultValue: '', description: 'firecracker jar version')
                 string(name: 'AUTHOR_ID', defaultValue: '', description: 'author id')
                 string(name: 'API_URI', defaultValue: '', description: 'api uri')
                 string(name: 'CONFIG_FILE', defaultValue: '', description: 'firecracker config file content- downloaded from Firacracker')
                 booleanParam(name: 'RUN_BUILD', defaultValue: true, description: 'run build or don\'t for parameterized call')
             }
             stages {
                 stage('run project') {
                     steps{
                         script{
                             if ("${params.RUN_BUILD}"  == "true") {
                                 build "${params.PROJECT}"
                             }
                         }
                     }
                 }
             }
             post {
                 always {
                     script{
                         if (!fileExists("practitest-firecracker-standalone.jar")) {
                             sh "wget https://github.com/PractiTest/practitest-firecracker/releases/download/v${params.JAR_VERSION}/practitest-firecracker-standalone.jar"
                         }
                         if (!fileExists("surefire-reports")) {
                             sh 'mkdir surefire-reports'
                         }
                         if (fileExists("surefire-reports/*")) {
                             sh 'rm surefire-reports/*'
                         }
                     }
                     sh "echo \'${params.CONFIG_FILE}\' > config.json"
                     sh "cp -a ../${params.PROJECT}/${params.RELATIVE_TEST_LOCATION}/. surefire-reports"
                     sh "java -jar practitest-firecracker-standalone.jar --api-uri=${params.API_URI} --reports-path=surefire-reports --config-path=config.json --testset-name=${params.TESTSET_NAME} --author-id=${params.AUTHOR_ID} create-and-populate-testset"
                 }
             }
          }
          
  4. Since we want to see multiple projects in Jenkins updating PractiTest, go to the project you want to build and trigger the Pipeline:
    1. Press configure.
    2. Scroll to Post-build actions. On the bottom left side you will see a button “Add post-build action”. Press the arrow next to it and choose “Trigger parameterized build on other projects.
    3. Fill the “Projects to build” field with the name of the Pipeline you created.
    4. For the “Trigger when build is” field, choose “Complete (always trigger)” - or something else that better suits your process.
    5. Press Add parameters --> Predefined Parameters. A Predefined parameter window will open. Inside this window, define your parameters using the text below. Add the relevant values for each parameter according to the explenation underneath the text box: Please add the relevant value as follows:
      • TESTSET_NAME= If you use an existing name - the FireCracker will update an existing testSet with a new run. If you give the test set a new name - the firecracker will create a new test set in PractiTest with the name you gave it.
      • PROJECT=The project name is the name of Jenkins project we are currently in and want to run.
      • RELATIVE_TEST_LOCATION = is the location of the test report XMLs that can be found under the Post build actions, Publish Junit test result report. This can be configured using the ‘Add post build action’ button
      • JAR_VERSION= is the version of the Jar file we downloaded from the help page. Can be found here
      • AUTHOR_ID= fill in the user id of the person running the firecracker.
      • API URI= for the US users: https://api.practitest.com, and for the EU users: https://eu1-prod-api.practitest.app
      • CONFIG_FILE= paste the content of the json config file downloaded from the FireCracker.
      • RUN_BUILD= Always keep false in order to prevent Pipeline from building the current project again and again in a loop
    6. Press Save.


From now on, each time this project will build, it will automatically update Pipeline which will update FireCracker and will send the build results into PractiTest automatically.