Published at: August 1st, 2020
Last updated: March 7th, 2021
By: Noa Segol
Test management tools are designed to assist companies in their efforts in managing their testing activities. With the abundance of testing tools available, managing all aspects of testing has become a growing challenge. Such as, the need to incorporate not only traditional, scripted testing, but also elements of exploratory testing, as well as automated testing.
With so many options to choose from, it can sometimes be difficult to choose the right tool, and decide what the relevant criteria to base your decision on are.
Before making a tool selection, we recommend that you follow our step by step guide on how to evaluate software testing tools:
- Evaluate your current situation- There is no one size fits all in life nor in the test management tool selection process. The first step should be identification of your current situation, how you are currently working. Versus, the areas where you’d like to improve using the newly selected tool.
- Define your requirements from the tool itself - The right tool should be the one that addresses your needs. This may be different from the needs of other organizations that are larger/smaller. Or organizations that have different regulation requirements, work with a different approach and have other tools in its tech stack.
- Establish an evaluation team that will be in charge of the tool selection process.
- Start hunting possible testing platform- using articles like this one.
- Read public testing tool reviews. Make sure you pay attention to areas which other users have identified as less ideal for the tools you are evaluating.
In this article, we have reviewed the major elements that need to be considered when selecting a test management tool. We regard: integration capabilities, with Atlassian Jira and others, as well as its ability to function as a stand-alone full-cycle ALM solution. Data visualization capabilities- reporting and dashboards. Main features of each product, deployment type, price level as well as other softer elements such as the support, training, and certification offering.
PractiTest is an End-to-end (ALM) application lifecycle management solution covering all aspects of testing from requirements, to test execution, issues, and reporting. PractiTest has the ability to act as a stand-alone solution with end-to-end coverage starting from the requirement, test execution, issue tracking, reports, and dashboards. In addition, it is a central hub of information for all types of testing conducted within the organization: scripted, exploratory, or automated. Allowing you to make wiser data-driven decisions and release better products. Offers training and certifications.
- Integrations: Built-in Jira Integration with Jira cloud, Jira Server, Jira Data Center, and a no-plugin option for large scale organizations. In addition, the system also integrates with Pivotal Tracker, Rally, Azure DevOps, YouTrack.
- Automation support: There are several ways to integrate automation with PractiTest. This includes the use of REST APIs, Xbot to schedule and execute automation tests and Firecracker for automatic import of CI/CD results.
- Pricing: Price starts at $39/user per month for the professional level with a minimum of 3 testers per month.
- Deployment method: Secure Cloud only solution. Certified as SOC-II compliant
- Main features:
- Hierarchical filter tree for smart data management
- Reusability of tests between test sets
- Advanced reporting and dashboards that can be externalized to non-users
- All-types Jira integration plus integration with other leading bug tracking tools
- Main customers: Rakuten, AIA, NYTimes, Wendy’s
- All in one solution to orchestrate all testing types: scripted, exploratory and automation
- Smart data management capabilities that help organizations of large scale testing
- Reporting and dashboards are granular and provide great visibility
- Enables anytime anywhere access with a secure and compliant cloud solution
- Top-notch professional support and customer success team
- Only supports cloud deployment - no On-premise options available.
- User management capabilities still need progress
- Not a part of a large scale organization
This product evolved from a product originally developed by Mercury interactive as “Quality Center”. They were later purchased by HP, transformed its name to Application LifeCycle Management (ALM) and recently was acquired by Micro Focus.
- Integrations: No built-in Jira Integration, can be offered through a third party, smooth integration with the other Microfocus tools.
- Pricing: Expensive, but prices are not publicly available.
- Deployment method: On-premise.
- Main features:
- Real-time visibility
- Process and compliance enforcement
- Easily build organizational integrations based on business needs
- Ability to manage traditional and agile projects in parallel
- Main customers: The world bank, Lufthansa, Amdocs, Accenture, Airbus
- Enterprise oriented, syncs smoothly with other Micro Focus tools, such as UFT; Octane; Project portfolio Management; Service Manager and Loadrunner enterprise. Limited integration with other tools.
- Advanced reporting capabilities
- Bug reporting tool which is user-friendly and intuitive
- Legacy solution. Was designed with a waterfall approach in mind, not a fit for Agile/DevOps approaches- MicroFocus has a dedicated solution for DevOps-Octane.
- No cross-browser support, working mainly on Internet Explorer. (IE).
- Very high price tag, difficult to set up with limited support currently.
Now owned by SmartBear, Zephyr was originally developed as a Jira Add-on to enhance its capabilities to support proper testing management. However, for medium-size and above organizations, with complex testing, this approach proves to be insufficient. Hence, the enterprise edition was created to better address the needs of such organizations., in terms of testing requirements, as well as, compatibility with enterprise requirements. For example, (SSO) Single Sign-On, (MFA) Multi-Factor Authentication.
- Integrations: Jira integration, Confluence, as well as automation tools such as Jenkins and Selenium.
- Automation support: Have a build-in Zbot tool for executing automation tests
- Pricing: Prices for the enterprise edition are not publicly available.
- Deployment method: On-premise or cloud.
- Main features:
- Support for Agile testing
- Real-time dashboards and metrics
- Open architecture
- Main customers: Hyundai, Fujistu, eBay, Walt Disney
- Commonly used- have a customer base of - 18,00 due to the legacy as a Jira Add-on
- Ability to map requirement to test cases to issues inside Jira
- Scalability to support thousands of test cases
- High price tag
- Reports still have room for improvements
- Some customers report on performance issues with the system loading time and more
Originally developed by Gurock software in Germany, this tool is now owned by the Idera corporation. TestRail is a test management system and not an ALM application management solution as it does not have requirements and issue management modules. Recently, a new Enterprise edition was released, to support the needs of larger organizations, with more complex testing types.
- Integrations: Jira (Cloud, Server and Data Center), Redmine, Bugzilla, Axosoft,Lighthouse, Fogbugz.
- Automation support: Ranorex,TFS and GitHub integrations,robust API library
Starting from $34/users/month for the cloud version and $351/user/year for the server version.
Enterprise versions start at $60/user/month for the cloud version and $2900/year/up to 5 users for the server version.
- Deployment method: Started as a web-based application but currently support both cloud and On-prem options.
- Main customers: AutoDesk, Citrix, Airbus, Continental
- Main features
- Real-time results tracking
- Test case management including test planning
- Integration with a wide range of tools
- Productivity and communication enablement
- Relatively easy to use. Easy to set up and friendly user experience
- Very customizable- which means the tool varies significantly between teams
- Traditionally had very friendly price level, hence has a strong user base
- No support for non-QA personnel, which becomes a necessity while working in Agile approaches.
- No built in requirement and issue modules
- There is a large selection of integrations they connect to, but these are rated below average.
Orcanos is an Application Lifecycle Management (ALM) and Q QMS solution. They are based in Israel. It is focused on regulated industries such as Medical Devices, Pharma, and High Tech. As a high-end product, they do not offer a free trial.only a demo to learn more about the system.
- Integrations: This tool is integrated with other solutions by Orcanos but there is no out of the box integration with external bug trackers such as Jira.
- Automation support: Easily integrate into any test automation tool
- Pricing: Starts at $990/month for up to 10 users per month. This price comes with 2 days onboarding with the Orcanos team. No free trial option, only a demo.
- Deployment method: Cloud, hosted on AWS
- Main customers: Netafim, Mazor Robotics, Johnson& Johnson, Zimmer
- Main features:
- Simple: Easily plan, organize and execute test cases
- Continuity: Report bugs from test run
- Tracking: Powerful dashboards to track testing status
- Traceability: Link test cases to any work item, such as risk or requirement
- Generate documents: Generate test design and test results documents in a click
- Streamlined integration with other Orcanos tools including risk management
- End-to-end traceability including requirements
- Has documentation generating capabilities with electronic signature
- Relatively high price tag
- No free trial option to explore if the tool is a good fit for your team
- Better fit for hardware related industries
This product which was originally released by QASymphony, is currently under the name of Tricentis, following the companies announcement of a merger at the end of 2018. It has a strong presence in the medium and large scale organizations. Offers Academy and certification.
- Integrations: Jenkins, Selenium, Jira, Soap UI, VersionOne
- Automation support: integration with other automation tools based on qTest launch
- Pricing: not publicly available.
- Deployment method: both cloud and installed options.
- Main customers:
- Main features:
- 2-way Jira integration with various Jira options
- Exploratory testing support using qTest explorer
- Out of the box and custom reporting based on qTest insights
- Trusted by leading Fortune-500 global companies
- Advanced API for improved automation integration
- Ability to share tests between different projects
- Reporting- gathering useful data can be challenging
- Stability issues in regards to ongoing usage in the application
- High price tag
This product is a great starting point for small teams as it offers a free version of the system and access to external users. On the other hand, it has limited functionality in regards to reporting, and communication which is critical for larger teams so it is worth considering in cases where you expect to grow in the near future.
- Integrations: Jira, Redmine, YouTrack, Slack and Github Bamboo and gitlab.
- Pricing: Starting from free for small teams with very limited functionality, then going from $20/user/mo to a higher price tag that is not publicly advertised for the full version.
- Deployment method: cloud.
- Main customers: SeatGeek, Job Target, BlackSky.
- Main features:
- Free version for team of up to 3 users with 500 Mb attachments
- Ability to share information with external users
- Bug trackers integrations and API for automation
- Reports and dashboards are limited and there is no ability to add personalized dashboards
- Reusability of test steps between test cases is limited
- Some usability issues are reported by users
With so many test management tools available, it is important to make sure that you select the tool which is right for your team. Take into consideration that implementing a tool is time consuming. You want to make sure that the tool you select has the ability to support your current needs. As well as, the ability to scale up with you as your organization continues to succeed and grow.
Beyond the desired technical features and abilities, there are “softer” elements to a successful test management tool user experience. Such as: customer support quality and ongoing commitment from customer success teams. These important factors should not be overlooked as they are whatensure you leverage any tool to its maximum potential. This also allows your and your team to focus your efforts on what brings value to your company and not on troubleshooting supportive software issues.