Does your team have automated tests?

• Do you run automation as part of your tests?
• How do schedule and execute your automated scripts?
• Do you generate reports out of your automation?  Do you do this together with your manual tests?

It’s great to see how test automation is becoming more common in today’s testing organizations.  Something that used to be mainly a buzzword in the industry, is becoming a helpful tool to many teams around the world.

Regardless if you use Selenium, QTP, or even home-brewed scripts, automation is more and more a part of everyday QA life.  Even though it is still far from being a Silver Bullet, as many tool vendors want us to believe, it allows to streamline our processes and to develop more stable products, helping us release high quality products faster than before.

The problem is that, in order to make proper use of automation you need to manage it and integrate it into your overall testing process. For example, you need to understand what part of your tests are automatic vs. manual; or sometimes you are asked to generate reports that will show your stakeholders the overall testing status of your product (regardless if the test is automated or not!).

What were we looking to solve?

After getting a number of requests from users to expand PractiTest’s functionality to manage (also) automated tests we started searching for the best possible solution.

Basically we wanted to come up with an approach that would allow users to:

1. Manage their tests and runs, whether manual or automated, in one place.
2. Increase the visibility of the automation efforts and teams, eliminating any communication or coordination issues.
3. To see the results of all my tests in one place, creating comprehensive reports.
4. Providing a solution that is simple to deploy and flexible enough to manage as many tools as possible.

Our solution – the xBot automation agent

So, after talking to our users and reviewing many possible approaches we chose to develop what we call the xBot automation agent, a small java utility that runs on every OS (Windows, Mac, Linux) and is able to execute scripts written on any tool or any language.

Users who want to run automated test simply map their automation scripts to tests in the Test Library, they then create a Test Set with their scripts, and schedule the time when they want their tests to start running.  When this time arrives the xBot gets from PractiTest the order to execute the scripts locally.  Finally after each test is run, the agent uploads the results to PractiTest as part of the test execution log.

Public release of the xBot agent

In the beginning of the year we started a private beta with a limited number of customers who had asked for this functionality.  Later on, in February, we released the public beta in order to get more feedback on the solution.

In May and June published a number of posts in our QAblog, linked-in and other forums, asking testers what are their biggest challenges when coordinating automated and manual tests.

Now, after taking the feedback from the Beta and the inputs from the web, we are proud to release this new solution to support automated testing via PractiTest.

We hope you’ll like our solution and invite you to keep providing us your feedback.  Here’s more information about the xBot and PractiTest’s support for Automated Tests.

Agile Development and Agile Testing

Today, more and more teams are shifting over to AGILE. One of the interesting facts about agile development is that it comes in many variations, from Scrum to Extreme Programming and more; but regardless of the approach you follow there a number of principles in agile development that everyone agrees to:

• Customer satisfaction is achieved by frequent delivery of useful software
• Changes in requirements are part of the software development process
• Regular adaptation is needed to comply with the changing circumstances

When you work based on frequent deliveries, constant changes and regular adaptations; how can you still manage an efficient testing process?

Part of the philosophy behind agile development & agile testing talks about shifting towards automated testing as the means for covering regression testing & test driven development to achieve more stable code from the beginning. This “advise” is helpful but it still falls short from providing the solution and the guidance needed by a QA team to cope with the challenges of shifting to AGILE.

There are books that talk about this, and Joel wrote about agile testing in his blog. But in the end there is nothing like the experience from working with many organizations that shifted to AGILE with the help of PractiTest, and from the knowledge we’ve gain from our own experience developing PractiTest as an agile team.  The tools you use will help you achieve your testing and development goals in the same way that a hammer and the nails help the carpenter to make his furniture, and flexibility & adaptability is a must when you are looking for a tool to help you manage your agile process.

PractiTest Test Management Solution for Agile Development

A quick search on the Internet will show there are many solutions specifically designed to handle agile development, and some of them even claim to support agile testing, but we still see many customers who check them out and find they are missing important functionality needed to cover the testing process.

As a methodological test-management solution, from time to time we are asked to show how we support agile development and testing, and an interesting fact is that we support Agile Testing without having any specific feature developed solely for this purpose!

How do we do it? The answer is simple, we believe in flexibility. PractiTest enables YOU, the user, to customize the system based on your process, your product and your needs.

Defining the "sprint#" Custom Field

If you working based on sprints, for example, you can customize the system by creating a custom field called SPRINT and adding it to your requirements (or user stories), to your tests and to the issues in your project. Once you have this field in place you can organize all your data and work based on the sprints you defined. You can then create views and reports that will make it easy for everyone to gain visibility into their tasks and those of the team and allow the whole team to manager their work fast and easily.

What’s even best is that you can modify the values in the fields and even the fields themselves with a small number of “clicks” and in a matter of seconds (without the need to of complicated customizations or processes).

Another aspect common to Agile Testing is the popularity of Exploratory Testing among testers.  This approach works mainly by defining testing charters up front and documenting your testing steps at the same time you execute the tests.   In PractiTest this is easily achieved with the functionality available that lets you edit your tests steps even when you are running them within the Test Sets & Runs module.

There are many aspects that define Agile Development & Agile Testing, and the truth is that each organization and even each team will approach Agile in their own individual and different way (based on their needs and their constraints).  In this same way, we believe that it is not correct or even possible to try to define for you how you should manage your agile testing process.   The best approach, the one we believe in, is to give you the freedom to let you decide how to work and to support you process your way!

More information about custom views and custom fields.

Would you like to manage and run your tests from your iPad? with PractiTest you can!

What do you need to do? just login from your iPad (using Safari)

A picture is worth a thousand words, so here it is (scroll down for the video):

run your tests from iPad

and here’s the video:

Why PractiTest?

Sometimes potential customers ask us to help them understand whether PractiTest is the best solution for their needs.  We believe that having Joel working with them to help define their process and customize the platform to their needs is invaluable.

But there are a number of additional cool features that make PractiTest a unique solution.  We want to show you (only) five of them:

1. Anti-Bug Duplication mechanism -  Duplicates, bugs that are reported more than once in the system, are a pain for every organization.  The overhead and the waste of time of looking for the original bug, defining it as a duplicate, and rejecting the new one is something that we all would like to save from our daily lives.
   What if you knew the bug you we are writing right now is a duplicate of an existing bug?
   PractiTest comes with a built-in solution to the duplicate-bugs problem:  the minute anyone starts entering a bug the systems instantly displays the similar bugs candidates, enabling the user to decide whether this bug is a new one or not.
   
2. Bug by mail - With PractiTest you can report Issues directly from your email client. This is great if you have customers or field engineers (sales, support, etc) who don’t have access to PractiTest, or if you want to forward someone’s bug directly from your inbox to the system:
   
3. Unbelievable Customizations - not only can you customize the fields for your Requirements, Tests & Issues, but you can also implement your own user-permission levels and even define the workflow of your issues.
   Read more about PractiTest’s customization
4. Hierarchical custom views – This is one feature that, once you understand how it works and the value you get from it, you cannot stop using it.
   The basic idea is that you can organize your tests-tree based on any fields you choose, for example release version, but ALSO, based on product component and sub-component, and ALSO based on Run Status, etc.
   Unlike folders where the Tests are only shown in one place, with views you can show a single test in multiple different trees based on your needs.  This is similar to how gmail works with Tags, but BETTER!
   Read more about our hierarchal custom views.
5. Integration with SVN – This is a must for many development teams who choose to work in this very organized way.  For each issue you can see all the SVN commits linked to your issues, and also instruct SVN not to allow a commit without an issue being assigned to it.
   You can about our integration with SVN
   

Writing this post made me realize we have a lot more to write about, but maybe I’ll leave that to a latter post.

We just finished recording a new web-tour of PractiTest.

The Tour covers the Issues, Tests, Requirements and Reports Modules, as well as some of the platform’s customizations. We hope you’ll find it useful (click on the Play icon):

Feel free to send us your comments.