How to Create a Software Test Strategy? Complete Guide With Examples

There is a lot of confusion around the idea of a test strategy. Some see a test strategy as a smaller version of a test plan, others may see a test strategy as just an undocumented direction for testing something.

The reality is that a test strategy is perhaps one of the most common test planning documents you can create, but it is not the same as a test plan. However, a test strategy is an ideal basis for a test plan and can help prevent planning the wrong test.

Strategic vs. Tactical

First, let’s examine the nature of a strategy in a general sense.

As an example, you might have a financial strategy for retirement, investing, sending kids to college, and so forth. The strategy might indicate the goals, ways to invest and general ideas for getting to the goals. But the exact investment vehicles, how much to invest, when to invest and so forth are all tactical.

Another common strategic example is that of a military strategy. The military strategy defines the main objectives and general approach to dealing with a conflict. The tactical side involves how many people are needed to fight the battle, the types of weapons, the timing of the battles, and so forth.

So, the strategic view defines the major objectives and approach, while the tactical view defines the details of what is needed to meet the goals and objectives.

If the test plan is the “big picture”, then the test strategy is the frame around the big picture.

What is a Test Strategy?

A workable definition of a test strategy can be found in ISO Standard 29119-1: “Part of the Test Plan that describes the approach to testing for a specific test project or test sub-process or sub-processes.”

It is interesting that ISO 29119 takes the view that the test strategy is part of the test plan. Yes, the test strategy can be a section in the test plan, but it can also be developed well before the test plan is created to ensure the test plan is in alignment with the major test objectives.

There are two notes to this definition in ISO 29119-1:

• “Note 1 to entry: The test strategy is a distinct entity from the Organizational Test Strategy.” This means that there may be an over-arching test strategy in place at the organizational level that addresses how the organization sees testing. For example, an organization may take a very risk-based perspective where all testing is prioritized based on risk analysis.
• “Note 2 to entry: The test strategy usually describes some or all of the following: the test practices used; the test sub-processes to be implemented; the retesting and regression testing to be employed; the test design techniques and corresponding test completion criteria to be used; test data; test environment and testing tool requirements; and expectations for test deliverables.” The key in Note 2 is that these items may or may not be part of your test strategy, and if they are, they should be at a high level. For example, in describing the test environment, it could be as simple as stating that a cloud-based test environment will be used. To go into very much further detail would be duplicating what should be in the test plan.

What is a Test Approach?

Another term that is often mentioned along with “test strategy” is of “test approach”. Some people use these terms interchangeably, which is not totally incorrect. In fact, that is stated in the ISO 29119 definition of a test strategy.

However, others may use the term “test approach” in a very broad sense, such as using an automated test approach where the goal is to automate as many tests as possible, or a confirmatory test approach where the goal is not to find new defects but to confirm recent changes work as designed.

The term “test approach” is not defined in ISO 29119-1, but is found in the ISTQB glossary: “The implementation of the test strategy for a specific project.”

The implication is that a test strategy is at the organizational level and the test approach is at the project level to tailor the test strategy to a specific project context. This may be the case when a test strategy is a very general statement at the organizational level. (Figure 1)

Figure 1 – Organizational and Project Levels

However, it is very important to understand that the above is just the ISTQB view. It is common to develop test strategies at the project level.

Why is a Test Strategy Needed?

It is important to understand why bother with documenting a test strategy at all.

However, it takes a relatively short amount of time to document a test strategy – perhaps 2 hours or less. And, the return on investment can be huge.

Here are some compelling reasons to create a test strategy:

• You can create a test strategy very early in a project – even before requirements have been defined.
• Defining a test strategy is a great early project activity to gain stakeholder input and agreement about how testing will be conducted.
• The process of creating a test strategy will reveal areas that are not understood and require further investigation and analysis.
• The test strategy can be the basis for the test plan. Much of the information contained in a test strategy can be further elaborated in detail for a test plan.
• Creating a test strategy helps assure that the test you plan will be the right test. It’s a bad thing to spend a lot of time in test planning, only to discover that key objectives have been missed or misunderstood.

What is Contained in a Test Strategy?

A test strategy is a very customized and tailored document. For organizations that create test strategies, each organization has its own needs and format.

With that understanding, here are some topics I often address in a test strategy at a high-level:

• Major test objectives
• Scope of testing
• Type of application
• Technical considerations (languages, environments, etc.)
• Major responsibilities
• Timeline
• Risks
• Critical Success Factors for the project
• Types of tools needed

Test Plan vs. Test Strategy

A test plan is like a project plan for testing. It describes scope, people, scheduling, environment design, methods, tools, risks and similar topics.

The test plan describes how the test will be conducted, while the test strategy describes why the test will be conducted, along with a major approach.

In Figure 2, we see several levels of test documentation. At the highest level is the test strategy. Below that is a test plan. Eventually, you can start to define test cases and test data. This is, in essence, top-down test design.

Figure 2 – Top-down Test Design

Critical Success Factors

One of the most important things to address in a test strategy are Critical Success Factors (CSFs). In other words, what must the project achieve to be considered successful.

In most cases, there are at least five or six CSFs. That’s a workable number. If you have more, then you probably won’t have enough time to test them all.

Some common CSFs are secure, efficient, correct, maintainable and reliable. A good reference for CSFs is ISO 25010 for software product quality. In that standard, you will find functional and non-functional software quality characteristics.

It is worth noting that each major attribute has sub-characteristics. In Figure 3, we see an example of three of these characteristics and sub-characteristics. For the complete set, please see the link above.

Figure 3 – Software Characteristics and Sub-characteristics

You may recognize these as non-functional attributes with the exception of correctness, which is functional. Each of these CSFs will require its own people, tools and processes to test. So, you must choose carefully.

It is possible to identify possible CSFs that are not included in the ISO 25010 standard. These may be important to your stakeholders, but not part of the “standard” list of attributes. An example would be to achieve or maintain customer trust.

Risk in the Test Strategy

There is a very close relationship between CSFs and risks. Risks can be seen as product and project in nature. Both are important to address in a test strategy.

A product risk is a potential failure in the product to be tested. For example, poor performance in a GPS application might cause people to miss turns.

A project risk is a potential failure in the project to deliver to product to be tested. For example, a contractor may not meet contractual obligations.

Once you identify the risks, the CSFs become apparent quickly. For example, if performance is a risk, then delivering a high performing product should be a CSF, and therefore, an important type of testing to perform. (Figure 4)

Figure 4 – Risks and CSFs

Both risks and CSFs are good basis for test objectives, which are documented in a test plan.

This is a great example of what happens when you don’t have test strategy. Some risks tend to be missed only to be considered at the end-stage of a product release. At that point, people have to rush to test that the risks have been mitigated.

Even worse, it might be learned that some risks can’t be adequately mitigated which can expose the organization to the larger risk of a failed project.

An Example of Test Case Design Based on the Test Strategy

Let’s say that correctness is one of your CSFs. In Figure 5, we see the effect of starting with correctness and applying it to a function, such as an ordering process.

Figure 5 – Identifying Test Cases Based on a Test Strategy

Now, we need to think about which aspects of ordering have the greatest need for correctness, such as tax computation, shipping rates, etc. That leads us to define different types of customers to test through the ordering process.

You would want to test domestic customers (perhaps even by state and locality), international customers (if that applies), new customers, existing customers, high-volume customers (frequent buyers with discounts), and so forth.

Taking this approach makes it a lot easier to define tests as opposed to just trying to come up with ideas as you may think of them.

Sample Test Strategy Documents

Please understand that these are merely samples to show the level of detail and types of information I typically include. You will want to create your own format based on your needs and context.

• Test Strategy One-Page Template Format (Sample) – This format is helpful to gather information that can then be included in an outlined document.
• Test Strategy One-Page Template Format (Blank Template)
• Test Strategy Annotated Template (Blank Outlined Template with explanations)

How to Create a Test Strategy

There are many ways to write a test strategy ranging from a solo effort to an interactive group activity. I favor the interactive group workshop because it is quick and can get wide stakeholder input if the right stakeholders attend. This kind of workshop can be easily conducted virtually.

The steps in performing the interactive test strategy group session are:

• Schedule a date, time, and place – Typically from 1 to 2 hours is best.
• Identify the key people to attend – This should include people from all project rules including end-users. Also, identify who will lead or facilitate the session, as well as who will document the discussion.
• Invite the key people.
• Send out any initial documents that might be helpful, such as project charters, acceptance criteria, concept of operations, etc.
• Conduct the workshop by brainstorming the items in the test strategy template.
• At the end of the workshop, summarize the strategy.
• After the strategy has been compiled in either one-page format or outline format, send a review copy to the participants.

If you can’t get people together even for one to two hours, then you may have to write the test strategy as a solo effort.

After writing the draft, it’s a good idea to send it to others for their review and input.

As a reviewer of many test strategies, there have been times when I had to call for a major re-write because it was obvious that the author did not understand what should be in a test strategy.

Helpful Tips in Creating a Test Strategy

• Don’t try to be perfect. You will make mistakes. You can keep working on something forever if you don’t declare a stopping point. I call this “polishing the rock.” Short and concise are fine. You can refine it as you learn more.
• Expect to learn more about the test strategy. In fact, this is why it is good to start thinking about a test in strategic terms first. You learn what you don’t know.
• Things will change, so be flexible to adjust the strategy.
• Any time you start to test a new technology, or one that is new to you, it is good to create a general test strategy just for how to test it. This has a huge benefit!
• Try to involve others, if possible. You don’t want a huge crowd, but it’s good to have a handful of knowledgeable people to help provide input and review.
• Allocate time in your testing estimates for creating both a test strategy and a test plan.

Frequently Asked Questions

Do I need to create both a test plan and a test strategy? Can’t just one of them suffice?

Like so many things in testing, “it depends”.

It depends on the scope and criticality of the test, the time available for creating test documentation and so forth.

However, it is important to remember that the test plan and test strategy are two different types of test planning assets, each with a different purpose. Trying to substitute one for the other often ends in communication gaps and omissions of critical items.

Is it really worth the time to create a test strategy?

Yes. Let’s consider the scenario where you invest five or six people-hours in creating a test strategy. Let’s also consider that testing is not a minor activity. It can often amount to 30% to 50% of a project. We just don’t see the total time expended because it is seldom measured.

So, that test strategy allows everyone to understand how one of the most critical and time-consuming tasks will be performed.

The test strategy also is the opportunity for collaboration very early on a project which has many benefits. The five or six hours will more than be recouped in time saved in less re-work.

Who should write the test strategy?

Similar to a test plan, the ideal person to do this is the test manager or test lead. Sometimes, they may choose to delegate the creation of the test strategy to another team member and then approve the test strategy.

One thing I have realized after teaching this for over 30 years is that test strategy creation requires experience and judgment. That’s why generals create military strategies. While any strategy may have flaws, experience helps to reduce those flaws.

Conclusion

Hopefully, this article has provided the information you need to create one of the most powerful forms of test planning – the test strategy.

A test strategy helps to ensure that everything else you do in testing an application is in alignment with the goals of the project. That alignment helps to avoid wasted time and effort by performing the right tests for the right reasons.

A test strategy is not a substitute for a test plan, but there may be times when just a test strategy may suffice in communication goals, risks, and responsibilities of a test. That decision rests on the nature of the project. The higher the scope, risk, and complexity, the greater is the need for good communication and planning, which a test plan can help achieve.