Test Strategy On Ongoing Projects: Building Airplanes While They Fly with Matthew Heusser

Test strategy needs to satisfy a variety of stakeholders:

• Delivery management wants a stable, predictable process that “just works.”
• The project manager wants to know when you’ll be done.
• Senior management might want to reduce time-to-release, time-to-identify, or time-to-fix (in production).
• The poor tester just wants to know what to do today.

Teams have some sort of test activity, based on their previous projects or methodologies. Over time, new test and review activities appear to catch defects that slipped through. This creates a testing process with numerous steps. Testing becomes expensive, slow, and ill-defined, all in the name of “process improvement.”

This talk invites you to take a step back and look at the whole picture. We start by breaking testing into primitive components — that is, what are the essential components of testing, what problems do they resolve, and do you have those problems. This enables you to identify gaps and redundant components in your test strategy, allowing you to build a strategy that comprises only the necessary pieces.

Some of the pieces of strategy we’ll discuss:

• Map features and determine how well they are tested.
• Identify potential risks.
• Compile historical lists of defects.
• Leverage tooling and tooling approach to accelerate work.
• Implement effective reporting and decision-making processes.

Once you’ve examined your process, identified the customers to that process and their competing needs, the next step — the tough step — is to figure out what you want. The goal could be to find more bugs, find more important bugs, prevent bugs, have faster deploys, eliminate regression as a human delay, or at least radically decrease it. Some teams want to ensure consistency, repeatability, or predictability. Each of these goals will lead to different measures and different outcomes. We’ll discuss a little bit on how to drive to a single kind of improvement, then how to measure it.

Come with your testing challenges and goals. Leave with a half-dozen patterns and practices to consider tomorrow. Many of these can be implemented over a few lunch hours and can materially impact testing at your company.