software test plans

Start with a software test plan template, then throw it away

There’s nothing more intimidating than a blank sheet of paper. Writers know this to be true, but so do test managers. The easy way out is to pull out a template and to start filling in the various “recommended” sections and details. An even easier approach is to pull out a past test plan and to just start changing project names, diagrams, and technologies. However, these approaches miss the point.

Recently while writing a test plan for a new project, I’ve noticed an odd habit I’ve developed. Ten years ago, when I wrote a test plan I started with a template. Four years ago, if I wrote a test plan I started with a blank sheet of paper. I noticed that when I write a test plan today, I look at templates, decide not to use them, and then end up pulling in pieces of them anyway.

The planning process isn’t about producing a document. Okay, well it shouldn’t be about producing a document. I recognize that in some companies it is. Instead, it’s about thinking about the problem. Software development problems are difficult and solving these problems requires time spent in research, comparing options, and prototyping. Our planning process, in the early stages, is about exploring those options and elaborating on what we think we’ll need to do (and when we’ll need to do it).

I find templates keep me from thinking about the actual problem. Instead they get me thinking about formatting and populating sections that aren’t yet filled in. When I’m using a template, I’m thinking about polish - not content.

However, there’s value to templates. They’re useful checklists for what types of things you should think about. I forget stuff just like anyone else. I’ve gotten a lot of good ideas from templates. So I’ve developed a habit of using a template to “prime the pump.”

I take an initial look at my templates and past test plans and use that to help get me started on problem solving. I’ll then switch over to a blank sheet of paper and start typing out my ideas and thoughts about what we need to test and how we should test it. Later, when I feel I’ve got most of my content, I’ll go back to a template and start pasting the content into the appropriate sections.

This technique keeps me from focusing on polish at the wrong time. There’s nothing wrong with polish, I just don’t want to be thinking about what font to use when I should be thinking about how I’m going to find or generate test data. This technique keeps me free from distractions when what I really need to be doing is focusing on the problem. This helps me deal with some of the intimidation of the blank page, but also allows me to be focused on the difficult topics when that’s what needs to be done.

Using taxonomies to help with test planning

A year ago, I was working on a project where we were doing a failure modes and effects analysis (FMEA) related to failover and recovery. As I was thinking about how to best start my analysis, I recalled that in the past while doing performance testing work I looked at many of the same aspects of the system while planning. As a way to generate ideas, I did some research to identify sources that could help me with my planning. You can take a look at some of the resources I found, or use different taxonomies if you have any that you particularly favor.

Here’s an example of how you might use a resource like this. Let’s take the risks listed in chapter three of Performance Testing Guidance for Web Applications. In the following figure from the book, you’ll see a summary of the risks presented in that chapter.

Figure 1: Performance testing risks, from the book Performance Testing Guidance for Web Applications.

I prefer working with the list of questions the authors have outlined in the chapter, but the graphic does a nice job summarizing things. For each specific risk listed, you want to:

• Ask yourself if you’ve accounted for that risk with your current plan. If you haven’t, figure out if you should. If you think you should, figure out what type of testing would be most appropriate for you. One nice thing about this particular taxonomy is that they give you some guidance there.
• For each risk, move from the generic to the specific. The risk “Can the system be patched or updated without taking it down?” is a great question, and an initial answer might be “yes.” But when I look at the system I currently work with, there are several major systems all working together. I might ask if I can patch all of them. And patch them in what ways; via software, database, run-time dependencies, services, etc.?
• For each risk, ask yourself if there are any slight variations on that risk that might be important to you. Good examples of the practice are the two risks listed in the book: functionality could be compromised under heavy usage; and the application may not stay secure under heavy usage. And you can vary different parts of the same question. In those two risks, they varied the quality criteria — functionality and security — but kept the risks, such as heavy usage, static. You could add other quality criteria or other risks.

The general idea is that you’re using lists like these to help you generate test ideas. In a way, you’re also using them to test the planning work you’ve done so far to make sure you haven’t forgotten or overlooked anything.