I consult with a lot of teams that think they need expensive tools. Some teams do need the features offered by expensive test tools. But most teams I’ve worked with don’t, they just think they do. Likely they could have gotten by with a workflow tool like JIRA or Bugzilla and some simple scripting languages and/or open source testing tools. I don’t understand the fascination with expensive tools. And I say that as a former technical writer for IBM Rational, whose tools I — mostly — really like!
I’ve used most major vendors’ tools; and while they’re nice, I don’t typically have the problems they propose to solve. A lot of times, I’m only solving problems because the tool says I should. An example is detailed traceability or distributed test execution. I like all the LoadRunner extensions, and I use them when I need them; but I also like JMeter, SQL, and Ruby. I like distributing thousands of GUI-level automated tests across a test lab using Rational Functional Tester, but I rarely have projects that have thousands of GUI-level automated tests.
I’m working on a project now where we don’t have a performance test tool, and someone on the team continues to mention that we need to purchase LoadRunner. I’m almost positive that we can do the testing using straight SQL and Perl along with some run-of-the-mill monitoring tools; but for whatever reason LoadRunner keeps coming up. This isn’t unique to this project. In my experience this happens a lot.
Here are some keys for figuring out if you really need that tool and its associated price tag:
- The tool provides a solution that has been proven to work for your non-trivial problem. An example is LoadRunner’s SAP extension; you likely don’t want to solve that problem with Ruby scripts. Buy or rent the expensive tool. Trust me, it’s likely cheaper. Another example might be iTKO LISA. If you have a couple of SOAP web services to test, you want SoapUI. If you have an enterprise of web services to test, with a variety of architectures and protocols, you might need iTKO LISA.
- They integrate with your existing suite of tools in a way that’s relevant to your context. For example, I’ve worked in regulated environments where traceability was truly required; purchasing tools that facilitated that was relevant to our project. I’ve also worked on projects where traceability wasn’t required, but we used certain tools because it was possible. We didn’t do it mind you, but hey, it was possible. Sheesh!
- You can’t find a cheaper alternative and don’t want to build your own. Some teams don’t want to be toolsmiths and they just can’t find a cheaper alternative. There’s nothing wrong with that. Just make sure you really looked around for cheaper alternatives before you select the big expensive tool. Don’t let it be your default choice just because they have a large marketing budget. Also, don’t read that to mean you should always be cheap. I personally like JIRA more than I like Bugzilla. That’s because I think they have a good price-to-value ratio. For me, they are often worth the price. But not always – sometimes I use Bugzilla.