when relevant content is
added and updated.
when relevant content is
added and updated.
In an interview on 7/24/2018 I mentioned the work of DeMarco, Lister and Weinberg. Those books may seem dated. Then again, the choice is read them or re-discover the learning yourself, one mistake at a time. For the most part I was talking about software delivery, but those lessons occur in test as well.
Let’s talk about them.
It’s been nearly fifty years since Winston Royce published “Managing The Development of Large Software Systems.” The ripples from the the impact of that paper continue to resound. Today we’ll cover the implications of that paper, particularly on test, how that played out in how the work is done — and a few of the more modern ways of thinking about test.
The Waterfall Meets Test
The great benefit of the waterfall model is that it is easy … at least for management. At the very beginning, perhaps even before the requirements are created, someone picks the delivery date. Then the team creates requirements (the problem to solve), the specification (what the software will do), the design (how it will be implemented), the code, then we have a find/fix/retest loop or two. At the end of the project, out pops a software system.
You may have heard the old question “Why do we never have time to do it right but we always have time to do it over?” The waterfall model takes this joke to an extreme. After all, every step is a translation of the earlier steps, probably done by a different person. Specifications are just translating the requirements into engineering-speak. Because requirements, specs and code are documents, many books confuse the documents themselves with the activities. The “testing” became translating the language of the specification into the language of testing. In that world, the requirement of “The user shall be able to log in” becomes the test of “The user can log in by following step1, step2, step3…”
With the waterfall model, we don’t just build it twice.
We build it five, six, or seven times.
Under this world view, the most popular document, the one that tells us how to test is the “test case.” The testing phase then consists of running all the test cases until they pass. Test Case Management Systems work sort of like a library. Testers can “check out” a test case and “mark” it as passed for any given test run.
The duplication of effort here is … well, there is a lot of duplicate effort. Four to seven levels of duplication, sometimes more.
It’s easy enough to beat up on the waterfall and test cases. I’m trying something different here — to glean lessons from them that explain why and how we should perform modern testing. The first one is just how obvious this duplication is – and how to remove it.
A Secret About Test
The first secret about test cases is that you don’t need them — at all.
No, seriously. The typical trained software tester, one who actually knows the system and knows how to test, can take a well-described feature and decompose it into test ideas. Give the same feature to two different testers, say on two different major shake downs of the application, and the two runs will have just slightly different approaches – enough to increase test coverage on multiple shake-down tests.
The variation is good.
Sometimes test cases include setup information. For example, a test case for a complex insurance product might include how to set up medical and prescription coverage. In my experience that information is often useful to development, customer support, and marketing. Putting the information in the test case actually limits re-use. So while that information might be valuable, I’m not sure that a test case is the right place for it.
The final argument for test cases is that they tell a programmer how to automate the tests. Next time, let’s talk about that.
For now, it is enough to say that test cases were an attempt to document what would be tested, how it would be tested, and, perhaps, what actually was tested, in a naive sort of coverage. If each test case mapped to a requirement, we could say we had “requirements coverage.”
There are plenty of visions for modern testing that are compatible enough. If you want a reference, consider Alan Page and Brent Jensen’s model, and compare it to the waterfall and test case vision.
Page and Jensen talk about the delivery of value. Eliminating excessive documentation and duplication can help.