Uncharted Waters

Nov 7 2017   4:49PM GMT

Are Unit Tests Wasteful?

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

Tags:
Lean
Testing
Unit testing
waste

Software development fads come and go every few years, and each time a new trend comes in something is demonized as waste. This time around, it is unit tests.

I was recently pointed toward a summary article explaining why most unit testing is waste. This text mentioned the common reasons for not testing — only tests derived from business requirements have value, too many tests slow development, tests have cost in terms of run time and maintenance, and the idea that tests don’t improve quality. Some of these points seem reasonable, and maybe obvious, but I don’t think they mean unit testing is waste.

An acquaintance on Twitter brought up an analogy in vaccination that I want to talk about briefly.

A few thousand people died each year from influenza before the1930s. The first flu vaccine was developed in 1938 and shortly after the flu mortality rate started decreasing significantly. Today, very few people die from the flu. It is mostly the very young and the very old that are in danger. Getting vaccinated yearly is the cause of the decline in deaths, and there is a large amount of data to support that claim. Despite this, there are people that don’t get vaccinated for various reasons — they claim to get sick from the vaccination, or to get sick despite being vaccinated, or maybe they are just scared of needles. When people stop getting vaccinated, someone, usually in at-risk categories, gets sick.

Unit testing works in a similar way in that if you do your unit tests, you are likely to have fewer bugs. We don’t have data, start talking about what is and isn’t a bug and things get iffy, but we do have lots of anecdotes.

unit testing

One company I worked with a while ago was developing a large Java-based monolith. We were in the wild west phases of trying to get product to market very quickly. You can translate that sentence to the developers were under pressure to release software and cut out some maybe-useful practices like unit testing, and management was cool with that choice. We went to production, and customers started calling in with problems. We started working on hot fixes and internally went through cycles of finding problems, getting a new build that was dead on arrival, sending the build back, and then trying again.

After some managers came and some left, we had a new policy enforcing coding practices. Minimum unit test coverage on each code repository was part of that policy. The unit tests of course aren’t what made the software improve. A conscious effort by our development staff to question what they were building, ask how it might fail, and build some alarms to know when it failed was the magic. All of that slowed down development, and created a maintenance burden. But, it also helped us get quality back to a reasonable level that the customer would be satisfied with. I’m sure those of you that have been in software for 5 years or so have a similar story. Maybe not about unit testing, but about something that looks like a waste of time but is really a crucial part of software development.

Waste is heuristic. Sometimes, we see some aspect of a development process (unit testing, code reviews, scrum, or whatever), don’t immediately understand why it is there or what it does, and dismiss it as stupid and wasteful. When those things get cutout, software gets worse. Be very careful when making blanket statements about what is bad for someones development practice.

1  Comment on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • StephaneTkoelbli
    You'll be able to get rid of unit tests , the day humans will be reliable and have a good level of logic! 
    Not sure this will happen on earth!
    There too many occurrences of dreadful situations where teams are unable to find "root cause" of a bug/problem. 
    Unit tests are there for such purpose: understand if results are conformant to specs.
    Of course nowadays where coding is likely to be a long list of calls to libraries where you do not have the source code or means to test thoroughly, I understand that some, under management pressure, may reduce such tests but I do not think it is a "safe bet".
    Asses the risks and costs of bypassing some test phases before doing so!
    40 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: