Uncharted Waters

Aug 25 2014   9:56AM GMT

ISO 29119: What you should know about software testing standards

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

ISO 29119 is a 5 part standard for software testing process and practice that was published in 2013. Standards are commonly used in other professions such as law and medicine to protect the consumers of these services. Medicine and law have existed for quite a while now and have had time to mature to the point where they have mostly accepted bodies of knowledge. Software is a very young profession, and software testing younger still and very much in flux.

stop 29119

Some problems this standard creates

When I first began in software testing, I worked for a company that required all test documentation to be developed according to the format defined by IEEE 829. This company was not in a regulated space. The idea was that we would end up with uniform documentation across different test groups in the business. And, that was exactly the outcome. We also ended up with a mass of documentation that did not accurately portray what testing happened or how it happened. This documentation was not often referenced but took increasing amounts of time to maintain. The result was a slower business doing less work to conform to a standard that did not support the ultimate goal, producing good software.

One interesting thing to note, is that if you search for IEEE 829 on google, after wikipedia, many of the top hits are for consulting firms selling services to help you conform to the standard.

Standards like this do more to disrupt the development of a field than they do to ensure good work.

Software testing has been stuck in adolescence for quite a while now. Partly because of standards like this which try to freeze how we do our work when we are just discovering how to be effective.

Fellow ITKE blogger, Matt Heusser, signed the petition and added the following note:

The AA battery is standardized on current and size — and interface standard. It was the /lack/ of a content standard that allowed the battery to improve over the past 30 years – from standard to heavy-duty to alkaline, all the while getting longer-lasting, less likely to leak, able to withstand greater temperatures, longer storable, and so on. ISO 29119 is a content standard; it will inhibit the progress of testing. Not only that, there is no consensus within the field on this and the methods advocated haven’t been tried – that is, the standard does not emerge from practice. For comparison, consider IEEE 829, and how that turned out.

ISO29119 is a content and process standard. If enforced, it will control and restrict not only the documentation of work, which is what IEEE 829 focused on, but also how the work is done. Any new discoveries and developments in the field of software testing could potentially be deemed illegal without expressed consent from ISO.

The standard for creating standards

The American National Standards Institute (ANSI) created a standard to guide the formation of standards in professional organizations. Some of the attributes of the process to create a standard according to ANSI are transparency, openness, impartiality, and consensus. All of these were lacking in the creation of 29119, especially industry consensus. A search on twitter for the #stop29119 hashtag will quickly show how little consensus there really is.

ISO 29119 should not be considered as a software testing standard according to ANSI because the formation did not seek to involve all parties with stake in the standards creation. The creation of this standard was monopolized by a group that stands to make a significant amount of money by selling consulting and certification services to organizations who may or may not be legally obligated to follow the standard.

What you can do right now

If you are interested in in preventing this standard from becoming standard, please sign this petition.

One of the main goals of this petition is to show opposition to the standard from a significant amount of active practitioners. This would most likely not cause ISO to withdraw the standard. The purpose this petition serves is to express that there is no industry wide consensus on this standard that supposedly represents the best and only way to do good software testing work.

There is disagreement and controversy about how we work and what it means to do good work for a software tester.

3  Comments on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • RobinGoldsmith
    Most of the opposition to ISO 29119 seems to be coming from people who have not read the standard itself and thus have no way to know what it does and does not say.  Some appear to be blindly following guidance from their own chosen arbiters of what’s good and bad in testing, i.e., folks who were not involved in developing the standard but similarly make money consulting and writing/speaking/training about their presumably-different ideas on testing.  Awareness of and participation in standards working groups indeed are issues that have been around at least 40 years, as have folks who use standards in ways that interfere with achieving value.  Both need to be address, but a more immediate concern is making standards available without the current prohibitive expense.  Let’s fix that and then limit the 29119 petition-signing to those who’ve actually read the standard.
    745 pointsBadges:
  • Matt Heusser
    " Both need to be address, but a more immediate concern is making standards available without the current prohibitive expense. "

    I'd like to agree, Robin, but I suspect that having the standards /not/ be available is part of the buisness model --

    4,960 pointsBadges:
  • Matt Heusser
    Oh, and I did pay the $313 (ON SALE!) to get access to the first four documents. The whole thing is a charade.

    And I say that as arguably the most active writer/editor in the software test community.
    4,960 pointsBadges:

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: