Uncharted Waters

Nov 19 2014   9:24AM GMT

NoTesting makes NoSense

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

It is not uncommon for people in technology to occasionally shoot a provocative idea out into the wild and then make a temporary show of it. David Heinemeier Hansson did this recently by claiming that TDD is dead, and Woody Zuill did the same with NoEstimates.

A lot of the time, the conversation softens a bit when you step away from the loud title and figure out what the person is really trying to say. That can be difficult to do.

Bob Marshall is now following suit with a twitter tag and blog post on something he is calling NoTesting. Daniel Kirmse wrote a post in a similar vein asserting that there are no agile testers.

His post outlines a couple of main needs in creating software such as customers wanting products that work, companies needing a positive reputation, and people on the development teams needing to feel competent.

0d24f7854461979825b0344daeb33149

The question being posed via the NoTesting hashtag is “Is testing the best way for a company to meet these needs?”

Lets take a closer look at the question, the premise, the rhetoric, and test this idea.

One important thing to note, is that no where in his blog questioning the necessity of software testing does Bob specify what he means by the term.

So what is testing?

Testing software among other things, is a process of discovering risks, things that will make the customer unhappy, that other parts of the development process are unable to find. Testing shines a spotlight on areas of the software that are hidden. Everyone on the project contributes to that mythical idea of quality. Some people discover what the customers needs and desires are, some people with management skills facilitate work, some people write code and build the product. Everyone has a significant effect on the outcome by contributing their expertise.

What about value?

In one part of his post, Bob makes the assertion that customers don’t actually want to pay for testing. That may be true, but we could also ask whether customers want to pay for product management, or scrum masters, or project management systems, or consultants. All of these things facilitate and support good, working software. All of these things contribute to what ends up being the product that solves problems and helps people do more work better.

Software testing is a cost center, but it certainly isn’t something that customers regret paying for. They usually aren’t directly paying for it anyway.

When people buy a product, they aren’t buying the specific services that went into making the thing. They are buying the end result, something that should just work. All of the stuff that went into making that product are embedded in the price.

Maybe you don’t need testers

There are some companies that are doing a lot of testing (or rather, checking) with code now. They employee no or very few testers. Facebook and Salesforce are some of the more notable examples. Both of these companies have fairly extensive suites of programmed tests acting as alarms for change detection and programmers that are testing their own work.

These companies don’t employ people with the title and specialization of software testing, but that doesn’t mean it isn’t happening to some degree. Often this formula ends in bad testing by people that don’t understand the work, and disappointed customers who just want something that ‘works’. There is also a trend occurring where companies that were previously using the ‘no testers’ model are slowly shifting back to use both people that are dedicated to testing and programmers to test the product.

If what Bob actually means with this post is that mass inspection doesn’t work, both Deming and Ohno would agree with that, and that it is important for developers to do their part by working to improve code quality, them I’m on board. Improving code quality may mean that fewer (but more skilled) testers are needed, it sure doesn’t mean they will go away.

Hopefully Bob has a follow up in the works to answer some of the many thoughtful questions that he has not addressed.

If I seem a little upset about this trend, it’s because I am. There is a sentiment in both of these posts that is sort of “what testers do is super easy, they should just show programmers the magic keystrokes and then go away”. This silly attack on the profession has been happening for a long time now. Thankfully, I get to work every day with developers that understand the value of good testing and am friends with people passionate about testing work.

Time will tell if Bob is right.

Take a guess which side of the argument I’m betting on.

UPDATE: Ben Yaroch let me know that SalesForce does have a number of testers in some divisions. It may be better to reference Twitter or Google as examples in the class of companies.

11  Comments on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • Veretax
    I'll ask this here, because I know the idea that Facebook/Salesforce maybe even Google its suggested they do all their testing with code and it is extensive.  I question that idea.  

    "They employee no or very few testers. Facebook and Salesforce are some of the more notable examples. Both of these companies have fairly extensive suites of programmed tests acting as alarms for change detection and programmers that are testing their own work."

    We know their testing 'experts' are limited or lacking (meaning people in such a role).  We know they use automation, they use TDD.  But is the TDD they practice really extensive? I think we do not really know that, especially since, I've seen arguments recently that TDD should only test just enough to drive the design, and not to try to overengineer or test too much.  (I've seen tests that assert nothing except if a certain exception is thrown, so in theory, that may be fine, but the test may not validate that anything else is working right, just that it throws an exception in X case.)  

    I think that teams are doing TDD and a lot more unit level checks with code is great.  That means finding issues early.  Yet there is always technical and testing debt, there will always be things missed, because we are human, even the best of us will miss something on occasion.  I think that's why you still need testing professionals, but their role and focus can change when you have the main line devs doing a lot of automation at the lower levels.
    8,675 pointsBadges:
    report
  • DanielKirmse
    Thanks for sharing your thoughts on this controversial topic. I will not comment on what Bob said or may have meant or not. But I would like to share my thoughts on what I've written in my post.

    First of all, what testers are doing day by day is by no means super easy nor easy nor something that we just could get rid of. At no point in my post I was claiming this. All what I was pointing out was that I would ask the question why there should be agile testers as a job title or a department. Why are there people whose job it is to clean up after the developers for they are not able to get the stuff right.

    As I pointed out, I would be happy about any agile tester with his or her special knowledge and expertise and unique character to join an agile team of user-story-to-product-feature-turners (to avoid developer or implementation here). I'm convinced that testers would add great value to these teams.

    What I would like to propose would be for them to mold in with these teams. But same goes for the former developers. They have to change their habits too. No more code and forget. No more let these QA guys take care of the mess. 

    Former developers have to learn how to test their stuff while doing it. This they could learn from testers in their teams. Testers could learn to code. Why not? More features could be done in higher quality. I want more testing to be done. Especially while coding the features. To me it makes no sense to introduce a tester to the team and keep in this micro world the division into dev and QA. I don't think this would make any sense.

    But as we all know there are aspects to testing that require a deeper understanding of how one could test a product. Well the tester still has that knowledge and could use it for the sake of the team. In the end he still has his unique expertise as other team members do have their expertise on one topic or the other. That's how teams are set up. Everyone should be a generalist to some extend in order to be able to take on important team tasks. And everyone will be a specialist in areas he feels drawn into. Fine with that. I just don't want this artificial division between development and testing. Because of the pure existence of QA developers behave the way they do far too often. Just throw their crap over the wall. 

    So the conclusion would be: There are no agile developers as well.

    So, please take the time and re-read my post. As an agile coach I put high emphasis on testing aspects in all trainings that I conduct for developers.

    And please feel invited to join this discussion. I really like to see where the disagreement is. I'm prepared to learn something. I might as well be wrong. Let's see
    30 pointsBadges:
    report
  • kinofrost
    Well said. I hear the echoes of my comments on that blog post in some of this, and I approve, especially concerning the definition of testing and the customer's say in what a company spends its money on.

    Here I'll go further and say that it's not possible to create software without testing. It's possible to create software with minimal testing, with bad testing, but not with "no testing". The concept makes no sense. If you respond to compile errors or run the program to see if it looks okay you've done some very simple testing.

    Once we've established that it's an intrinsic part of software development the arguments just appear to be rewordings of old arguments, and I've read enough Test Is Dead arguments to last me a lifetime.
    20 pointsBadges:
    report
  • Justin Rohrman
    @Daniel 

    Thank you for the note. I agree that testing is a role, a function assumed by someone, and doesn't necessarily have to be a job title. The role exists because there is need for people that specialize in the type of critical thinking and problem solving that can discover difficult problems in software. Problems that are not likely to be found otherwise.

    I also agree with you that there can be great value from developers and testers working very closely together as well as dev doing their part to make the job of finding problems far more difficult.

    I disagree with your point about everyone being a generalist. The division between the roles of development and testing are not artificial. The two roles are optimized for solving entirely different problems. People specializing and studying their role of choice are what helps move software development forward. 

    Some people can code and do a little testing when needed, or test and program a little when needed but I'm not sure that means we should aspire to do both.
    2,130 pointsBadges:
    report
  • dsynadinos
    Daniel - “As an agile coach…” Try applying your reasoning to your own title and job function. Would you reach the conclusion, “There are no agile coaches”, as well? And, ad infinitum, are there any jobs, at all? Or, is everyone doing everything?
    30 pointsBadges:
    report
  • DanielKirmse
    @ dsynadinos: I assume the job title "developer" as it has been understood in the past and currently is does not have to last as well as tester. I consider this distinction very artifical. The consequences of this separation are simple an excuse for developers not to think about good design, maintainability, supportability and high quality. Developers have got ther safty net named testers.

    @Justin Rohrman Testers do have to develop these special skills because developers perform so poorly on their main task. I observe this every day. Hard to find issues, hard to test issues, "sporadic" test failures and stuff alike which requires testers to have these special skills are a consequence of developers not doing their work well. 

    I've witnessed teams where both roles were mixed up in each and every single person. Tell you what, within 9 to 12 months we collect as much as 5 bug reports from our cutomers. Average in department would have been about 7 a week.

    About generalist vs. specialist: I do not propose anyone knowing a bit about anything nor do I propsoe everyone knowing everything about a very small topic. The truth lies inbetween. Its about balance. People are different. People have different affinities. Some are far more generalists than specialists and others the other way around. What I would like us to end up with would be this: Everyone should keep his specialties but everyone should have average knowledge on some other topics as well. I'd expect them to have an interest in and a good understanding of the product they build, about the requeirements customers have. 

    Combining people like this in an agile team leads to more flexibility in task assignments (bus factor) as well as better designs for everyone would be able to get a rough understanding of what is going on. At least this is what my practical experience showed me. As soon as this rule is broken teams start to deteriorate into a bunch of specialist which get things done more slowly in worse quality (own experience). 

    My point of view is that the roles of developer and tester as we lived it over the years could be dropped. We need an agile engineer that takes responsibility for his work well before the first keystroke and that sticks to it well after the shippment of the product.

    The introduction of a tester mitigates issues developers introduced I'd rather work at the root cause.

    To be very clear about one thing. Testers do have unique skills and an awfull lot of experience with the bad code shipped by developers. I'd rather have them in my team to build quality right from the start. Just as developers have to change their attitudes towards quality and have to learn new stuff testers should go where no tester has gone before :-)

    @dsyadinos: I would be proud of my work as an agile coach if I render myself superfluous one day because things have changed and there is no need to caoch anymore. I'd call this a celebration day. 
    30 pointsBadges:
    report
  • Counterfactuals
    @Daniel,
    In your blog post I don't really see an understanding of testing, as understood by people of the context driven school.  Given that I am not sure if it makes sense for you to comment on not having testers.  Do you think it would make sense for you to understand what good testers do and then explain that just so everyone has a common understanding?
    20 pointsBadges:
    report
  • Counterfactuals
    @Justin

    Is there a single agile developer who has shown an understanding of testing as practiced by context driven testers?  Can you point me to a blog post or book which demonstrates that understanding or let me know if you know someone who does?

    Isn't that a prerequisite to make a comment on testing?  I don't mean to say people should not comment.  I just don't understand why they refuse to learn and then comment whether it makes sense or not
    20 pointsBadges:
    report
  • DanielKirmse
    @Counterfactuals

    I'd love to learn what would be meant by the sentence: "Testers do have unique skills and do special things developers can't do". This one or a variant of it I heard quite often these days. Unfortunately no one ever really told me what exactly is meant by that. 

    So if you would volunteer to provide me with this information please go ahead. I'd appreciate it very much!

    I plan to have a follow-up on my blog post for I've had a couple of very useful discussion on that topic recently. What you offer would perfectly add to my knowledge.

    I've learned that the first blog post did leave out some important ideas I do have in mind and which I explained in those discussion I've had which led to an agrreement that my statement wasn't that wrong or silly as it has been received in the first place.

    So looking forward to what you share with me
    30 pointsBadges:
    report
  • kinofrost
    @Daniel

    I assume the job title "developer" as it has been understood in the past and currently is does not have to last as well as tester. I consider this distinction very artifical. The consequences of this separation are simple an excuse for developers not to think about good design, maintainability, supportability and high quality. Developers have got ther safty net named testers.

    Is it? Who says? I think that consideration of good design, maintainability, supportability and quality are signs of a good developer. I consider "developer" to incorporate these factors. A developer that sees a tester as a safety net is a poorer developer for it... and I don't see how changing the names to conflate the roles of people who feel strongly and passionately about their expertise in their own domain is going to add more value than it will take away.


    Hard to find issues, hard to test issues, "sporadic" test failures and stuff alike which requires testers to have these special skills are a consequence of developers not doing their work well. 

    Really? Can you trace these problems directly back to developers? In my own experience these problems are triggered by developers only because they're the ones to build it, but if a poor customer talked to a poor architect who talked to a poor foreman with poor communication and poor understanding would you blame a good builder for a house that doesn't meet the customer's needs?

    I've witnessed teams where both roles were mixed up in each and every single person. Tell you what, within 9 to 12 months we collect as much as 5 bug reports from our cutomers. Average in department would have been about 7 a week.

    1. Counting customer bug reports is not the same as having quality
    2. We can't ascribe good quality to a renaming of roles. If we're going to try to prove that changing what we call the team members results in better quality we need a more scientific approach with reproducible results, control groups and so on.


    Everyone should keep his specialties but everyone should have average knowledge on some other topics as well. I'd expect them to have an interest in and a good understanding of the product they build, about the requeirements customers have. 

    I still don't fully understand this. I also expect people building something to have an interest and working knowledge of the product and of customer requirements. Where does conflating the dev/test role come in? Also what are the "some other topics" that each role should have knowledge of? Do we mandate that people with no interest in the complexities of coding have a pragmatic understanding of them? Do we mandate that people with no interest in the complexities of the infinite open-ended processes of testing have a pragmatic understanding of them?

    Combining people like this in an agile team leads to more flexibility in task assignments (bus factor) as well as better designs for everyone would be able to get a rough understanding of what is going on. At least this is what my practical experience showed me. 

    Isn't that the fault of the agile system rather than role conflation?


    My point of view is that the roles of developer and tester as we lived it over the years could be dropped. We need an agile engineer that takes responsibility for his work well before the first keystroke and that sticks to it well after the shippment of the product.

    Why is the first statement connected to the second? I want the second part, and I don't see how the first part helps.

    The introduction of a tester mitigates issues developers introduced I'd rather work at the root cause.

    To be very clear about one thing. Testers do have unique skills and an awfull lot of experience with the bad code shipped by developers. I'd rather have them in my team to build quality right from the start. Just as developers have to change their attitudes towards quality and have to learn new stuff testers should go where no tester has gone before :-)

    Then get your testers working with the team early on. Give them a culture of mutual respect and the power to voice their views and see what happens. I don't understand how renaming them all helps here.

    People will do whatever they feel that they want to do, whatever they feel that they have to do, and much of what they're told by people who pay them. If you told Doctors and Nurses that they'd all go by the new term "DocNurse" and you expect them to have knowledge of each others work then very little would change. The doctors would go to the doctor conferences and the nurses would go to the nurse conferences and they'd each research whatever they were interested in, do what drives them and excel in the skills they have driven by the passion that made them choose their role in the first place. I don't see anything different in role-conflated companies. I sometimes see developers pretending to test with automation and BDD. I sometimes see people with one title where the roles are simply forced into being tacit as if renaming a group of people was going to change the culture. I'm still to be convinced of the value of it.

    I wonder how many excellent testers a company would miss out on hiring because their job requirements involve so much non-testing? I'd rather hire people who are passionate masters of their craft than people who claim to be able to do some of everything.
    20 pointsBadges:
    report
  • Justin Rohrman
    @counterfactuals Matt Heusser got context driven in the early 2000's, and was a programmer ... but became a tester later.
    2,130 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: