Software Quality Insights

Feb 17 2010   10:06PM GMT

What’s going on with Google’s social network testing?

Matt Heusser Matt Heusser Profile: Matt Heusser

Headline: “Google admits buzz social network testing flaws.”   The same head was covered on MSNBCThe Business Insider, and USA Today.  Regarding the mess, software testing authority James Bach said publicly :”These problems with Google Buzz could have been averted using the most basic kinds of critical thinking!”

Here’s the issue in a nutshell: Google Buzz is a new social tool that allows you to share your opinions and status with other users.  Because it integrates with Gmail, Buzz knows to whom you send email the most. Buzz also had an added feature that automatically picks your “friends” — those with whom you share — out of the box for you.  It also made your communication stream public by default.

The automatic friending is no problem for a lot of people; but what if you are having a legal dispute or custody battle, don’t want the boss to know that you’re interviewing with another company or just have two in-laws that don’t get along very welll?

Oopsie.

Google Buzz — and, arguably, Microsoft Vista before it — actually represents a unique kind of testing challenge, one we don’t deal with much in the industry.  You see, Buzz actually works.  No, that is not a typo. Buzz works according to it’s specifications.

The problem is, the specifications were wrong, objectively wrong, in that they were not in the public’s interest.

I’m certain the folks at Google tested the heck out of Buzz.  They probably had test scripts that exercised every possible feature, and it all worked per spec.  I bet they had metrics and greenbars all over the process.

But they had the wrong tests.

You could claim this was a product management failure.  After all, the software did what it was supposed to do, but “what it was supposed to do” was the wrong thing.  It is possible, even likely, that in some meeting a tester or developer asked the question:”Do we really want all thing information to be public?”And got an answer that was a resounding, “Yes!”

So it might be more fair to say that this was a systems-thinking flaw. It expands the idea of testing, beyond “conformance to specification” and into another land, one of “fitness for use.”

A test for fitness for use needs to do more than “requirements traceability,” it needs to ask very tough questions about what the software does and if it does the right things? Those are questions of quality. While I’m reluctant to blame “poor testing” on a product management decision, certainly more testing might have reduced the risk for a bad rollout. Even the folks at Google agree that, in hindsight, they should have had a larger, more public beta.

As social media continues to become more integrated into our daily lives, and mashup platforms that combine Facebook and Twitter, etc., become more common, I expect we’ll have more testing challenges of this nature.

I wonder what will happen to the profession of software testers in the years to come.  Will we bury our heads, claiming: “It worked per spec. It’s a security flaw. Talk to project management”?  We will we might expand our view of testing to include critical thinking, as an investigative approach that can not be trivially scripted into clicks that “prove” the specification “works” or easily automated away?  Only time will tell.

For right now, we should all buckle up; It’s going to be quite a ride.

5  Comments on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • Dckismet
    Matt-- That was a very good post, I really enjoyed reading it. It resonates with me, and I'd like to see to what degree others have similar experiences. For me, my role as a tester has almost always included that blur into "are we doing the right thing?" I've always felt like as a tester I am largely an advocate for the user, and it has included a little bit of testing that the thing does what it is supposed to, but also includes things like "Is it usable?" "Does it make sense?", *and* "Does it really solve the user's problem?" This happens more at smaller companies, I would guess, where there are not user experience people, or sometimes not even business analyst types. My personal role as testing and quality advocate has spread far into the business side of my more recent jobs, as I have begun to spread the word of building quality in from the beginning. I've also seen the situation mentioned wrt Buzz, too, where some tester is out there saying "I told you so!". I've been told that the question I was asking was invalid because it was, in fact, what they wanted, and seen it explode. I've been told to stay in my own sandbox, and have seen the train wreck that I was trying to prevent happen anyway. I think the most stark example I have heard of this being a problem was with the Challenger explosion, where there was someone who had identified the possible danger of the ice hitting the shuttle, and had been ignored. Lives were lost as a result. I'm glad I don't work in a life/death industry ... at least, not quite as directly. Getting back to the "where will testers go" question, I have struggled with this myself. As I move into a "build quality in" advocate-type position more and more, I am finding that I am less and less able to got down and dirty with the technical details. Some days, I am concerned that I can't do both, there's just not enough time. Some days, I wonder if I will have to choose the technical role or the ... rather undefined, other role (management track? quality advocate? business analyst?). I want to do both, however, and when I feel most uncertain, I think of our friend Elisabeth -- she seems to be able to handle being a Testing Advocate, as well as be able to jump into someone's Java code during dinner at a conference.
    0 pointsBadges:
    report
  • Mgaertne
    Critical thinking was always part of my testing. The more difficult problem is the one of not being heard about the concerns. Don't know how this works for google, but I doubt that they have the same project management attitude as I experienced this far. Social Media and Privacy are at odds. Don't understand this as an excuse for poor critical thinking. Social Media relies by default on sharing some information. Privacy on the other hand hides the same information. So, the more social media we use, the more transparent we become in the world wide web. Google made a business out of it. Cirtical Thinkers assume they record each and every data information that goes over their platform. I neither believe nor reject their premise. Instead I try to keep myself suspsicious enough. So, what we will be experiencing is a rising claim of data protection specialists to fight against the abuse in social media.
    0 pointsBadges:
    report
  • Amcharrett
    I wonder though if its good to look at Buzz in terms of a failure. The way I see it Google tried something out that had never quite been done before. So, some assumptions and thinking turned out to be incorrect, but when you are trying out new stuff its hard to know exactly what is or what is not going to work. I think Google were spot on when they said they should have had a longer beta, and probably they could have done without the marketing hype before launching. The challenge is to look beyond these sorts of things in terms of pass or fail and see what can be taken from such an interesting venture.
    0 pointsBadges:
    report
  • Matt Heusser
    [I]I wouldn't frame buzz itself as failing, as much as a project that had [B]a[/B] failure within it. So I'm saying you could call that bug [B]a[/B] failure. And yes, I agree, how we respond to it is what matters. I hoped that was clear from the post, but thank you for the gentle reminder, Anne-Marie.[/I]
    1,105 pointsBadges:
    report
  • Fiwedding
    I like it
    0 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: