Software Quality Insights

October 21, 2009  4:00 AM

First takes on Boston SPIN with Damon Poole and STPCon

Matt Heusser Matt Heusser Profile: Matt Heusser

    As I write this entry, I’m in the air above Boston, just about to land for the Software Test and Performance Conference. The trip is actually a twofer; I’ll be at the conference tomorrow, but tonight I’ll be out at the Boston SPIN (Software Process Improvement Network), a local user group.

    … time passes …

    After touchdown at Logan International Airport (Boston), there is no airport shuttle to the conference. Believe it or not, one of the editors at works and lives in town. Not only is he going to the conference, but he also offered to pick me up at the airport.

    And, true to his word, there he is. Also, true to his word, his modified black T-Bird is the loudest car in the parking lot.  I suppose I can’t complain about a favor from a friend.  And the FireBird has some serious horse power.

    After I check in at the hotel, it’s time for The SPIN meeting on “Agile Software Development: Does it make a difference?”

    Nearly as importantly, there will be pizza.

    The Boston SPIN

    Yes, there was pizza. So much pizza, in fact, that I missed a little meeting in the corner: a roundtable discussion of Reliability Metrics. Instead, I sit through an informal discussion with a few people who choose to sit at our table; deep enough that Dan Mondello does his own blog post on it.

    I do get to see the main event, which was an introduction to Agile Software Development by Damon Poole. Damon is also the author of Do it Yourself Agile, which is nice – and free – e-book on the concept.

    The slides from Damon’s talk are online. You can view them here.

    A few of the ideas in the talk caught my ear enough to want to share here, in my words:

    * We tend to like to think of our work as “done” when we have written code, or run tests, or completed requirements documents. The only “done” that matters is running tested features.

    * Development with Legacy Software is just hard to do in general. Yet if you take it a piece at a time and slowly begin to apply agile techniques, over time, the software will become more malleable.

    * There is no business value in a complex architecture. Instead of designing for extensibility, just build what the customer asks for and design a system that can be easily changed in the future. (Matt Heusser adds: In my entire career, every time we designed the software to be extensible in a specific direction, that darn customer asked to have it extended in a direction we hadn’t anticipated, and we had to start over at square one.)

    * Agile development enables the team to change priorities every iteration.  In this age where a new product like the iPod can radically change the business landscape in a matter of months, it might be better to not be locked into a requirements document with a multi-year release plan.  Just a thought.

    The big reveal

    At the end, Damon didn’t see a business case for ‘Agile’ development. Instead, he layed out 14 different practices – such as continuous integration, daily standups, refactoring, user stories, and one piece flow. According to Damon, each of these practices can stand on their own, with perhaps a 5% performance improvement.  Because the practices build on each other, doing several of them have a compound-interest effect.

    I don’t think Damon meant us to take this literally or scientifically, but merely metaphorically, as a thought experiment. If every Agile practice stands on it’s own, and the lot of them, taken together, provide even more benefit – then why not try an Agile approach?

October 20, 2009  6:40 AM

Boston SPIN: A small group’s big ideas about agile development

Daniel Mondello Profile: Daniel Mondello

I was invited to attend a SPIN (Software Process Improvement) meeting near Boston. SPIN has been in running for 17+ years and has over 1,100 registered members.

At the meeting, I joined a group gathered around a circular table to finish off my pizza slice and drink my apple juice. The group was already deep in conversation over the topic of our assembly: What exactly is agile software development, and where and how can it be practiced?

For Tru Hong, a QA engineer and analyst, “agile describes the process of telling small stories, small stories that would help in the telling of larger stories.” “Stories” are used commonly in requirements-based in agile, and storytelling is defining long-term goals and then setting shorter goals in which to accomplish the final vision.

Everyone seated at the table had a reasonable working knowledge of agile and its derivatives. While all agreed that agile is their preferred methodology, they also admitted that it’s difficult to define and sometimes to implement.

Two testers from Pembroke, Mass., discussed their experience with agile, which seemed to be similar to that or others at the table. Their company, which will remain anonymous in this post, was recently bought by a rival vendor that uses agile. As a result, the company had begun a restructuring process which included a move to agile. Outside a general knowledge of the term “agile,” his company’s management had only a vague estimation of what the buzzword referenced. So, management didn’t really understand the scope of the move, and it was left to the development team to implement.

Keith Willett, software product director for American Science and Engineering, Inc. (AS&E) located in neighboring Billerica, Mass., spoke highly of his company’s success with Scrum. His team was an early adopter of the methodology, and he’s Scrum-certified. well. Willett’s background, much like the others at the table, was in traditional waterfall approaches. He called waterfall the “the most medieval of software development methodologies,” and others agreed.

Complete adoption of agile is AS&E’s goal, Willett said. His team is making everything agile, everything from approaching a software project to tool usage to the way in which revenue is gathered and distributed amongst the project.

Matt Heusser — a test pro, agile advocate and frequent contributor — joined the discussion and shared his work experiences with agile. Company-enforced agile had led to his development teams turning out iterations bi-weekly. Sometimes achieving that goal meant “a modest amount of overtime,” Heusser said. Though the deadlines are challenging, he said, he’s found the work completed in agile is more rewarding, because it leaves you with a sense of ownership over a project.

October 19, 2009  8:29 PM

Pulling compliance audits, procedures and practice together

Rick Vanover Rick Vanover Profile: Rick Vanover

In my last post I mentioned that it is worth the effort when drafting work instructions that match actual practice. Now, I want to take that one step further and hit on a point that can many of us that go through compliance audits.

In a perfect world, compliance audits are a quick check under the hood and a handshake. Unfortunately, they are much more involved than that. One practice point that I want to share that can help the compliance audit practice is related to structuring work instructions and established procedures to meet compliance requirements. For things like software updates and critical vulnerability scans, you can make your life simpler by a little forward thought. Take for instance PCI audit requirements for software update frequency, if your work procedures are written to the requirements of PCI you are one step ahead of the game. Further, if you address your authoritative system inventory with a procedure that is used to perform periodic security scans; the compliance requirements, documentation requirements and actual practice go full-circle without any additional work.

Saving additional work is critical. Having disparate compliance efforts, documented procedures, and actual practices are just asking for things to get out of sync. The investment in making the procedures match compliance requirements is not a new one, but common sense for those with any regulatory footprint. This can assist staff that are new to compliance requirements as well as make business plans clear in time-critical situations such as a company acquisition.

Do you write your procedural inventory to the compliance requirements? If so, share your efficiency tips below.

October 13, 2009  8:44 PM

Project management consultant/author extends savings to SearchSoftwareQuality readers

Daniel Mondello Profile: Daniel Mondello

Barbee Davis is an experienced Project management consultant and author who routinely writes for a semi-monthly international publication, Community Post on project manager concerns as well as ways to successfully negotiate projects.

She is offering our readers this 30% savings coupon on her latest project management book published through O’Reilly: 97 Things Every Project Manager Should Know.

To receive this 30% title savings, apply this promotional code ABF09 through the O’Reilly website

For more information on Davis click here.

Related Content
Software expert on Agile’s rise, avoiding project management mistakes
Software project management consultant opines on importance of agile development, common errors in PM, PM career preparation and more.

October 13, 2009  6:25 PM

Is your software test team rigorously incompetent?

Jan Stafford Jan Stafford Profile: Jan Stafford

What happens when a software development or test team’s manager preaches rigor in processes, but doesn’t make sure that team members do processes correctly? “Rigorous incompetence,” says veteran tester and author James Bach. In this video, shot at the recent Software Quality Engineering STARWEST Conference, he shares his views on managers who insist their teams look busy, but don’t ensure that they’re doing their jobs well.

Related Content
Testing rigorously, learning on your own: A chat with James Bach
Author and founder of Satisfice Inc., a software testing and quality assurance company, describes best practices in testing and learning.

James Bach interview: Dispelling software testing myths
The current drive for more rigorous software testing processes and more test documentation is misguided and wastes time and money, says James Bach, author and software testing veteran.

October 12, 2009  2:24 AM

How software test teams’ people skills affect results

MichaelDKelly Michael Kelly Profile: MichaelDKelly

I wasn’t always the brilliant, charismatic, well-loved tester that I am today. Well, I’m not really any of those things today, but I often like to think I am. That’s my problem. Sometimes my ego gets in the way of my best work. I’m also susceptible to all the bad behaviors that come with tight deadlines, layoffs and work I don’t particularly like. Like everyone else, I can let stress tax my professional relationships.

Karen Johnson will be tackling this problem head on during her talk — titled “Building Alliances” — at the Pacific Northwest Software Quality Conference (PNSQC), which takes place Oct 26-28, 2009 in Portland Oregon. Karen will look at how to create and foster successful professional relationships with teams and people. That means not just fostering a collaborative spirit, but also looking for ways that teams can look out for each other and work together.

“The talk focuses on building healthy relationships at work,” Johnson told me recently. “In theory we have a relationship with everyone we work with, at least at some level. But how many relationships do you have on your team or at your company that you can count on when the going gets tough? The answer is usually a handful of carefully maintained alliances. Those alliances are the solid dependable work relationships my talk is focused on. How do those alliances get built? And how are those relationships maintained?”

Karen Johnson is an independent software test consultant and has been involved in software testing for more than two decades. She has extensive test management experience, and her work often focuses on strategic planning. A frequent speaker at major conferences, she has published numerous articles and recorded webcasts on software testing, many of them here on She is also a contributing author to “Beautiful Testing,” a O’Reilly book coming out later this year.

Most recently, Johnson’s focus has been on developing a sense of community for software testers working in the area of regulated software testing. She’s been studying office politics and how they affect people. In her PNSQC session, she’ll reveal her finding on positive practices of healthy, functioning teams and how those practices build good, collaborative relationships.

“I find it interesting that sometimes people in our software testing community recollect with horror painful people events that have taken place from their past work experiences,” she said. “And sometimes people talk about successful alliances at work and people they’ve been able to depend on. But what I haven’t heard people talk about is how to go about creating those healthy relationships. After a number of years of working in a variety of companies, I have a collection of positive and negative experiences. My talk draws from both.

“I don’t like politics. I’m not especially good at them and earlier in my career, I figured I could avoid politics altogether. Once I became a test manager, I realized, I couldn’t avoid politics. Much of what I wanted to say and plan to say, comes from my experiences, my stories and from my heart of what I’ve learned.”

In typical Karen Johnson style, her talk will also be well referenced. Karen told me that in addition to her own experiences, the talk also draws on the work of others. “There is an amazing amount of material on the topic,” said Karen. “These references are especially good reads…”

Clearly, her talk is not focused on the technical aspects of software development. It’s about people. For those who might not feel topics like this are important, Karen offers the following:

“Let me offer this thought: One of the skills I’ve focused on acquiring and sharpening over the years is being observant. When I test software, I’m making observations. When I work with people, I use my observation skills to notice relationships and dynamics. I need to be able to observe changes with people and teams in order to get work done. It’s not just the raw technical skills that get the job done.

“If you want to test software and quietly report defects, I suppose technical skills might be enough. If you want to be someone who has influence to improve a product, you have to be able to work with people. I think it makes sense to have talks at our testing conferences that address people topics.”

Karen will also be teaching a full day tutorial on SQL at PNSQC. The class is a more advanced class focused primarily on joins and queries. For more on the upcoming show, check out the PNSQC website. You can also learn more on Karen Johnson’s website, Testing Reflections blog or by checking out some of her Expert Responses and Tips on

October 8, 2009  7:31 PM

Testing rigorously, learning on your own: A chat with James Bach

Jan Stafford Jan Stafford Profile: Jan Stafford

My conversation with James Bach yesterday was certainly the high spot of my StarWest 2009 experience. We talked about topics dear to our hearts, such as critical thinking, self-education, wrong-headed notions about what constitutes good testing practices, Shakespeare and more.

Bach is author and founder of Satisfice Inc., a software testing and quality assurance company, but mostly he’s known as an articulate, passionate writer and speaker. His philosophy of testing is controversial, because he believes that what software testers know from experience and self-education is as valuable, if not more important, than certifications.

He knows, he told me, that certifications and degree help testers get jobs from strangers. Both “pieces of paper” serve as verification that the person has the skills needed for the job. Unfortunately, neither really verifies that the holder can think critically or creatively about what’s needed to do the job in various situations or work on a team.

That said, Bach is much more interested in personal fulfillment and the acquisition of real and not rote learning. He serves up his lifelong self-eduction journey in his new book, Secrets of a Buccaneer-Scholar. It’s taken years to write, he said, because he had to have experiences before he could write about them.

Besides self-education, Bach is currently on a quest to get test organizations to re-examine the concept of rigor in software testing. “Too often, they’re doing their testing rigorously, but they’re also doing their testing incorrectly,” he told me. Rigorously doing something wrongly compounds problems instead of solving them. He sees too many test managers equating success with the large number of tests run and defects found instead of effectiveness. You can read more of his views on test rigor in Mike Kelly’s interview with him, titled James Bach interview: Dispelling software testing myths

As I said, our conversation took off on many tangents, touching on our love for Kenneth Branaugh’s Shakespeare films and the cool things about testers, one of which is their curiosity. This experience was what actually going to conferences is all about, meeting people in person and talking about a little bit of everything.

October 8, 2009  6:47 PM

Consultant Lloyd Roden: Choosing realistic software test team challenges

Jan Stafford Jan Stafford Profile: Jan Stafford

Some challenges are not worth taking on, said consultant Lloyd Roden during his StarWest 2009 keynote here at the Disneyland Hotel today. A software testing expert for U.K.-based Grove Consultants, Roden started off his talk about testers’ top challenges today with a warning about setting up the wrong challenges for a test organization.

“We can’t fight every challenge. You can’t do everything,” Roden said.

Examine the preparation needed and the talents of your team before setting goals, Roden advised. “Climbing everest would be a challenge, but it would be a stupid one to undertake without preparation and skill,” he said. “Cooking a meal for 20 would be a challenge for some and not for me, because I love cooking and have cooked a lot.”

A good challenge improves the people who take it on and opens their minds to different approaches to achieving the goal at hand. “Bad challenges are harmful and have undesirable consequences,” he said.

After advising testers there to “choose your battles carefully,” Roden gave a personal example, recalling his daughter asking for pierced ears when she was 14 years old. At first he was against the ear piercing, his daughter’s first request, until his wife explained that piercings didn’t have to be permanent. So, he chose not to say no to piercings. He did say no to tattoos, which are permanent, until she was 18. He didn’t feel she was prepared to make a decision with permanent consequences.

One way test managers can know what challenges to set for test teams is to do testing himself. He often runs into test managers who do no testing and believes they’re missing the opportunity to really know what goes on with his team everyday. Remaining outside of testing can lead test managers to set unrealistics goals and challenges.

October 5, 2009  2:30 PM

James Bach interview: Dispelling software testing myths

MichaelDKelly Michael Kelly Profile: MichaelDKelly

Software consultant, trainer and author James Bach is a bit of a lightning rod in our industry. Most of his notoriety comes from his stand on software testing certifications. He doesn’t like them and isn’t shy about letting people know it. Bach also challenges other widely held beliefs in our industry.

Bach takes his off-the-certification-track testing tutorial, How to teach yourself testing, to StarWest 2009 in Anaheim this week. He’ll also take on some of the myths about rigor in software testing at the Pacific Northwest Software Quality Conference (PNSQC), which takes place Oct 26-28, 2009 in Portland Oregon.

The software myths talk gathers into one place and in succinct form arguments that Bach has been making indirectly for years.

“I keep hearing that more rigor is good and less rigor is bad. Some managers who’ve never studied testing, never done testing, probably have never even seen testing up close, nevertheless insist that it be rigorously planned in advance and fully documented. This is a cancer of ignorance that hobbles our craft.”

Bach’s talk is about clearing up what he calls the “silliness and sloppiness” surrounding the notion of rigorous processes.

“Managers want to say ‘let’s get a lot more rigorous in our processes!’ They may say ‘formal’ or ‘repeatable,’ but it’s all the same sort of thing,” he told me. “But getting rigorous is no panacea. It’s actually a bad idea, in many cases, because rigor can interfere with learning by locking us into bad practice. We need to apply rigor without obsession or compulsion, and at the right level, so that our testing is flexible and inexpensive.”

Bach has been a test manager or consulting tester since Apple lured him from a programming career in 1987. He spent about 10 years in Silicon Valley before going independent and traveling the world teaching rapid software testing skills. He is the author of Lessons Learned in Software Testing, and a new book, Secrets of a Buccaneer-Scholar: How Self-Education and the Pursuit of Passion Can Lead to a Lifetime of Success.

I asked him to offer an example of one of the myths facing our industry. He shared some of the impact of detailed documentation around testing processes and procedures:

“One myth is that a process becomes more rigorous just because it is written down. Well, as anyone ought to know who works with procedures, what is written is not necessarily what is practiced. In fact, it rarely is, in my experience. Moreover, it is not even possible to write down everything that matters about the processes that skilled people use.

“The impact of this myth is that a huge amount of money and time is wasted trying to chase an unhelpful obsession with documentation. It makes good theater – you look busy – but it doesn’t necessarily help you do better work.”

Bach has recently been training to fly a float plane. During PNWSQC he plans to draw some examples from that experience, such as the fact that pilots have to learn many procedures and protocols in order to fly safely. “Rigor is an interesting challenge for a pilot.” he said. “In the talk, I talk about how we use checklists and some of the problems with those checklists.”

A lot of the philosophy behind James’ stance can be found in his latest book:

“Compulsory public education is an example of rigor myths getting drunk and going wild. Many millions of my fellow humans have bought into the idea that education is possible only through schooling. Rigor applied to education is usually presumed to mean that you subordinate your will to that of a schoolmaster, priest, guru or some other authority. I practice a different sort of rigor — education strictly inseparable from life. My life is my education. My life is my own. Education is not some side activity that I do to prepare for life.

“That applies to testing. I am a tester. That is a big part of my life. So, I can’t accept these silly certification programs and bad standards and process guides. Those are examples of rigor, however they represent bad rigor, and my standards are too high for that. I’m trying to raise the standards of the industry so that it will laugh at bad work instead of enshrining it.”

You can learn more about James Bach’s testing practice on You can also follow him on Twitter.

September 29, 2009  10:32 PM

How software project managers can react to recessionary trends

Jan Stafford Jan Stafford Profile: Jan Stafford

Project management (PM) consultant Michelle LaBrosse shared some quick tips for PM strategies in recessionary times with me recently. These ideas gelled during an interview with Carey Earle, president of Green Apple Marketing, on her syndicated radio program, Your World, Your Way.

“We talked about great ways to use these trends as a great launch pad for new ideas, solutions and direction in the workplace,” said LaBrosse, founder of Cheetah Learning, a PM consultancy, and aPM issues blogger.

Many projects are on an “economic slim-fast” diet, LaBrosse said. Not surprisingly, she’s seeing may project managers focus on practices that save their teams’ time, cut spending and improve quality and time to market. Also, businesses are trending toward novel perks in these days when raises are rare and budgets tight. She’s seen good results when project managers reward teams in inexpensive and creative ways, such as making a premium parking spot available to a top achiever each month.

The secrecy and lack of honest documentation of the early 2000s has caused an about face, in a trend that LaBrosse and Earle call The Full Monty. “Technology brings us a whole new level of honesty whether we like it or not, but underneath the technology is a new human desire for trust and transparency like never before,” LaBrosse said. “Think: Is your documentation in good shape? Are you leaving a trail that you’re proud of? How can you be more transparent in your business?”

Finally, LaBrosse advises project managers to start noticing trends on their own. “The art and science of noticing the dramatic or subtle changes taking place will help you and your team continue to seek out future opportunities and successes,” LaBrosse said.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: