Software Quality

Agile, management tools help small team boost software quality

Scheduling, documentation, tracking bugs and –- most critically -– producing great products with a small development staff are the headaches Dan McRae, Comet Solutions Inc.’s software engineering manager, faces every day. I talked with McRae recently about how he’s combined best practices and lifecycle management products to get the upper hand on quality assurance (QA) and deliver strong products on deadline. Indeed, Comet estimates that these tactics have improved software quality by 25% and time-to-market by 10 to 20%.

McRae oversees a team of about 10 developers, two of which are dedicated to QA, although all do some QA work. They produce about four major and four point releases each year for Albuquerque, N.M.-based Comet, which provides design engineering workgroup software primarily aimed at enabling early simulation.

“Our biggest challenge is that we only have two full-time QA people,” McRae said. “The rest of us generate more features than they can thoroughly test. As manager, I direct what QA’s focus is. Some of those directions are based on word of mouth from our developers; but I also get direction from our chief technical officer, who points to things that are high-risk and need to be addressed. There are always challenges because there are always the latest defects that have the highest priority, and several of those pop before you can fully QA the previous ones.”

Comet’s development team uses agile development and automated tools to reduce the development and QA burden and improve quality overall.

Using waterfall methods for product development became very difficult as software and users’ needs became more complex. Under waterfall, it took too long to handle customer requirements and changes. Today, McRae’s team uses such agile techniques as daily standups to discuss current relevant issues, working on a two-week iteration and having release planning meetings based on Scrum practices.

While moving to agile processes, the development team found that its homegrown tools couldn’t evolve to improve the team’s ability to track defects, feature releases and scheduling.

“We had been using a home-concocted system for tracking defects and feature requests internally ourselves, and it wasn’t able to help us keep up with changes in customer requirements or QA,” said McRae. “That system didn’t have scheduling abilities, so planning was done on a kind of ad hoc basis.”

After scouting around for better tools, Comet’s team did a trial run on Rally Software’s Agile lifecycle management solutions. The trial led to adoption.

“Rally gave us a more robust system for defect tracking and feature request tracking. It also added scheduling, so we can schedule our development efforts, give a structured process to the collection, evaluation and implementation of feature requests as well as defects,” said McRae. “That gave us the opportunity to look ahead and plan our time out, knowing what features could be done in what time. Rally provided us with flexibility to swap and replace as needed.”

Today, when a new business need arises while development is in progress, McRae can look at the project plan in Rally and “see that we need to add certain features that require certain resources and determine what we can take out that was equivalent. Rally gave us the platform for making that judgment call without the guesswork that was part of the process before.”

Though the team has made great productivity gains, there are always improvements needed. Documentation is one area “that’s always a challenge,” McRae said.

“Any released software needs documentation, but if you don’t have a functioning product then documentation isn’t worth anything. There’s always the balancing act between building features and working on the product and trying to find time and the appropriate resources for the right level of documentation to be done. It’s something we don’t have a specific process to handle yet.”

McRae plans to look at Rally’s features to get some documentation metrics, such as finding which user stories and defects were completed. But he’d love to get his hands on an automated documentation tool. Any suggestions?

Book gives managers a software testing reality check

Software testers and quality assurance managers know that perfect software doesn’t exist.

Unfortunately their project managers, company executives and legal teams often don’t, and these people have “unreasonable expectations,” experience “constant disappointments” and make “disastrous decisions” regarding software testing, said Gerald M. Weinberg, author of many IT and software quality books, in our recent phone conversation.

Weinberg wants those people to read his new book, Perfect Software And Other Illusions About Testing.

“This book is designed for people who don’t fully know what testing encompasses, but make decisions that affect how and how much software testing is done,” said Weinberg.

The current economic downturn and resulting budget cutbacks in software development make informing decision makers and influencers about when, how and why software must be tested even more urgent. While it seems obvious to software testers that cutting software testing is a sure way to produce faulty software, it’s not as obvious to managers.

“Management thinks of testing as what is done after the software is developed,” Weinberg said. “They schedule testing at the end of the process, and that’s the easiest place to cut if there’s a cost overrun.”

Naturally, that’s when the QA manager should step in and explain why testing shouldn’t be cut; but often the lack of funds defeats such arguments at the end of a project.

Establishing processes wherein software is tested early and continually is a better practice than testing at the end of development. “There should be more people wearing testing hats at each stage of development, because mistakes made early are very hard to find later,” Weinberg said.

Reducing development costs by reducing testing is a penny-wise, pound-foolish approach. More and more, Weinberg said, software-related lawsuits and malpractice cases verdicts come down to what software wasn’t tested adequately.

Another common management mistake is not putting software support on the same ledger as development.

“Accurate cost accounting takes into account post-release costs,” Weinberg said. “Usually, the cost of fixing errors doesn’t get attributed to development managers and project managers. It should be.”

Weinberg hopes this book will help managers get “tuned into reality” about software testing. The key is getting it into the right people’s hands, he said, noting that testers and QA pros may want to keep it handy to do just that.

If you’re having trouble getting support for testing in any area or face testing cutbacks, our resident site experts can offer helpful advice. Just send your questions or describe your problem in an email to  editor at searchsoftwarequality.com. This software security pro wrote in and got advice on how to get management support for security quality.

How to maintain software quality, complete projects in a recession

Software consultants, vendors and project managers are already seeing software project failures and slowdowns resulting from the new recession.

That finding and the advice for software developers and project managers offered in this post comes from interviews I’ve recently conducted with 13 software industry experts and the blogosphere.

“Companies are panicking due to the economy. They’re compressing projects and schedules. As a result, key projects are failing,” Lawrence Oliva — senior consultant/program manager with CH2M HILL, an Englewood, Colo.-based engineering and program management firm –- told me in a recent conversation. “In part, this is the result of trying to get things done faster with less resources.”

Consultants are seeing reductions in software testing being used to cut costs. That tactic is a recipe for project failure, said consultant Karen Johnson in my recent post about why software project fail.)

My sources also spoke of seeing several large, current software projects in crisis mode because senior developers were laid off and the remaining, less-experienced developers didn’t have the know-how needed. Unfortunately, many development groups have been running lean for a long time and running leaner pretty much means stopping development.

“Lean computing is nothing new,” Oliva said. “Companies have been operating at skeletal levels since the economic downturn of 2000, when that downturn caused intense development staffing cuts. It really is sad to think about having to be even leaner.”

That’s the situation, but it isn’t hopeless. Gleaned from my interviews, here are some tips for maintaining software quality and project success during this recession:

Analysts and others advise development teams that lean computing and the agile model can help them do more with less.

Some companies have already made this move and feel ready for tough times. For example, Des Moines, Iowa-based Principal Financial Group –- which provides software products for the financial industry — uses the agile model and has standardized project management, testing and quality assurance processes and tools used by all its development groups.

“Standardizing has reduced redundancies in tools, processes, documentation and more, running as lean as possible,” Principal’s senior IT system analyst Mark Ford told me. Principal uses HP Quality Center to manage development.

Project managers whose companies have not already made cuts to development staffing or budgets need to advise decision-making executives about the business benefits of their projects, industry insiders advised. Also, when cutbacks have taken place or are proposed, PMs need to explain the business impact and guide where the cuts will be made. IBM’s director of Rational offerings, David Locke, offers this advice:

“Always talk in the context of how the project will help the business. If I take this 10% off, I will have the least impact on the business. Give real information, not data. Remind managers that the company creates software to make our businesses more streamlined, more competitive and to deliver better ROI. Your company may need immediate cost savings, but eliminating software that could give the business the most cost-effective competitive edge is probably not the way to go. Talk about business impact, always.”

This is certainly an appropriate time for PMs to push for investments in software testing automation tools, like automated code checkers, experts told me. However, Oliva added:

“Don’t trust in those automated systems absolutely. You could get the wrong results from the use of automated systems.At some point human beings have to be involved. Delivering quality software is not a machine-to-machine process. Not having quality monitored and managed by humans is dangerous.”

Another suggested labor- and cost-saving move is adopting streamlined models of programming, like Extreme Programming or Structured Programming.

The most important strategy for weathering the recession is becoming more realistic about software needs and reducing complexity of software. “If companies do this, it could be the one good thing that comes from the recession,” Oliva said.

Continue the conversation: Please tell me what adjustments you and your development team are making to deal with the tight economy. You can respond by commenting below or writing to me at editor@searchsoftwarequality.com.

Here are some resources for more advice and information:

• Find out what this software developer did when his project’s staffing went south in Survivor’s lessons in test management.
• There are many links to articles on the recession’s impact on software projects in this blog post which offers advice on succeeding during a recession or anytime.
• Ryan Martens writes about how to make good staff reduction decisions.
• In this article on project management, Lawrence Oliva discusses
  the economy’s impact on software projects and strategies for PMs.

Value of SANS’ list of top software errors rests on project managers

Whether the CWE/SANS list of the 25 most dangerous programming errors will contribute to the creation of better software depends on whether managers, rather than developers, read it and take action, according to Jack Danahy, chief technology officer and co-founder of source code vulnerability analysis firm Ounce Labs Inc. I talked with Danahy today about the follow-up and follow-through that could make the list a valuable turning point in development, rather than a partially-remembered checklist.

“It’s one thing to come up with a list of 25 things that developers should consider, but we haven’t arrived at a point where anyone is meaningfully asking or requiring developers to consider these things,” Danahy said.

Project managers should support developers spending time to research these issues, in Danahy’s view. The best-case scenario would be that software development managers -– the program manager, business unit manager, software auditor, etc. -– would use this list in specifications, asking for metrics to make sure that those errors have been looked for and eliminated before the software rolls into production.

“Developers won’t remember this list off the top of their heads, but if it becomes codifed as a requirement they will remember,” Danahy said. “A team could come up with distilled list of 5to 12 key design criteria that would provide the essence of keeping these errors from happening.”

To read more opinions about the CWE/SANS research, check out software development pro Mike Kelly’s post on using the list and SearchSecurity.com’s news report.

What will your organization do with this list? Will it have an impact or be quickly forgotten? Sound off in our comments below or by writing to editor@searchsoftwarequality.com.

Why software projects fail and more will fail in 2009

Why do software projects fail? There are many reasons –- and they’re spelled out below –- and 2009 may bring more failures than usual as budget cuts spur project managers to make cuts in the wrong places. So said software quality consultants Lawrence Oliva and Karen Johnson when I talked to them about 2009’s software quality landscape.

Human error causes most software project failures, said Oliva — senior consultant/program manager with CH2M HILL, an Englewood, Colo.-based engineering and program management firm — so most are avoidable. Here is his list of the mistakes he sees most often and his comments:

1. Unclear requirements: “Most people don’t know what to build because they’ve never defined it. When they build the software, it fails because it doesn’t meet people’s needs.”

2. Overly optimistic and/or unrealistic schedules. “People rush or skip things if the schedule isn’t realistic. Also, companies are panicking due to the economy. They’re compressing projects and schedules.”

3. Lack of user input: This links back to requirements mistake #1. “Developers don’t talk to people who are going to use the software.”

4. Lack of executive sponsorship and support: “When management doesn’t support and protect the project, it can often be undermined by internal politics and budget cuts.”

5. Turnover and layoffs: “Projects often fail when key people leave the project early in its lifetime.” Companies’ modern habit of laying off senior and, thus, higher-paid workers -– such as senior developers –- in favor of less experienced, lower-paid workers puzzles Oliva and me. “It doesn’t make sense, because the more experienced people take a humongous amount stability, experience with them that usually isn’t available any other place,” he said. “That hurts a project and hurts a company.”

I agreed, noting that the time lost due to less experienced workers’ mistakes and learning curve probably negates the savings in salaries paid. Oliva and I discussed that companies could do total-cost-of-layoff analyses; but Oliva said companies probably wouldn’t take the time to do that. “In a poor economy, companies often make hasty and project-wrecking decisions,” Oliva said. 

Besides layoffs, the recession is leading to other foolhardy cutbacks. It seems obvious that skipping testing is a path to software project failures, but software testing consultant Karen Johnson is seeing companies do just that. Johnson told me that companies are cutting down on or even skipping software testing altogether as a recessionary cost-saving method. If this trend continues, look for more embarrassing outages caused by admitted software failures or for “undisclosed reasons.”

While researching software failures, I read a Code Diesel post on software failures by developer Sameer Bora. To Oliva’s reasons why projects fail, Bora adds these common mistakes: sloppy development practices and poor reporting. Bora also created a handy chart on why software projects fail. Printed out or used as a screensaver, even, it could provide a visual reminder of project pitfalls.

Now it’s your turn: Do these reasons for failure ring a bell? Can you think of others? Let me know by commenting below or writing to me at  Trackback URL