Software Quality Insights

January 13, 2009  10:46 PM

Using SANS’ 25 most dangerous programming errors list

MichaelDKelly Michael Kelly Profile: MichaelDKelly

In a small cry of victory today, someone on the team found this article from the BBC detailing the “top 25 most dangerous programming errors.” I say small cry of victory, because he had recently logged a ticket in JIRA detailing one of those errors, but when the ticket came up for review it was ignored. It was acknowledged as an issue, but pushed to a later sprint for work.

While I agree with the reasons we used when we prioritized the ticket, for me this incident demonstrated a common pattern I see in the teams I’ve worked with. First, there seems to be an expectation that the testing team shouldn’t be looking for errors like this — that is, unless you’re a high-priced security tester. Second, that when issues like this are found they take a backseat to the more traditional functional defects.

I like research like this (both SANS and OWASP do great work in this area), because it gives me a way to structure the conversations that take place when these issues come up. I find that programmers typically respond well to links to catalogs of errors with descriptions. They are unrelated to the software they are working on. It makes it less personal I think — less close to home.

That said, these issues aren’t always burning, top-priority issues. Like I said, in the context of our current project where we found one of these, we have time to fix it. Given the current list of commitments, the issues we know we need to work, and the relative risk of this causing a problem — this specific issue can be sidelined for a couple of weeks until we get to it. That perspective is important as well, and it’s one that I sometimes forget. There’s always a business story to tell as well with issues like this — context is important.

For those interested in the more details on the list of programming errors, you can find the full list of errors from SAN here.

January 12, 2009  1:06 PM

Why software projects fail and more will fail in 2009

Jan Stafford Jan Stafford Profile: Jan Stafford

Why do software projects fail? There are many reasons –- and they’re spelled out below –- and 2009 may bring more failures than usual as budget cuts spur project managers to make cuts in the wrong places. So said software quality consultants Lawrence Oliva and Karen Johnson when I talked to them about 2009’s software quality landscape.

Human error causes most software project failures, said Oliva — senior consultant/program manager with CH2M HILL, an Englewood, Colo.-based engineering and program management firm — so most are avoidable. Here is his list of the mistakes he sees most often and his comments:

1. Unclear requirements: “Most people don’t know what to build because they’ve never defined it. When they build the software, it fails because it doesn’t meet people’s needs.”

2. Overly optimistic and/or unrealistic schedules. “People rush or skip things if the schedule isn’t realistic. Also, companies are panicking due to the economy. They’re compressing projects and schedules.”

3. Lack of user input: This links back to requirements mistake #1. “Developers don’t talk to people who are going to use the software.”

4. Lack of executive sponsorship and support: “When management doesn’t support and protect the project, it can often be undermined by internal politics and budget cuts.”

5. Turnover and layoffs: “Projects often fail when key people leave the project early in its lifetime.” Companies’ modern habit of laying off senior and, thus, higher-paid workers -– such as senior developers –- in favor of less experienced, lower-paid workers puzzles Oliva and me. “It doesn’t make sense, because the more experienced people take a humongous amount stability, experience with them that usually isn’t available any other place,” he said. “That hurts a project and hurts a company.”

I agreed, noting that the time lost due to less experienced workers’ mistakes and learning curve probably negates the savings in salaries paid. Oliva and I discussed that companies could do total-cost-of-layoff analyses; but Oliva said companies probably wouldn’t take the time to do that. “In a poor economy, companies often make hasty and project-wrecking decisions,” Oliva said. 

Besides layoffs, the recession is leading to other foolhardy cutbacks. It seems obvious that skipping testing is a path to software project failures, but software testing consultant Karen Johnson is seeing companies do just that. Johnson told me that companies are cutting down on or even skipping software testing altogether as a recessionary cost-saving method. If this trend continues, look for more embarrassing outages caused by admitted software failures or for “undisclosed reasons.”

While researching software failures, I read a Code Diesel post on software failures by developer Sameer Bora. To Oliva’s reasons why projects fail, Bora adds these common mistakes: sloppy development practices and poor reporting. Bora also created a handy chart on why software projects fail. Printed out or used as a screensaver, even, it could provide a visual reminder of project pitfalls.

Now it’s your turn: Do these reasons for failure ring a bell? Can you think of others? Let me know by commenting below or writing to me at

December 19, 2008  1:35 AM

Security flaws and Agile boom top software quality news in 2008

Jan Stafford Jan Stafford Profile: Jan Stafford

Security vulnerabilities and the boom in Agile development adoption topped the news charts in 2008. Here’s a rundown of the five most-read news articles and their significance.

Three of the top five articles focus on Agile development. In the #1 story, Predicting software quality trends for 2008, software quality experts predicted that Agile adoption will increase. Two other articles in the top five were about Agile. In fact about one-third of the top 20 news stories were about Agile.

Agile development: Not just for ‘agilists’ anymore (#4) discussed Agile’s move outside of its early adopter niche and the impact of its wider usage. Next came Software development groups take many routes to Agile (#5) which revealed results of’s 2008 Agile Trends Survey. Most Agile technique users, according to that survey, use Scrum (41%). Next in popularity was Extreme Programming (XP) at 15%. Others use hybrid Agile methodologies, while about three percent use Crystal and Dynamic Systems Development Method (DSMD) each.

The SSQ 2008 Agile Trends Survey showed that 45% of software pros follow Agile methodologies, while 44% use waterfall. Other development methodologies cited were test-driven development (19%) and RUP (15%).

Security flaws in leading open source software platforms drew a lot of attention, and articles on security issues in the Spring Framework and open source Java projects ranked in second and third place, respectively.

While open source development projects typically fix flaws quickly, as happened in these cases, the emergence of serious vulnerabilities may have taken some IT pros by surprise. People expect problems with Microsoft products, but not with open source products, said Kevin Beaver, CISSP and Principle Logic LLC consultant, commenting on the fact that these two stories got so many clicks.

“We’re seeing more and more that open source has its own security woes,” Beaver said.

Don’t blame the open source community and its developers, Beaver advised.

“The fact is that as long as human beings are developing applications on the complex and extensible OS (operating system) architectures we have, there will be security problems.”

Most importantly, he said, the emergence of vulnerabilities should not scare anyone away from using open source software like open source Java or the Spring Application Framework.

It is important to point out that just because a static analysis tool vendor finds flaws in open source code, that doesn’t mean the vulnerabilities can/will ever be exploited.

Keep using open source software, Beaver concluded, and “take this marketing tactic with a grain of salt.”

Now that you’ve checked out these stories, please let me know your choices for the top software quality news stories of 2008. You can write to me at

December 18, 2008  7:29 PM

Top five software quality tips of the year

Beth Pariseau Beth Pariseau Profile: Beth Pariseau

Check out’s most popular tips of 2008 — these top five tips for developers, QA testers, project managers and other software pros include indispensable expert advice on performance testing, software testing estimates and more. If you missed these great tips the first time around, don’t miss them now.

1. Cracking passwords the Web application way: Do you know how well your Web applications can stand up to authentication attacks? In this application security tip, Kevin Beaver, CISSP, lists some common Web app vulnerabilities and explains how to perform password cracking tests.

2. The ABC’s of software testing models: There are as many models for testing software as for developing software. Learn the basics of the common software testing models including waterfall-style, iterative-style and agile-style testing.

3. Testing for performance: Assess the problem space: Part one in a three-part series on performance testing, this tip explains how to assess the initial problem space before the first round of testing, including developing an understanding of your goals, how the systems will be used and other strategies.

4. What to include in a performance test plan: Find out what features a solid performance testing plan needs to include at a minimum, and how to reduce the likelihood of miscommunication.

5. How to estimate for testing on a new software project: How do you estimate for software testing on a brand-new project when you have no historical data for reference? Learn some methods in this tip.

December 17, 2008  5:02 PM

Open source, agile help move to lean software development

Jan Stafford Jan Stafford Profile: Jan Stafford

Bloated applications, platforms and architectures slow application development and make quality control and everyday usage time-consuming and nonproductive, said Forrester Research principal analyst John R. Rymer in a phone conversation yesterday. In this post, I’ll share Rymer’s thoughts on why software pros should join the lean software movement and his advice on how to create appropriately sized and efficient software. 

Forrester’s newly-published report, Lean Software is Agile, Fit-to-Purpose, and Efficient, lays out how software got so fat, costly and inefficient; the evidence that IT organizations are moving to lean software; the challenges involved in lightweight software development; and strategies for joining the movement. Rymer told me that the demand for lean or lightweight software is coming from conventional business application users, the ones who first signed up for mainstream — and now bloated — apps from IBM, Microsoft, Oracle and other major software vendors. Many took the path of least resistance, as in following the safe IT path that spawned the saying: “Nobody ever gets fired for choosing IBM.”  

While major software vendors piled on features that add complexity and can foster customer lock-in, in came the lean software approach of Linux and open source developers. 

“Open source is the driving factor for lean computing,” said Rymer. “Now people can replace conventional application servers, for example, with open source application servers and get lower cost and more innovative features. People are comfortable with open source software, which is now in its second wave of adoption.”  

Agile development is another popular path away from traditional “big-bang” software development.  

“Agile development is independent of any technical platform or development approach,” said Rymer. “It’s a method. What’s neat about this is that people are delivering application features sometimes every two weeks or each month. Rather than deliver the big-bang project after years of work, they’re delivering applications in increments. They’re working in an incremental fashion to deliver features over time, providing value quickly and continue to add value. That’s a way to modulate your costs, to spread your costs and investment over a period of time.” 

This Forrester report’s recommendations advise software pros to update their application platform and tools strategies. How do your tools and platforms fit with the lean software approach? Right now, many organizations are working with platforms that are too bloated and nonproductive, Rymer said. 

“A lot of shops adopted J2EE, and they’re really struggling now to keep up with the demand for new applications. It’s not a real productive environment. Just coding things up in Java takes a lot of time.” 

The era of one-development-platform shops should be over, Rymer said. 

“If you have a variety of application scenarios, don’t assume you have to adopt one platform to do all of them. There are a variety of tools now. People who choose to use Spring (Application Framework), for example, are oftentimes using it alongside their J2EE. They can run the Spring on an extra app server. So, it’s not like there are these hairy choices that force you to throw away what you’ve got. Pick the right tool for the job, and if you’re smart about it you can integrate these things.”  

Rymer suggested that the PHP framework is built for speedy development. It’s now “a real framework and not a collection of modules,” he said. “You can build certain Web applications very quickly, much more quickly than you can with either conventional .NET or Java development.”  

Don’t think that lean computing is a movement to oust established vendors, Rymer noted before we signed off. Remember that even Microsoft is involved in the second wave of open source development tools adoption. “If you want to use Ruby or Python in the .NET world, you can,” he said. When change is driven by developer and business IT pros, big vendors like IBM, Microsoft and Oracle will join in. 

December 10, 2008  4:03 PM

Security boost for LAMP stack

Colin Smith Colin Smith Profile: Colin Smith

LAMP, an open-source Web development platform based on Linux, Apache, MySQL, and PHP, is getting some added protection from attacks thanks to Metaforic.

Metaforic, a provider of anti-tamper solutions, announced that upon request it will provide free versions of secured Apache and MySQL to enterprises. Utilizing a lightweight version of MetaFortress, Metaforic will provide anti-tamper protection and continuous integrity checking for critical parts of the LAMP stack to help defend against multi-vector attacks designed to discover and exploit the weakest point of an organization’s server infrastructure.

The move is important because as more enterprises deploy open-source technology, cybercriminals will target the security vulnerabilities within that infrastructure. And those criminals are looking for any weak spot, whether it’s the operating system, Web server, database system, or the application layer.

MetaFortress Open is specifically targeted at network infrastructure applications. Like the flagship MetaFortress, Open is an anti-tamper solution that inserts security and integrity checks into an application’s source code to prevent against hacking and unauthorized usage.

December 9, 2008  7:13 PM

Recession causing software developers to rethink processes

Colin Smith Colin Smith Profile: Colin Smith

With the recession weighing down on all of us, I’ve heard a few people talk about not letting this crisis go to waste. Ryan Martens in his column last week said now is the time for companies to take a close look at their development processes and make changes that will reduce costs now as well as in the future.

HP Software is also talking about taking advantage of the crisis. Mark Sarbiewski, director of product marketing at HP Software, said the company recommends leveraging the crisis to do three things:

  1. Get control of IT spending — Determine priorities and eliminate low-priority things.

  2. Put solutions in place now that allow you to centralize, eliminate redundancy, and maximize your experts

  3. Drive through process change and automation — Standardize on best practices and automate, focusing on the development process and operations.

To help companies accomplish those things, HP Software today announced two significant products: HP Quality Center 10.0 and HP Universal Configuration Management Database (UCMDB) 8.0.

“We look at Quality Center 10.0 as the heart of how you can change the software development life cycle,” Sarbiewski said. “It includes requirements management, test management, and defect management all in one place.”

At any time you can see how you’re doing against the software requirements. Additionally, HP has improved the ability to share things between projects.

“In Quality Center 10.0 we’ve expanded beyond the simple project model and can promote processes to all projects and share things with all projects throughout the entire cycle,” Sarbiewski said.

Quality Center 10.0 also integrates with HP’s other testing solutions.

To help on the operations side, HP Universal Configuration Management Database (UCMDB) 8.0 can help organizations continually track how everything connects and manage change across all the tiers.

“We’ve now integrated this dependency map into all those systems. I’m monitoring the parts of the biz service and all the pieces support it. I can automatically notify if I see something going wrong in any piece,” Sarbiewski said. “We’re moving from being reactive to being predictive.”

December 4, 2008  5:06 PM

Praising unit testing

Colin Smith Colin Smith Profile: Colin Smith

A few weeks ago I wrote about how many programmers don’t consider unit testing a priority. Reasons given:

  • They don’t know about it
  • Good unit tests are hard to write
  • It’s a waste of time and productivity
  • Writing the tests would take too long (especially if they’re doing frequent iterations)
  • Regression testing is more effective

Since writing that editorial, a few people have spoken up in favor of unit testing, saying it must be a priority.

Ralph Perry wrote, “Without an effective unit and integration test process, my experience is QA/system test becomes a dumping ground, code/build/dump with frequent loads just to get to a stable testable product.”

While Jaideep Khanduja wrote, “A lot of flaws or shortcomings of the product can only be tracked only through unit testing.”

This past week Kevlin Henney, an independent consultant and trainer based in the UK and a frequent contributor to, added to the discussion with his article, “Making unit testing a priority.” Henney says there is an expectation that programmers do some sort of testing of their own code. The key is that they must write good unit tests, and doing so takes practice and skill. Bad unit tests “can be worse for a project than no unit tests at all,” he said.

But saying you won’t run unit tests because it’s hard to do well is “a curious and somewhat dubious justification,” he said. Instead, make an effort to improve your skills. Your projects will be better off for it.

December 4, 2008  3:38 PM

Software simulation tool integrated with IBM’s requirements product

Colin Smith Colin Smith Profile: Colin Smith

iRise Connect for IBM Rational Requirements Composer will soon be available. This integration, built on IBM’s open Jazz technology platform will make high-fidelity iRise visualizations instantly accessible from within IBM Rational Requirements Composer.

This integration is designed to eliminate wasteful cost overruns and delays by ensuring IT organizations are documenting and tracking the right business needs the first time.

The iRise solution gives business analysts and project managers the ability to build working simulations of software before development begins. (Read “Simulation software a cure for hospital’s requirements validation ills” to learn how one customer uses the product.)

IBM Rational Requirements Composer is a collaborative toolset that provides the ability to visually capture requirements information as process sketches, storyboards, user-interface sketches, and rich text to better articulate and communicate the context of requirements.

The combination of the two products gives requirements professionals the ability to embed live, high-fidelity software visualizations directly into the Requirements Composer product by leveraging iRise SmartView. Business analysts, business stakeholders, developers, projects managers, and other IBM users can interact with “live” visualizations and fully experience simulated pages, scenarios, and masters directly within the Requirements Composer environment.

The visualization assets are then published in real time from iRise to the Requirements Composer repository and can be linked into the web of requirements artifacts.

For more information, visit iRise’s website.

December 1, 2008  3:05 PM

IBM software quality tools help organizations collaborate, reduce risk and costs

Colin Smith Colin Smith Profile: Colin Smith

Following up on its June announcement to release 20 products for its Jazz platform, IBM last week announced two new products — Rational Quality Manager and Rational Test Lab Manager.

Rational Quality Manager is a collaboration hub that includes involvement from the business side down through development and testing. “It streamlines the development process to make sure requirements are met and that they’re quality requirements,” said Scott Hebner, vice president of offerings for IBM Rational.

By ensuring all relevant members of the workforce are in sync and have access to data in real-time, a company can more easily make informed decisions, better assign and utilize their resources, and react quickly to changes in the marketplace at a lower cost, IBM said.

“It provides a more defined process for how people work together and produce software,” Hebner said.

You determine everyone’s roles and the process you have to go through, and then decide the policies, procedures, and who can make decisions. You don’t move on in the process until the policies are met, he said.

Additionally, any communication between people and documentation that results is stored and becomes part of the workflow. That data is updated in real-time as changes are made.

“It provides more real-time updates and data,” Hebner said. “It will significantly lower the cost and risk of shipping poor quality software.”

Another new tool is Rational Test Lab Manager. Feeding off of the Quality Manager, it automates configuration of all the test machines. “This will help improve test lab scheduling and help them better utilize their resources,” Hebner said.

In economic times such as these, when companies are looking to cut costs and inefficiencies, Hebner said tools such as these will help. They will become more efficient and will reduce risk.

“As customers look at cost reductions, they’ll see inefficiencies in their IT department and will need to improve that,” he said. “These products are the kinds of offerings they’ll hopefully turn to in order to reduce costs and become more efficient.”

In related news, IBM is launching several other new releases of products within its Quality Management Portfolio, including IBM Rational Application Performance Analyzer, IBM Rational Functional Tester, IBM Rational Quality Manager Express, IBM Rational Performance Tester, IBM Rational Service Tester for SOA Quality, IBM Rational Test RealTime, IBM Rational AppScan Tester Edition, IBM Rational RequisitePro, IBM Rational Measured Capability Improvement Framework Assessments, Telelogic Rhapsody TestConductor, and IBM Rational Requirements Composer which is expected later this year.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: