Based on an analysis of 745 applications submitted by 160 organizations in 10 industry segments and representing 365 million lines of code, CAST calculates it costs businesses millions of dollars to fix technical debt — and companies are not budgeting for it.

“The findings revealed an average technical debt of $3.61 per line of code,” said Bill Curtis, CAST’s chief scientist and senior vice president of CAST Research Labs.

That debt adds up: Nearly 15% of the applications examined by CAST had more than a million lines of code. Just like the kind of debt that weighs on many of us 99%-ers, technical debt incurs interest as the violations go unfixed, so it just gets bigger and bigger over time. Research house Gartner predicts global technical debt will reach $1 trillion by 2015.

Notable findings in the CAST report:

• Java apps, accounting for about 45% of the study sample, scored lower on performance and carried more technical debt than apps using other languages — $5 per line of code compared with the average $3.61.
• COBOL apps (yes, these monsters are still around) scored highest in security. They deteriorate in quality as they get bigger, however, unlike their less secure but more modular, newer relatives, Java EE and .NET. (.NET apps scored lowest on security.)
• Structural defects were equally prevalent in outsourced apps and those developed in-house. This finding might be skewed, however, by the fact that most outsourced apps were developed in-house originally before being farmed out for maintenance, Curtis said.

“Even though we have known for two decades that things like cross-site scripting, SQL injection and buffer overflows are huge opportunities for hackers to break in, we still see those things in the code; and that is a huge problem,” Curtis said. “The problem is that you don’t always know which violation in the code is the one that is going to cause the outage or offer a hacker the way in.”

But you do know it’s going to cost millions to fix when it happens.

This leaves CIOs between a rock and a hard place when it comes to managing the risk of technical debt. You can’t fix everything — and you don’t want to, Curtis said. What CIOs need to identify are the most severe violations that carry the highest cost for the maintenance of the system or have the highest risk to the business — “for an outage or data corruption or a security breach or performance problem” — and then go fix those.

Consolidation is, of course, nothing new. Fifteen years ago, you might have been a DEC VAX/Alpha shop running RDBMS and Informix, with all sorts of now extinct middleware.

But today’s context is different. We’ve long left behind “Nobody ever got fired for buying from IBM” for a much more uncertain and free-wheeling sourcing scene. CIOs have gradually imposed order on the situation through standards committees. At some companies, standard means things like Java, XML and 10 Gb Ethernet. At other companies, it means Dell, Microsoft and Cisco. At most, I suspect, software and architecture standards are a mix of protocols, open standards and vendors.

During this period of the rise of standards committees, CIOs have also had to confront a different problem: the dreaded “too many vendors” dilemma. “One is too few, three are too many” is conventional wisdom these days. Nice for hardware, not always possible for software.

The Sun acquisition could bring hope or despair into many CIOs’ lives, depending on how it plays out. Java, Solaris, SPARC boxes — these are significant building blocks for many shops. Some CIOs may cheer the continuing expansion of Oracle into some more niches in the upper end of computing; others may cringe at the thought.

But how many CIOs, I wonder, are rethinking the whole concept of “strategic vendor” in light of this deal and the sense that more are to come as the tech industry deals with the down economy? How are you seeing this? Are you looking for comprehensive vendors that can supply it all because they’re safe or for specialized vendors that each dominate a category (think EMC)? Or are you looking to commit to standards that are widely implemented by many vendors and to buy implementations that offer a good tradeoff between standard and “enhanced” features?

Another legitimate question posed by the dissolution of such a rock of many IT architectures as Sun is whether it’s time to apply new thinking to choosing strategic vendors — techniques like risk analysis or perhaps an even newer paradigm.

Legend has it that Wild Bill Hickok always sat in the back corner of the saloon so he could observe everyone who came and went. Good advice these days, although it didn’t quite work out for Wild Bill.