I’ve been reading a fascinating book by Andrew Jaquith, Security Metrics – Replacing Fear, Uncertainty and Doubt. This book takes lots of buzzwords, like “ROSI,” “Risk Management,” “key indicators,” “accountability,” and “compliance,” and turns them on their heads.
It has always bothered me that IT security and IT audit pundits and promoters propose all sorts of theories masquerading as fact for assessing risk. Everyone has a different unit of measurement, including some very large standards organizations. This is simply an attempt to justify the cost of securing data. It has always bugged me because I have yet to see a good explanation for measuring events that have not happened. If there is a solid security architecture, Bad Things don’t happen. Mostly. How to get this across in measurable terms is deplorably difficult to the non-IT parts of the business (usually management).
We’ve been reduced to using “compliance requirements” to justify the cost for “security initiatives” across an enterprise, and that limits their applicability to what the regulations require, rather than basing our efforts on solid evidence for security improvements. Measurements and quantification just do not exist. (Gasp! Heresy, I know.)
How do we differentiate between an organization that has no security incidents because of their solid security practices, and an organization that has no incidents due to blind, dumb luck? Or my personal favorite, no incidents because they don’t have any way to even know if such incidents occur? Yes, we’re fine because we have no idea.
Jaquith does a great job of picking apart the BS Bingo, especially flashy terms used by vendors, who must continually sell you something to stay in existence. (When did true improvement turn into the next release?) If you run a Google search on “compliance,” there are 133 million results. Try the same query minus “.com,” and the results fall to a measly 12 million or so. No wonder most of our security spending has gone to product, not process. Companies have turned to compliance as a metric for good security.
Yes, we have no real idea what constitutes good information security practices.