In the days since that conversation I’ve put some thought into the frameworks because in the end the aforementioned CISO was committed to finding the NIST experience and eventually did.  But what did that really mean?  Having fairly recently had the occasion to have both NIST 800-53 and the ISO 27000 documents  in front of me it was striking how similar they both were with only a few obvious distinctions to be made between the two.  Essentially the differences reflected more on the cultures that created them than the risk factors they were focused on (NIST = U.S.A and ISO = European).  But information technology architectures fundamentally are identical the world over so despite formatting and spelling they both are addressing the same challenges whether or not they realise it. And for those of us who have familiarity with both, to know one is to know both, even if those who are committed to either one disagree.  If you’ve worked on audit/assessment projects leveraging ISO 2700o material you’re immediately qualified to work on projects using the corresponding NIST framework and vice versa.   And if you have experience working with PCI standards guess what?  You can pretty much step in and work with either NIST or ISO content (except of course you have to expand your sights to include the entire infrastructure, not just on whatever touches PAN data).

My preference is that we would consolidate globally into the ISO frameworks where applicable and maybe even fit that in to the SSAE 16 process.  I’ve read enough toothless SAS 70/SSAE 16 reports to know that it’s easy enough to rig the system to your advantage.  And unless you’re a government agency that has to comply with NIST there’s little meaningful value to using NIST whereas being ISO 27000 certified carries a great deal of weight within the audit/assurance community.  Plus there’s the added benefit of having InfoSec practitioners all getting trained and practiced at both building out ISO 27000 compliant solutions and also knowing how to test the related controls.  Think about that, a single global security standard regardless of where you enter into the profession.  Having run a few practices in my career and way more than my fair share of engagements I can tell you that has great appeal.  Plus it would help eliminate awkward dialogues where my sixteen years of real and relevant experience is at least partially marginalized because it hasn’t all been with one particular standard.

Ultimately in the end a frameworks only meaningful advantage is that it theoretically ensures consistency in how controls are identified and assessed.  If you have someone who knows a framework but doesn’t really understand the details within that sort of defeats the process anyway, no matter how robust or thorough it may be.  Perhaps that’s why I consider it a non-issue when it comes to which frameworks a practitioner has used.  I’d much rather work with someone who understands the technology and has a good feel for the details rather than someone who knows that SDLC is addressed in SA-3 for NIST or Section 12.5 for ISO 27002.  But than again, I’ve always been more concerned with real risk, not perceived risk so this shouldn’t be surprising to anyone who’s read my content in the past.

A security framework by any other name would be just as comprehensive, you know what I mean?

I’ve personally reviewed and/or audited somewhere close to fifty business continuity/disaster recovery (BCP/DR) plans over the past decade.  I’ve also written or edited several of those as well in the past five years since moving into professional services for financial institutions.   Furthermore I’ve participated in roughly a half-dozen tests while still working within the infrastructure during the first part of my career.  Suffice to say I have at least an informed opinion regarding the viability of any such BCP/DR strategies.

Fundamentally there are a few varieties of  BCP/DR plans:  Those that are current and viable, those that convince your examiner that it’s current and viable and those that may have been viable years ago but bear no resemblance to your current business profile.  And beyond those there’s the worst of BCP/DR realities, the non-existent one.  But really in the end what your current state of preparedness comes down to is this – either you’re ready for an event or you’re not.   And in the past forty-eight hours that’s been made abundantly clear in the form of how many of my clients affected by Hurricane Sandy have navigated through what’s now clearly one of the worst weather events in my lifetime.

Around noontime yesterday (October 29, 2012) as weather conditions worsened and major metropolitan areas were literally shutting down for business I started checking up on a few clients.  The first thing I did was visit the website of every client that my practice has assisted with their BCP/DR strategy – each of them had updated their website to announce that branches in the affected areas were closed.  Some had a pop-up window with the update, others had a message displayed in either bright red letters, bold font or both.  As a standard design consideration each of them also had phone numbers clearly displayed and when I called a sampling real people answered and were available to assist me.  I inquired of a few of them where they were physically located and they were all located remotely and not on site in affected areas (much to their credit they were reluctant to share too much information).   The second thing I did was visit the website for a few clients whose BCP/DR plans were tagged during an audit/assessment as either being deficient or missing.  The websites were not updated and in all but one case I only learned that they were closed for the day after calling into a branch (one had an 800 number that was redirected to a real person).

Now I know this wasn’t a very deep or meaningful test of anyone’s ability to continue operations in the event of a disaster.   But what it did prove is that those institutions who had plans that were current and whose management team knew to rely upon had already thought through the little things that make a difference.   Someone knew to update the website, management knew to reroute calls away from unmanned branch locations.  I can only assume that the appropriate parties desginated to do so also contacted their regulators to inform them of their closing and that a phone chain was initiated informing staff thus keeping them off the roads and safe.  And because an important part of the plan creation/update process is both training and testing stakeholders are able to navigate through the decision tree and take appropriate related steps without having to think through it – one of the biggest challenges confronting management during a crisis.  The very best part of having a viable and current plan is that all the thinking has been done in advance and has been reviewed and validated which greatly reduces the chances that something (or someone) will be missed.

Here’s a sanity test:  If you didn’t know exactly where to begin the decision-making process or who to engage you’re in need of a new plan.  And if you did know but can’t be absolutely certain that others would be able to do the same in your absence, you’re in need of a new plan.  One of the rebuttals I’ve heard all too often when identifying a deficient or missing BCP is that management knows what to do should some manner of disaster strike.  That may be true but what happens if key people are unavailable or can’t be reached?

Seriously, when something like Hurricane Sandy occurs it’s the best time to consider how you’re institution would fare when navigating such an event.  Block off an hour within the next week with your key people, pull out your BCP/DR documentation and try and step through how you’d handle things under similar circumstances.  In a very short time you’ll gain a sense of whether or not you’re prepared and if necessary afford you the opportunity to improve.

Trust me on this – you don’t want to be in the middle of a disaster scenario and find out that your plan doesn’t work.

Truth be told, while I’ve spent somewhere near seventy-five percent of my time over the past ten years working for financial institutions I’ve also done a fair amount of work for insurance companies, mostly centered on SOX with occasional diversions into general risk assessment work.  The drivers in the insurance industry are different in terms of oversight and requirements and so the volume of work isn’t nearly the same.  But that by itself begs a question: Why isn’t the insurance industry as regulated as financial institutions?

I’ve now done major audit and assurance work for financial institutions, insurance companies and health care providers and for most of them the risk profile is almost identical in terms of non-public personal information.  So why isn’t the level of scrutiny equal across all three of them?  While some might start spouting about how it is, about how states routinely audit insurance companies and how the health care industry has to comply with HIPAA the truth is that banks and credit unions are held to a much higher degree of accountability than any other vertical.  Why is that?

I’m fond of routinely, almost incessantly beating the drum about how it’s all about the risk.  I get my initial client opportunities because I have a deep resume with relevant experience but I generate repeat business because I tend to whittle things down to what matters most both to my clients and to their oversight providers (auditors and examiners alike).  Compliance exists because risks need to be addressed – if the risks aren’t credible or likely the work should be adjusted to reflect that.  But where the risks are real they’re really real.  The type  of data shared with an insurance company is in many ways even more sensitive than anything shared with a bank and most of what’s shared with insurance companies is also shared with health care providers.  Yet there’s no true Federal oversight for the insurance industry and HIPAA is about as much of a toothless tiger as anything I’ve ever encountered.

I recently completed a boatload of documentation to get my family on a new health insurance plan.  I turned over every piece of sensitive information I have for every member of my family minus my bank account information because that’s what was required.  I had to provide all of this online and follow that up by sending them an impressive array of hard-copy documents with even more sensitive information that should never be kicking around in the public domain.   In the past I’ve also been required to provide my bank account information because one plan in particular would only provide coverage if they could automatically deduct monthly premiums via ACH drafts.  So now the insurance industry has access to it all; name, address, social security number, date-of-birth, maiden name, medical history and banking information.  And yet there’s no true oversight agency that’s responsible for making sure they’re protecting all of MY information.

To compound my frustration, of the four insurance companies I’ve conducted work for since 2006 (two of which are Fortune 5oo’s) exactly none of them have something akin to a Chief Information Security Officer.  They all have risk people focused on the business side of things (because that’s necessary to protect profitability) but that’s it.  There’s typically an information security manager who’s part of the infrastructure team but who almost never reports right into the senior-most technology person (e.g. CIO, CTO).  Any audit work that occurs is coordinated across multiple IT managers and on rare occasions there will be an audit/assurance manager.  However in the one example I personally know of where that position exists the person in the role was really just a converted IT manager who obtained a CISA designation – no fundamental audit or assessment experience.

The question has to be asked:  Why is it that banks and credit unions are heavily regulated regarding protection of non-public personal information but other industries with similar risk profiles are  not?  Why aren’t insurance companies required to comply with FFIEC-type guidance?  Why isn’t there a Federal regulatory agency that is responsible for keeping an eye on the insurance industry the way the FDIC, OCC, FRB and NCUA do so for their financial institutions?  And trust me, whatever oversight exists for the insurance and health care industry is largely ineffective.   Why is my sensitive information considered more at risk within a banking infrastructure than it is within an insurance infrastructure?  Having been on site for both and examined their internal controls  I can’t answer that question, that’s for certain.

Now I’ve designed and supported more than my fair share of related content.  I understand that sometimes the best way to tell a story is to paint it in the form of a picture; I get that part.  But way too many times I’ve witnessed such initiatives spiral out of control to the point where it takes an army of people working ridiculous hours to pull together a deck of metrics that either fails to answer anyone’s questions or, even worse, generates requests for more metrics to provide clarity.  And once a metric becomes a standard part of any reporting package it often stays there until management changes, and sometimes even beyond.

But I think there’s a bigger issue with metrics that exceeds my simply not thinking they’re “all that and a bag of chips”.  Where are the controls around generating them?

Seriously, we have this vastly complex framework wrapped around financial reporting (SOX) to provide reasonable assurances that what management is reporting to its investors is accurate.  We have industry, federal and state legislation requiring all manner of controls around sensitive information.  There are auditors (internal and external) and regulators from all over the place that comb over everything with a fine tooth comb (or at least claim to) to make sure everything being done is done right – but in my nearly fifteen years in the audit/assurance industry I have never heard of a finding or issue regarding the veracity of metrics.  Which is only a problem if the people running an institution or company rely on them to make decisions.

The reason why it’s a problem is because so much of the metrics out in circulation is pulled together from disparate sources, cobbled together in spreadsheets or non-production databases and manually generated.  There’s no easy way to verify the source data, or know that it’s unaltered in any way or even know if it’s the right information.  And even if the data source used is from a secured production-like environment, still there’s no real auditing conducted to ensure the information is accurate or better yet, is even the right information.

I once took over a change management process and assumed responsibility for a series of reports that were generated for the Managing Director who in turn used that as part of his reporting package shared with the CIO.  One of the key metrics being reported on was scheduled releases and the IT departments on-time implementation percentage.  The numbers looked great showing that they were on-time more than ninety-five percent of the time over a two year period.  The only problem I could see with the metric was that it was misleading to the point where it was almost a lie.  The scheduled release date was being pulled from the system used to migrate changes into production and that date was only determined once the development team had completed all of their work.  So the scheduled implementation date was chosen once they knew they were ready to move into production.   Of course the on-time numbers looked great, they always knew they were ready before committing to it.  The Managing Director incorrectly assumed that there was a legitimate release schedule with forecasted dates and that the on-time numbers reflected on a well run process; wrong.   No one ever questioned the numbers or their source and had I not inserted myself into what was described as a well honed, efficient process the problem might have never been identified; and there a few more just like it.  My trust in metrics was permanently altered after that.

Metrics represents an excellent way for decision makers to quickly understand status and identify problems.  I’ve quoted here before about how someone I respect quite a bit was fond of asking her team “If you can’t measure it, how can you manage it” and she’s absolutely right.  Metrics is the ultimate management means of measuring key activities and issues within their world.  But how far do you go and how much effort do you expend pulling the related reports together?  And even if you’re able to automate the process and shorten the time necessary to generate the reports, how do you know that you’re either measuring the right things or that the underlying data is unaltered?  Ultimately I think that senior managers should be provided with something akin to a cost-benefit analysis for each metric they’re given.  Have them understand the degree of complexity and the amount of effort required to generate a number before deciding whether or not it’s worth it.  Perhaps I’m being naive but I’d like to think that most C-level executives would eliminate a significant amount of their reporting if they could see how much it was really costing them.

Here’s the part that should really concern you the most though: Metrics is a key component of Board reporting, they make all sorts of decisions based on what these reports tell them.  How can that be allowed unless the process used to generate them is locked down and audited?  Where are the regulators in all of this?

Here’s the sequence of events:

Wednesday morning I received an email alert from a company I use that my automatic monthly payment was declined.  Knowing full well it wasn’t a balance issue I assumed correctly that my bank had cancelled the card.  As I travel extensively and rely on the card exclusively I made my way to a local branch later that morning.  Along the way I called into the service center and confirmed my suspicions, that Visa informed the bank that my card was part of a range of numbers that was possibly exposed via a breach.  I asked if it was possible to learn the name of the offending vendor and was told (same as last time) that Visa doesn’t share that information.  As I am now a two-time victim it’s easy to spot the trend and hard to ignore the possibility that it might have involved the same vendor both times.  It wound up taking three visits to a branch to straighten me out and actually get a functioning card in my wallet.  The inconvenience is more than benign as I use the card in several places and will now need to make manual, one-off payments with the temporary card while awaiting the permanent card so that I can update the affected accounts.  By the time this is all said and done it will have resulted in my exhausting more than a half day of billable time trying to fix a problem I didn’t create.

A few things need to change.

• First, as part of the breach notification the card issuer needs to share with the cardholder the source of said breach.  I’ve been hit twice in six months, there’s a better than even chance that it involved the same vendor and/or processor and I deserve to know if that’s true.
• Second, affected cardholders should receive status updates providing details about the breach including the suspected source, the techniques potentially used and a description of any follow-up actions including investigative and (hopefully) criminal prosecution.
• Third, issuers need to have a better system in place to address breaches.  The fact that I have to overtly take action in order to replace the card is a joke.  I’m a billable resource and taking time out to wait to talk to a customer service representative results in loss of income; I’m being punished twice as a result.  I should have been offered the option to have a card overnighted to me or have been able to receive a card at any teller window and have it activated right there and then (I had to first activate at an ATM before I could use the temporary plastic).  The card replacement process needs to be streamlined.

We collectively as an industry and a society need to accept that both identity and card theft is a mainstream occurrence and adjust accordingly.  Legislation is needed to further insulate the victims (like me) from any extended damage or inconvenience and ensure as smooth a process as possible to allow us to continue living our lives.  Because right now I don’t just feel like a victim, I feel like I’m being punished for being one and treated like I simply don’t matter.

Hey Washington, make the industry tell us what’s going on and to treat the consumers better!

Oh, and PCI Security Standards Council, how’s that framework working out for you?  I’m thinking the only one benefiting from your content are the practitioners making money by supporting it.

Seriously, something needs to change.

Things are looking up a bit because I have a new favorite regulatory agency to follow, the Consumer Financial Protection Bureau (CFPB).  And here’s why:  They focus on things that impact my day-to-day life (and yours as well).

I started tracking what the CFPB was doing about five months ago by accident.  Someone I know who used to be an examiner for the FRB switched over to the newer agency right at its infancy and I noticed this courtesy of a LinkedIn update.  Because I consider the Fed to be the Big Kahuna of the regulatory agencies I was surprised (you don’t leave the Yankees to sign with an expansion team unless you have to, or so I thought).  Compelled a bit by the update I started poking around the CFPB website.  For the first few months of this year it seemed to have potential but was little more than brochure-ware.  But last month that all changed.

The first CFPB update that caught my attention was labeled 12 CFR Part 1070 and it was all about the protection of consumer data, only with a slight twist.  Basically it was all about how any information they received as part of their field work would be protected exactly the same way that any third party vendor would be required to.  Despite their being a Federal agency they weren’t going to hide behind that as a means to simplify their lives.  They spearheaded an update to the underlying regulation that frames their charter so that consumers and their institutions can be assured that all PII and NPPI would be protected.  For me it was a rare win-win topic; protection of PII and NPPI combined with a reference to vendor management (these are a few of my favorite things).  And really for me it was that much more significant because I’ve known of a few situations where representatives of Federal and State regulatory agencies were responsible for the outright loss of confidential and/or restricted data.  Beyond a slap on the wrist there wasn’t much else done to the offending examiner or their agency.  And the affected institution couldn’t really complain too loudly because it’s always a bad idea to challenge your regulators, even when you’re in the right.  So I thought this was all at once a compelling and remarkably sensible update by a regulator, not something I’d expect to see.  That was the first points on the board for the CFPB.

The second set of points were scored almost on the same day.  I wanted to check one of the details related to the aforementioned update and noticed this one “Consumer Financial Protection Bureau report finds confusion in reverse mortgage market“.  Because I have a parent who is a senior citizen and who I think might one day soon be open to at least exploring a reverse mortgage I read with great interest.  The report was in plain English, was oriented in such a way that I could share it with my family and have them understand the issues and concerns detailed within and most importantly it made sense.  Reverse mortgages are growing in popularity and its main audience is the senior citizens segment of society.  Seniors tend to be  more easily misled, they’re under greater pressures to find new money sources (courtesy of our recession) at a time in their lives where going back to work is often not an option.  And because a parent would do almost anything rather than turn to their children for financial assistance they see a reverse mortgage as a way out of their predicament.  So for me having this content available was quite the relief.  I can caution and advise all day and night but the risks presented by a reverse mortgage are much more credible coming from an authorized source.  And so I celebrated July 4th this year by declaring the CFPB my new FDIC (the Sheila Bair inspired version, not the current blah one).

Here’s my really bizarro advice to any of you with even the slightest interest in regulatory oversight; if you haven’t already done so visit www.cfpb.gov and take a look around.  It’s oriented towards lay people, not just lawyers and regulators (and practitioners like me) and addresses topics and concerns that affect the majority of our population.  Basically it’s what I would expect from a regulator that still has that new agency smell but nothing like I’ve come to know from those that preceded it.  To those who have had a hand in defining its charter and organizing its content, great job!   Now repay my kind words by going out and getting me some juicy enforcement stories to write about.

Which begs the question, why bother supporting ineffective controls when they fail to control anything?

I wish it was rare that I encountered similar situations with my clients but it’s not. My favorite ineffective control is the manual visitor sign-in sheet I often find when auditing/assessing my clients physical data center controls. My hosts often make a big deal out of asking me to sign-in before allowing me access to their data center or server room and I typically play along. However, I’m fond of using an alias to see if they validate the information I provide (usually they don’t). The manual sign in sheet falls under the category of “better than nothing” but in its own special sub-category I call “but not by much.” The list is always a bit lite and is often missing sufficient evidence to prove that it’s consistently relied upon. Another favorite of mine centers on production change control. Some of my clients have fairly robust processes to track changes to application software but ask them for evidence of system software updates or hardware configuration changes and I’m met with blank stares as they try and figure out how to tell me they don’t really track those things formally. So you have to wonder why you’d even bother to track some of the changes if you’re not tracking all of them? If something went wrong within a clients infrastructure how would they know if any recent changes might explain it if they don’t know about everything that changed?

Here’s a bit of a radical thought; stop supporting ineffective controls and save the time and effort required to support them.

Seriously, even though a control might appear critical in nature, if it’s poorly designed, poorly supported or just flat out ineffective, just kill it altogether. No decent examiner or auditor is going to be tricked into thinking it’s providing value and it’s likely going to call into question the validity and reliability of all your other (hopefully) effective controls. If you feel strongly that the control needs to be in place and doing its job than do something about it. Either redesign things so that it’s viable and effective or scramble like crazy to identify compensating controls that render the control unnecessary.

We live in an age where compliance rules all. There are all manner of controls that are required in order to satisfy our oversight agencies and auditors and that’s a list that will only continue to grow. No one has the luxury of wasting time or the precious few resources they have to work with and so it’s that much more critical that these things be thought through and validated. Expecting people to support control related activities that ultimately fail to satisfy their objective is flat out wrong. And because this is the age of regulatory enlightenment those who toil within the financial services industry are a bit more savvy about how these things work. They have an idea of whether or not what they’re being asked to do makes sense and will resist or defer participating if they think it’s a waste of time. The only thing worse than an ineffective control is one that’s poorly supported.

It’s why I often wonder what would happen if I simply drove across the lawn closest to my Mom’s building and completely avoided the main gate. I’m thinking that if it’s after sunset when there are no golfers walking the links I could probably pull it off. Of course I’d have to deal with the compensating control of an angry mother once she figured out what I did but perhaps, just to prove a point it might be worth it.