Right after one of the debates I found myself knee deep in a debate about Dodd-Frank.  A close personal friend of mine, a very bright bulb who I’ve never found a reason to disagree with brought up Dodd-Frank as an example of horrible legislation that’s crippling banks and contributing to our horrible economic conditions.  Whoa, whoa, whoa…. rail against taxes, complain about government spending, assail the current administration for the dramatic escalation of our national debt.  But leave Dodd-Frank out of it because that’s not one of our bigger problems.  I can offer a five thousand word defense of the best parts of Dodd-Frank without even pausing to organize my thoughts but I don’t need to go that far.  I can sell it’s virtues in a single, simple sentence:  Any legislation that created the Consumer Financial Protection Bureau is instantly more effective than anything that’s come before it in my lifetime.

No, seriously… in my lifetime.

I’ve already screamed from the rooftops about how much I like the CFPB.  In my own geeky, nerdy way I’m proud to admit that I look forward to getting their regular updates and announcements because they always seem either ridiculously relevent or illuminate how they’re hot on the heels of yet another predatory business practice.  In barely a years time they’ve pushed deeper into the heart of the issues that crashed Wall Street in 2008 than anyone could have hoped (that’s my opinion but one I’m willing to defend).  And their examiners appear to be freaky efficient.  I’ve been hearing from our banking clients that they’re drilling in on details and covering more territory than was expected and that they’re discussing issues much closer to protecting customers (and members).   Our practice recently issued a bulletin to our clients alerting them to the fact that CFPB examiners are expecting related oversight to be pushed down to external business parters and vendors.  This is not a new consideration, it’s exactly the same as what’s supposed to happen with regards to GLBA (and one of the reasons we developed our related software and services for same) but still, we anticipated this would take several exam cycles to surface.  CFPB cut right to that chase in a heartbeat, which is stunning for such things.  It’s almost like someone told them where to look and what to look for which to a certain extent is true.

The CFPB didn’t start as most new agencies do.  They didn’t recruit green examiners and place them under the management of a few practiced hands.  What they apparently have done is to hire well seasoned examiners from related regulatory agencies (e.g. FDIC, FRB, OCC) have them contribute to creating the necessary procedures and then send them out to bring it all to life.  So on Day One they already know where the bodies are likely to be buried and what to do about it.  It’s brilliant, it’s efficient and it’s the very best example of  your government doing its job.

Here are some snippets from my in-box:

• Regarding the three main credit reporting agencies, the CFPB released a report that said “Among the key takeaways in the report, which is one of the most comprehensive studies of credit reporting to date, are that credit card history dominates the information in credit reports and that debt collection items  generate the highest rate of disputes”.  This becomes important for consumers who are trying to either establish or repair respectable credit ratings.  The news release further explained about the report that it “will help educate regulators and consumers about how this important industry works,” said CFPB Director Richard Cordray. “If consumers know how these companies handle their credit histories, they can make better decisions on how to handle their financial lives.”
• This was another headline “CONSUMER FINANCIAL PROTECTION BUREAU HALTS ALLEGED NATIONWIDE MORTGAGE LOAN MODIFICATION SCAMS”.  The news release explained that the CFPB is  “taking on schemes that prey on consumers who are struggling to pay their mortgages or facing foreclosure,” said CFPB Director Richard Cordray. “We are especially concerned with those who misrepresent government programs or websites to divert distressed homeowners from needed assistance.”
• And even still, another headline “CONSUMER FINANCIAL PROTECTION BUREAU PROPOSES ALLOWING COMPANIES TO RUN TRIAL DISCLOSURE PROGRAMS”.  And while this may seem dry to so many not close to the related issue this is signficant because right now most of us ignore all the small print.  The CFPB is trying to figure out better ways to present disclousre information so that us consumers both think to read it and, more importantly, understand what it’s telling us.  Rather than try and stuff a once-sized-fits-all solution down the industries throat they’re opening it up and authorizing institutions and lenders to explore different approaches.

And the kicker about these three items?  This was all issued this month (December 2012) and we’re not even quite halfway through it.

Which is why I don’t much care for any manner of compliance-based assessments that are self-administered.

Companies have had this crazy notion for more than a decade now that the best way to identify and address risks inherent within the infrastructure is to ask key stakeholders a somewhat generic set of questions and use their responses to figure out what’s what.  Most of the time the people driving these initiatives are either information security professionals or corporate compliance people who either believe they already know where the problems are or are looking for the simplest and easiest way to satisfy some requirement.  But what they often fail to grasp is that it’s almost impossible to draft a common set of questions that either apply to the vast majority or worse, will be interpreted consistently across the stakeholder population.  Plus the perceived benefit of using a self-assessment approach to reduce effort and required support resources is almost always an illusion.  Most of the time saved in not having someone ask the questions and record the answers is instead consumed by needing to explain the format, explain the questions or trying to clarify and clean up the responses.  While supporting one such program recently each assessment required a kick-off meeting, a follow-up meeting to review the status of the assessment, a third meeting to review the initial draft of the questionnaire, a fourth meeting to review the resulting report(s) and a largely untracked number of hours to help generate all of the related support documentation.  Regardless of the size of the entity being assessed each one consumed somewhere close to eight hours.  While that might seem like a scary large number, the really scary part was that based on which risk analyst was responsible for the assessment and the personality/mindset of the stakeholder completing it the results looked very different from one another.  It was almost impossible to generate meaningful metrics across the assessment population because a “Yes” answer for one question might mean the same as an “N/A” in another; there was no way to know that.

Another issue I’ve always had with the self-assessment approach is that while some stakeholders take it seriously and do a remarkably thorough job, others race through it with little hesitation just to fill in the blanks and get it off their desk.  Sometimes you can detect which is which, sometimes you can’t.  Plus the approach fails to capture much of the rich and relevant information related to each question and the underlying risk behind it.  I recall conducting a team-driven risk assessment years ago where one stakeholder after the next covering a very broad sampling of the infrastructure kept lamenting on the lack of a proper disaster recovery plan.  They had something to show auditors/examiners but to a person no one believed it was a truly viable plan.  All but the CIO brought it up as a concern and when pressed a bit about why that was they all shared a common concern: If their main office was closed unexpectedly for twenty-four hours, regardless of the reason, they were likely out of business.  A related self-assessment question would ask “Do you have a current and recently tested DR plan?” – most respondents on that engagement would simply have selected “Yes” and moved on to the next question without ever being challenged to share their concerns.  Where’s the value in having a repository of questions and answers when it fails to capture the true essence or dimension of risk? 

And the biggest issue I’ve always had with self-assessment questionnaires and their related templates is that they’re so often poorly designed.  I can guarantee you that each of them has at least one question which makes zero sense to anyone who reads it.  They either answer it based on what they think it’s asking, answer with an “N/A” or require follow-up with the people managing the process to have it explained.  And you’d be amazed how many times even the author is challenged to provide a meaningful answer (including this guy).  One thing’s for certain, a self-anything needs to be designed and written so that everyone understands what they need to do without having their hand held.  Plus it’s rare that questionnaires are customized so that each stakeholder is only asked those questions that truly make sense.  An application owner should never be asked if their anti-virus solution is current and up-to-date.  A business process owner should never be asked about software change management.  Yet seldom have I encountered a self-assessment process which does anything like this and so the audience is burdened with time consuming yet unnecessary questions.

Really though in the end my overriding problem with the self-assessment approach is that it fails to capture the expertise and guiding hand of true risk and assurance people.  The process is often supported by analysts who don’t really have a feel for conducting assessments and are satisfied that all of the blanks are filled in.  I have a nose for when there’s something beyond a simple answer and know when to scratch at the surface to bring it to light.  By not allowing expert hands to guide the process potentially huge amounts of valuable and possibly critical details are being missed thus undermining any perceived value of the process.  When you consider that all tolled and tallied the self-assessment approach versus the guided assessment approach doesn’t really save you much time (if any) and that it results in a weaker finished product, why would you elect to use it?   One answer is that regulators push for it because perhaps it’s better than nothing (I can’t get any of those I know to comment).  Another is that the people sponsoring these initiatives lack the fundamental comprehension to understand their options and chose what they perceive as the less complicated approach (again, I don’t know for sure it’s just a theory).  What I do know is that when done right a risk assessment is managements best friend, a fundamental belief behind the recent spike in ERM activity.

While recently having my car serviced the mechanic discovered a nest of some sort in the engine block, he thinks it was probably squirrels.  Because of this discovery he went searching for all the wired connections to make sure they weren’t chewed up and destroyed, quite a few were as it turns out (the car had been idle for several months).  The bill only added the cost of the replacement wires but nothing significant for the time it took to first find which were affected and then replace them.  Had I attempted the repair myself I might have noticed the nest and likely would’ve cleared it but know for certain I never would’ve thought to check the wires, where to look for them or what to look for.  I was smart enough to rely on a professional with a nose for that sort of thing and it saved me time, money and best of all the aggravation of having the car break down somewhere unexpectedly.  Good thing I didn’t go the self-repair route.

Now I’ve designed and supported more than my fair share of related content.  I understand that sometimes the best way to tell a story is to paint it in the form of a picture; I get that part.  But way too many times I’ve witnessed such initiatives spiral out of control to the point where it takes an army of people working ridiculous hours to pull together a deck of metrics that either fails to answer anyone’s questions or, even worse, generates requests for more metrics to provide clarity.  And once a metric becomes a standard part of any reporting package it often stays there until management changes, and sometimes even beyond.

But I think there’s a bigger issue with metrics that exceeds my simply not thinking they’re “all that and a bag of chips”.  Where are the controls around generating them?

Seriously, we have this vastly complex framework wrapped around financial reporting (SOX) to provide reasonable assurances that what management is reporting to its investors is accurate.  We have industry, federal and state legislation requiring all manner of controls around sensitive information.  There are auditors (internal and external) and regulators from all over the place that comb over everything with a fine tooth comb (or at least claim to) to make sure everything being done is done right – but in my nearly fifteen years in the audit/assurance industry I have never heard of a finding or issue regarding the veracity of metrics.  Which is only a problem if the people running an institution or company rely on them to make decisions.

The reason why it’s a problem is because so much of the metrics out in circulation is pulled together from disparate sources, cobbled together in spreadsheets or non-production databases and manually generated.  There’s no easy way to verify the source data, or know that it’s unaltered in any way or even know if it’s the right information.  And even if the data source used is from a secured production-like environment, still there’s no real auditing conducted to ensure the information is accurate or better yet, is even the right information.

I once took over a change management process and assumed responsibility for a series of reports that were generated for the Managing Director who in turn used that as part of his reporting package shared with the CIO.  One of the key metrics being reported on was scheduled releases and the IT departments on-time implementation percentage.  The numbers looked great showing that they were on-time more than ninety-five percent of the time over a two year period.  The only problem I could see with the metric was that it was misleading to the point where it was almost a lie.  The scheduled release date was being pulled from the system used to migrate changes into production and that date was only determined once the development team had completed all of their work.  So the scheduled implementation date was chosen once they knew they were ready to move into production.   Of course the on-time numbers looked great, they always knew they were ready before committing to it.  The Managing Director incorrectly assumed that there was a legitimate release schedule with forecasted dates and that the on-time numbers reflected on a well run process; wrong.   No one ever questioned the numbers or their source and had I not inserted myself into what was described as a well honed, efficient process the problem might have never been identified; and there a few more just like it.  My trust in metrics was permanently altered after that.

Metrics represents an excellent way for decision makers to quickly understand status and identify problems.  I’ve quoted here before about how someone I respect quite a bit was fond of asking her team “If you can’t measure it, how can you manage it” and she’s absolutely right.  Metrics is the ultimate management means of measuring key activities and issues within their world.  But how far do you go and how much effort do you expend pulling the related reports together?  And even if you’re able to automate the process and shorten the time necessary to generate the reports, how do you know that you’re either measuring the right things or that the underlying data is unaltered?  Ultimately I think that senior managers should be provided with something akin to a cost-benefit analysis for each metric they’re given.  Have them understand the degree of complexity and the amount of effort required to generate a number before deciding whether or not it’s worth it.  Perhaps I’m being naive but I’d like to think that most C-level executives would eliminate a significant amount of their reporting if they could see how much it was really costing them.

Here’s the part that should really concern you the most though: Metrics is a key component of Board reporting, they make all sorts of decisions based on what these reports tell them.  How can that be allowed unless the process used to generate them is locked down and audited?  Where are the regulators in all of this?

Things are looking up a bit because I have a new favorite regulatory agency to follow, the Consumer Financial Protection Bureau (CFPB).  And here’s why:  They focus on things that impact my day-to-day life (and yours as well).

I started tracking what the CFPB was doing about five months ago by accident.  Someone I know who used to be an examiner for the FRB switched over to the newer agency right at its infancy and I noticed this courtesy of a LinkedIn update.  Because I consider the Fed to be the Big Kahuna of the regulatory agencies I was surprised (you don’t leave the Yankees to sign with an expansion team unless you have to, or so I thought).  Compelled a bit by the update I started poking around the CFPB website.  For the first few months of this year it seemed to have potential but was little more than brochure-ware.  But last month that all changed.

The first CFPB update that caught my attention was labeled 12 CFR Part 1070 and it was all about the protection of consumer data, only with a slight twist.  Basically it was all about how any information they received as part of their field work would be protected exactly the same way that any third party vendor would be required to.  Despite their being a Federal agency they weren’t going to hide behind that as a means to simplify their lives.  They spearheaded an update to the underlying regulation that frames their charter so that consumers and their institutions can be assured that all PII and NPPI would be protected.  For me it was a rare win-win topic; protection of PII and NPPI combined with a reference to vendor management (these are a few of my favorite things).  And really for me it was that much more significant because I’ve known of a few situations where representatives of Federal and State regulatory agencies were responsible for the outright loss of confidential and/or restricted data.  Beyond a slap on the wrist there wasn’t much else done to the offending examiner or their agency.  And the affected institution couldn’t really complain too loudly because it’s always a bad idea to challenge your regulators, even when you’re in the right.  So I thought this was all at once a compelling and remarkably sensible update by a regulator, not something I’d expect to see.  That was the first points on the board for the CFPB.

The second set of points were scored almost on the same day.  I wanted to check one of the details related to the aforementioned update and noticed this one “Consumer Financial Protection Bureau report finds confusion in reverse mortgage market“.  Because I have a parent who is a senior citizen and who I think might one day soon be open to at least exploring a reverse mortgage I read with great interest.  The report was in plain English, was oriented in such a way that I could share it with my family and have them understand the issues and concerns detailed within and most importantly it made sense.  Reverse mortgages are growing in popularity and its main audience is the senior citizens segment of society.  Seniors tend to be  more easily misled, they’re under greater pressures to find new money sources (courtesy of our recession) at a time in their lives where going back to work is often not an option.  And because a parent would do almost anything rather than turn to their children for financial assistance they see a reverse mortgage as a way out of their predicament.  So for me having this content available was quite the relief.  I can caution and advise all day and night but the risks presented by a reverse mortgage are much more credible coming from an authorized source.  And so I celebrated July 4th this year by declaring the CFPB my new FDIC (the Sheila Bair inspired version, not the current blah one).

Here’s my really bizarro advice to any of you with even the slightest interest in regulatory oversight; if you haven’t already done so visit www.cfpb.gov and take a look around.  It’s oriented towards lay people, not just lawyers and regulators (and practitioners like me) and addresses topics and concerns that affect the majority of our population.  Basically it’s what I would expect from a regulator that still has that new agency smell but nothing like I’ve come to know from those that preceded it.  To those who have had a hand in defining its charter and organizing its content, great job!   Now repay my kind words by going out and getting me some juicy enforcement stories to write about.

Of course the truth is much more complicated.  I don’t just focus on computers, my scope expands to include anything that involves sensitive information.  While that always includes a variety of devices it also includes paper-based and people processes as well.  I frequently share stories about the enormous amount of printed content that’s to be found throughout an institutions physical locations.  I occasionally tell stories about how careless people can be when on the phone or in conversation and sharing all manner of sensitive information.  It’s never just about computers, it is however always about information and how it needs to be protected.

Truthfully though what I really do is search for controls that protect information, identify those that I find and try and measure their effectiveness and more importantly identify where controls are missing and work with my clients to remedy that.  At the heart of the regulatory requirements I focus on it’s all about the risk introduced by the presence of information, from personally identifiable (PII) to non-public personally identifiable (NPPI).  Risk: It’s what drives every single project I work on, it’s what drives every product and process I help develop.  And really, if you take the time to read through the literature, it’s what’s behind just about every piece of regulation known to the banking world.  Risk, risk, risk and risk.

One of the reasons I’ve enjoyed spending so much time working with the community banking and credit union sector over the past few years is that it’s a simple enough argument to make with fewer people to convince; everything you do to comply with the regulations should be risk-based.  It doesn’t really make a difference if it’s complicated to do or time consuming, you prioritize based on where they are found and make decisions accordingly.  But that gets much more difficult to do as the institutions grow in size and complexity.  Over the fifteen years I’ve been building and supporting compliance initiatives I’ve worked with Fortune 50′s, 100′s and 500′s and a whole lot of financial institutions that merely read Fortune magazine.  But while their overall size varies widely risk is still risk and that never changes.

I wish more practitioners embraced this simple concept.  While some do, many still don’t.  There’s often a rush to come up with a standard set of decision criteria to drive the work based on factors not necessarily aligned with risk factors.   Those who have worked with or for me will tell you that when presented with questions about which vendors or applications to assess or what to look for when conducting any type of assessment my first line of logic is to try and figure out where the greatest possible exposures to be found.   Assessing a low risk application yields little value  no matter how complete it may be.  And reviewing a vendor where the dollar spend is high but the risk factors are low does little to protect the institution.

Beware the practitioner who wields a hammer for they only know to look for nails.

Your regulator doesn’t want you to blindly implement compliance programs, they want you to identify and manage risks, real risks.  They want to be able to understand the logic and approach being used and find credible evidence that you’re focusing your efforts on the right things.   Go back and read through the library of FFIEC documentation and pay close attention to the hooks inserted throughout where they talk about conducting assessments and talk about using approaches which are appropriate for the size and complexity of your institution.  Then scan through your related program inventory and figure out if you’ve designed things accordingly.  Are they actually protecting your institution from credible threats and risks or are they just filling binders on your compliance officers shelves?

For me, professionally I’d prefer to always only do meaningful work and in the audit and assurance world meaningful is code for risk-based.

A week went by and I didn’t hear from the client about how the exam was progressing.  Another week went by and despite pinging the CIO a few times I still didn’t hear back from him.  Nearly two weeks after the exam should have concluded I finally received an email from the client and all he had to share was that the examiners hardly asked for any of the things that were missing and only dinged him on a few minor points.  OK, so it’s entirely possible that I overstated how bad things were there and the examiner simply didn’t share my opinion.  That is until I took a big mental step back and thought about it.  They didn’t have a vendor management program (or anything even close to one), they didn’t have a business continuity or disaster recovery plan, they hadn’t done a vulnerability assessment or pen test in more than two years and their firewall allowed me to establish a remote desktop connection on my guest machine while plugged into their network.  How is it possible that anyone with the slightest bit of audit/compliance experience did even the tiniest amount of real fieldwork during the exam?  Sadly it wasn’t an isolated situation.

I’m routinely amazed by how often I encounter financial institutions that have real and significant issues sitting right out in the open and somehow their examiners don’t notice.  And every time it happens I’m left wondering who examines the examiners?

One of the reasons our practice first committed to developing our vendor management software was because of how many of our existing clients were badly in need of a solution.  We almost always found that either they didn’t have something already in place or it was at best partially baked (a spreadsheet does not a vendor management program make).  We reasoned that if we could offer something that was user-friendly and focused on what the regulations required we’d have an anxious and ready market to sell to.  Fast forward three years and while we’ve had a healthy measure of success the number of institutions still needing help with vendor management remains shockingly high.  Why is that?  Because no one is going to spend money on a solution or commit resources to working on something their examiners never seem to care about.  And why is it that examiners don’t seem to care about it?  It’s either because they don’t look for it or they don’t know exactly what to look for.

So here’s my head scratching moment: How can anyone ever pass an IT exam without having a truly viable vendor management program?  How can someone pass an IT exam without a business continuity plan?  How can someone pass an IT exam without providing evidence that their network is secure?

Two years ago we anticipated a spike in services work when the Red Flags regulation from the FTC was due to go into effect – we’re still waiting.  Most of our clients have something in place to show when asked about Red Flags but when pressed to provide evidence of its effectiveness they have little to share.  This was not some obscure requirement that’s been around forever or is ancient or poorly designed or explained – this had awesome marketing material to accompany its launch so that everyone who had to comply clearly knew how to do so.  Everyone was talking about it in the months leading up to the effective date and everyone made sure they were working on some sort of program.  And still there’s little to show for their efforts.  Isn’t anyone paying attention to this fact?

What makes all of this extra frustrating is that there are safeguards in place where exams are audited.  But there are limitations to how much can be covered and if what I suspect is true, they’re not so much focusing on artifacts that are missing but rather on making sure that conclusions formed based on available evidence are solid.  So if the examiner doesn’t collect a current BCP and doesn’t write it up anywhere that it was missing or inadequate no amount of double or triple checking will identify a gap.   And to compound my frustration the blind spots are generally regional in nature.  Some of our clients get hammered on everything and others are barely pressed to provide evidence.  When we take a step back to see if a pattern emerges it does and it’s almost always defined by geography.  How does any of this make sense if all of the examiners are trained using the same methodology?

I don’t think I’m expecting too much from the process.  I’d like to know that if my banks main data center is hit by a meteor they have a plan in place to ensure that I can still access my money and pay my bills.  I’d like to know that my social security number is not being shared with a vendor who subcontracts out their work to a rogue group comprised of known felons.  I’d like to know that the tellers in my local branch aren’t able to cut and paste my account information from their teller software into a Yahoo email on their workstation and send it to an accomplice.  Or in other words, I’d like to know that my bank is compliant with GLBA.  Is that too much to ask?  I don’t think so, I really don’t.