risk

Should bank examiners rely on audit and assessment reports?

A favorite cliché of mine is “if it wasn’t for the last minute nothing would ever get done.” Personally it’s sort of the way I’m wired and in my industry it’s an unwritten rule when it comes to many annual activities. There’s an appreciable uptick in services work each year beginning in early fourth quarter as banks and credit unions wake up to the realization that the audits and assessments they are committed to conduct have yet to be done. And examiners typically don’t pay much attention to the timing of the work; they only care that it’s done during the expected time frame, so oddly enough this approach works.

But this leads to another interesting quirk about how the examiners often operate. Generally speaking, if the reports are available, they don’t dig much (if at all) beyond the reports contents. And so the information security and IT components of many exams become more about inventorying recent reports and not much else. We see evidence of this all of the time when we conduct a first-year audit or assessment and discover gaps or issues that have been in place for years and which the exams never picked up on.

I’ve written in the past about how surprisingly few institutions maintain a current business continuity plan and even fewer properly test that plan. But what surprises me more is that these conditions have existed for years spanning many exam cycles. How is that even possible?

I’ll tell you how: There’s a documented plan that is provided upon request and by and large the examiner conducting the fieldwork checks off that they received it and voila, you have a non-issue. And because the people in the field are typically given too few hours to cover too much landscape, they don’t have enough time to dig in deeper. Sometimes it happens where an examiner happens to actually open the document and vet it for key details - every now and again we come across a DoR or an MoU where the absence of a recent business impact analysis was tagged - but that happens almost never.

I’m fond of advising clients that you conduct much of the required compliance work for one of two reasons: You do it because it’s the right way to manage your institution and reduce your risk or because you have to. Because of the approach taken by examiners, way too many institutions lean towards the latter and simply want to have a report available to hand over when asked. But is this really the right way to run a financial institution?

And when you consider that the value of the report is largely defined by its contents and the competency of the practitioners conducting the fieldwork behind it, isn’t there an increased likelihood that there are important issues that go undetected? If all you do is pay for the report (often issued by the firm submitting the lowest bid) and all the examiners do is check off that the report was available and issued during the appropriate time frame, is there any real value in even bothering with this process?

I’m a bit biased regarding the value of reports. My firm is on a constant hunt for real risk and not just simply working our way down a checklist to kick out a document and collect our money. We tend to examine our clients infrastructure as if though we have our own money deposited with them and tie what we see straight back to GLBA and NCUA requirements. The value in this approach is that we produce a report that the board of directors can relate to, not just the IT folks.

But again, if no one really even cares about the content of the report and only that it exists, why bother doing a good job?

Maybe our industry needs to adopt an approach similar to the PCI folks. Maybe the FDIC and NCUA should issue certifications to practitioners validating them as properly trained and educated experts with regards to GLBA. There would still be a variance from firm to firm to a certain degree, but at least there would be a recognized standard and an increased likelihood that if an examiner is going to rely on the competency and completeness of a report there’s some justification behind that decision.

Something’s going to have to change though and hopefully sometime soon. Because using the “last minute” logic is flawed and only serves to reinforce my own bad habits.

The COBIT framework isn’t an audit solution

I have an associate who has an addiction to certifications. He’s one of those “too smart for his own good” geniuses who often decides to change his career course and starts by obtaining whatever accreditation or cert is needed to do so. When he lists all of these accreditations and certs after his name it looks as if though someone tossed their alphabet soup lunch. But his logic is that having the appropriate governing body’s seal of approval is akin to knowing the secret password needed to gain access to the right job.

Sometimes I think COBIT is used much the same way.

For those of you who aren’t familiar with COBIT, it’s a framework that has revolutionized the world of governance and compliance for the better. It was the only beacon in the vast, dark ocean of SOX insanity a few years back, providing much needed guidance for corporate America to follow and continues to serve as the best source when designing controls within the infrastructure. It’s comprehensive, well organized and when understood and applied properly, it can be very effective.

But it’s not akin to the Bible and it’s definitely not an IT audit framework or program.

And yet I often hear fellow practitioners dropping COBIT references like it somehow validates them as legitimate members of the IT audit club (which by the way is called ISACA and only requires an annual membership fee).

Just this week, I heard that someone discussed conducting a COBIT-based audit when asked about their approach to conducting an IT general controls (ITGC) audit. Two weeks ago, my partner asked me about an RFP we received in which the institution wanted to know if we based our ITGC audit on COBIT or any other recognized framework. It’s gotten to the point where the term “COBIT-based” has become ubiquitous within the IT audit domain. Years ago during the aforementioned SOX insanity, there was a running joke with a client in which every sentence was laced with a SOX reference (e.g. Good SOX morning, Happy SOX New Year, etc.). Now it seems as if though COBIT has replaced SOX in that regard.

Um, has anyone actually read the framework? I mean actually sitting down and reading it from executive summary through to ME4 (the last of the control objective areas in the PDF). And how many people have actually tried to implement COBIT as it’s intended to be used? It’s a mountain of information that requires a ton of analysis and customization prior to being implemented. And it’s not intended for organizations both big and small. For many of the community banks and similarly sized credit unions that I commonly work with, it’s simply overkill.

But again, it’s not an audit framework and it’s not an audit program. And it’s entirely possible to build out an IT controls framework and never once rely upon COBIT to do so.

By the way, for those of you who aren’t familiar with the IT Governance Institute (ITGI), it’s a research think tank that exists to be the leading reference on IT governance for the global business community. In the time since COBIT made its inroads into corporate America and the audit vernacular, ITGI has amped it up a notch. Now they also publish Val IT and more recently Risk IT.

So now I’m bracing for the onslaught of risk assessments that are “Risk IT” based. But I never had a problem conducting a risk assessment before this standard existed and I doubt I’ll crack it open when conducting one in the near future. Did we really need this? And how will this drive the audit and compliance industry?

Frameworks have a place in this world, don’t get me wrong. But it’s like when I bought my Roto Zip hand saw a few years back; I walked around my house looking for things I could use it for rather than simply using it when it made sense. COBIT is awesome and it’s helped provide clarity in many, many ways. But it isn’t the official book of record for audit and compliance within IT; it’s just another tool in the toolbox. I realize that on the planet of ISACA that’s akin to blasphemy, but I offer no apologies. I refuse to build an audit program for a community bank that’s supported by two IT resources based on the 200 plus control objectives in COBIT.

And on that note I bid you a good COBIT day.

IT audits versus reviews

I had mentioned in my last post a recent conversation with my partner regarding a proposed IT general controls (ITGC) audit. My primary role in our practice is to head up regulatory compliance services which includes audits, assessments and program development; my partner’s primary role is head of sales and business strategy. However, there’s a significant amount of overlap between our two sides and I sometimes forget that I’m the compliance expert when we’re discussing the industry. His knowledge of the myriad regulations is impressive and there are times where I’ll vet ideas through him to validate my own thinking.

Anyway, he’d mentioned a conversation with the client around the proposed ITGC audit in which the project sponsor asked what the difference is between an audit and a review. I knew right away where the question stemmed from because of my experience in the industry. Many firms we compete with (and some I’ve even worked for in another time and place) don’t conduct audits, they conduct reviews. Sometimes they don’t even conduct a review or an audit but rather an assessment. I’ve struggled with this blurred use of terms because in my mind there are very clear delineations. The lifecycle of the governance, risk and compliance (GRC) domain is as such: identify and assess risk, design controls to mitigate those risks and test to validate that those controls are functioning as expected. And so, a risk assessment is conducted, governance elements are introduced in the form of policies and procedures, and regularly scheduled tests occur to make sure that the whole enchilada is actually getting the job done. And so when practitioners offer services that aren’t specific to one of the key areas of the GRC spectrum, it bothers me.

See, an audit is an audit is an audit; you determine the control objectives that are supposed to be supported by the entity that are in scope for the type of audit, identify the related control activities that either are or should be in place and then design a series of test steps to determine if those activities are occurring and tie back to the overall objective. The auditor has some leeway when it comes to offering an opinion as to what the results actually mean (one auditors pass is oft times another auditors finding) but fundamentally the audit results tend to be binary, you either pass or fail. It’s all fairly straight forward.

Now a risk assessment is not an audit; it’s a bit more arbitrary. Management generally is polled to determine what areas of their infrastructure (including finance, operations and technology) they are most concerned with, factor in regulatory and industry requirements and then come up with a plan for conducting the risk assessments. As these assessments occur, what they reveal would be factored into the overall audit plan to make certain that the areas presenting the greatest risk to the business are being examined closely and in a timely fashion.

So here’s my question: what exactly is a review? If you’re conducting tests and examining evidence you’re conducting an audit. If you’re interviewing stakeholders and determining what controls are in place but not testing their effectiveness then you’re conducting a risk assessment (assuming you’re asking the right sort of questions – a post for another day). Is there some odd dimension between the risk assessment and the audit I’m unaware of where this review occurs? And what exactly are you expecting from the results of this review? Because I’ll tell you this much, examiners only recognize risk assessments and audits. You can present them with a report that’s called a review in the title and tell them it’s actually an audit but you’d better be prepared to produce work papers because they’re going to ask you for them, trust me on this point. But my experience is that reviews don’t produce work papers because evidence is generally not formally collected in support of the report and requires significantly less effort to conduct. Which is why when we discussed the semantics of our industry, my partner is fond of saying that the difference between an audit and a review is about 50% of the proposed amount.

And as long as I’m ranting on the topic, how many of you out there include work papers as a deliverable when you contract with external entities to conduct audits? I’ve long been amazed at how many audit projects I’ve managed where work papers weren’t required or provided because it wasn’t included in the statement of work. I’m of the opinion that a summary audit report by itself is useless if you don’t have the supporting documentation because it’s the only true way to confirm that the testing occurred and support the findings. And of course it’s important to loop back to my earlier comment: Examiners will absolutely ask you for the work papers. I routinely receive phone calls from clients I’ve worked with at my previous firms asking if I still had access to their work papers because their examiner is asking for them. It’s awful for me because I have to tell them that while the evidence is still available (I keep everything for seven years for fieldwork I’ve conducted) there are no formal work papers to provide. Thus the reason that when we started our firm and developed the methodologies, I made certain that part of the scoping process included asking the client if work papers were to be included in the deliverables. It adds time to the project and so there’s a cost implication, but it’s the right way to run an audit and so really a need-to-have. You can take a short cut where applicable and only document findings but even so, how can you prove that a successful test was actually successful? For those practitioners looking for shortcuts it provides the wrong incentive.

If you want/need an audit, schedule an audit. If you want/need a risk assessment, schedule a risk assessment. If you’re considering a review or a generic assessment reconsider; decide which of the two proper categories your work fits into and than schedule accordingly. This isn’t rocket science; the FDIC and NCUA have been quite clear as to what you’re required to do so keep it simple. And if the resources you’re using aren’t speaking in the common audit and compliance vernacular force their hand and make them do so. Audit or risk assess, pick one.

Risk is at the heart of what matters most.

I had two great conversations this week regarding risk assessments (jeez, does that ever sound geeky).

The first conversation centered on what an associate was expecting  to accomplish via the risk assessment process and the second one was a general conversation about the proper approach to conducting one in support of PCI.  Fundamentally, at some higher level a risk assessment looks like a risk assessment regardless of its intended purpose.  But at the ground level, there are huge swings and variances between different types of assessments and how they’re conducted.

The first go-round centered on GLBA and was with a fellow practitioner I’ve worked with before on audits.  The conversation started because my associate was preparing to conduct an information security risk assessment later this month and was talking about all of the planning effort required before, during and after the fieldwork.  I commented that it sounded more like he was conducting an audit rather than an assessment; his reply “same difference.”  And I thought to myself “funny, it sounded like he just said that an audit and an assessment were the same thing.”  So I asked him to clarify and he did by stating that the only difference between the two was whether or not you needed to create work papers.  I was stunned by this shocking admission and obvious lack of understanding of our trade.  An audit tests in-place controls to determine if they’re working as designed and are sufficient to address the related risks those controls are intended to manage or mitigate.  An assessment examines objectives or assets, identifies risks to either and then looks to see if there are controls in-place to manage those risks.  I can get more granular than that but at its most basic level, that’s how it’s supposed to work.  Risk assessments identify risks and related control activities and audits test to see if the necessary controls are present and that they’re working properly.  He also shared with me that he uses the same general outline of questions and topics for both, that they’re both based on FFIEC guidance and that it’s exactly what the examiners are expecting.

OK, first of all, the examiners are not expecting that the same work is being done for both an audit and an assessment.  I’ve spoken with enough of them to know that plus I’ve heard several of them stand before large audiences and explain the nuances of both and so I know they get the differences.  Second of all, the entire purpose of a GLBA-based assessment is to ensure that at some acceptable frequency (which secretly means annually) each financial institution should conduct a thorough assessment of their infrastructure (not just IT, by the way) and identify and measure risks and threats to the customer data they manage and store.  If you move right past the assessment and conduct an audit you’ll never truly know if you’re missing something because you’re only testing what you can see.  I’ve participated in and conducted enough of these to know this to be true.  And if you conduct an assessment like it’s an audit, you have the same basic issue.  Thus the reason why you typically schedule an assessment first and an audit second.  I was about to explain this to my associate but decided against it; I somehow thought it would’ve fallen on deaf ears.

In the second conversation I participated in around risk assessments it was all about PCI.  This one was much easier because the team running the assessment had built a high-level approach based on guidance from the PCI DSS Council itself and displayed a clear understanding of what they were to do.  While they hadn’t developed any of the templates to be used (it’s not due to be conducted until next month) they were well into the planning stages.  My role was to simply listen and provide any meaningful feedback.   The one potential gap in their intended approach is that they were taking an asset-based approach to the assessment.  What that effectively means is that they only look for and attempt to measure risks around what they can see be it a piece of equipment or a documented process.  But if there’s something that doesn’t appear on an inventory, a network diagram or exist in a binder somewhere, they’ll potentially miss it.  A great PCI for-instance is what happens when a customer service rep writes down sensitive information on a pad because their system is hanging and they’re trying to keep the call within acceptable time frames?  It’s not supposed to happen but I’ve been in a dozen call centers over the past few years and have personally witnessed it happening in almost all of them.  If you were to rely exclusively on what’s documented, you would note that typically erasable whiteboards are to be used for such situations and so you would consider that to be low risk.  But the risk increases significantly if you move past the “thing” you’re looking at and poke around a bit.  Assuming that they do use a scratch pad on occasion, what happens to that piece of paper after the transaction is completed?   Is it thrown out and if so, is it placed in a secured bin or in the regular trash?  And so I advised the client to approach the assessment like it’s an Easter egg hunt.  Deviations and violations are always swirling about (we’re human, we make mistakes) and as part of the assessment process you should be looking for where that might be happening.  But the best part of the Easter egg approach is that it gets everyone involved in the conversation thinking a bit outside the box and that’s where the really neat information is found.  And really, in the end, that’s what a well-run risk assessment should be all about: seeking out and measuring risk wherever it may be hiding.

When the PCI conversation concluded I was feeling a bit energized because in brainstorming with the client I had the ideas flowing freely.   And than I recalled the first conversation and I could almost hear the screeching tires and smell the burning rubber where my creativity came to a complete halt.  It’s amazing to me how the term “risk assessment” can mean completely different things to different people.  I was as impressed with my client asking for guidance and wanting to get it right as I was disappointed in my colleague for having it all wrong.  This wasn’t an issue of principle but rather results; how can you properly manage this thing called risk if you don’t even know how to begin looking for it?

Speaking of risk, check back next week when I jump back onto the GRC train of thought and bring you up to speed with something I’m working on.

Who put the G in GRC?

I’m something of an advocate for Governance, Risk and Compliance (GRC) and have been for several years.  I’ve been known to rant a bit how it’s not properly organized as an acronym because everyone who knows knows that risk comes first and so it should’ve been RGC.  But as a discipline and as an approach to designing and implementing controls I’m all for governance being used as the driver to assess, measure and manage risk.  And of course if you’re properly managing risk you’re also naturally falling into alignment with all things compliance.

For the most part whenever I see references to GRC in the marketplace it almost always is associated with a software product and not a discipline or a methodology.  And in those rare instances where it is in reference to something being practiced it’s often depicted as an advanced formulaic concept that requires a PHD to understand, let alone practice.  But I’m certain that’s going to change.  With all of the layers of regulatory requirements already placed upon Corporate America and with the very real threat of even more looming large on the horizon I know that eventually companies and institutions are going to be forced to abandon their all-too-common one-off, silo-centric approaches to compliance and commit to a single, well thought out governance program.  My best guess is that once the economy begins the slow, steady climb out of its current abyss we’ll start seeing signs of progress on the this front.

And so I’m always monitoring the GRC landscape looking for subtle shifts and changes that may indicate a new advance or important discovery.

Two weeks ago one of those subtle shifts landed tap-dead center on my GRC tracking radar only it wasn’t so subtle.

While working for a client who is suddenly confronted with the demands of a brand new set of regulations I committed to building out a cross-reference matrix by which they can identify commonalities between their different frameworks and look for economies of scale in the work required to comply.  But I’m sometimes lazy and decided that somebody somewhere must have already done something like this; I’m smart but I’m not often the first one to think of something.  And so a-Googlin’ I went.  Imagine my surprise when I not only found what I was looking for but also found that there was a company that created a product that incorporates pretty much every regulation currently known to civilized man and developed a master cross-reference to illustrate all of their interdependencies.

The product is called the “Unified Compliance Framework” and for those people who understand governance and are committed to advancing it from theory to practice this is something akin to the Holy Grail.  Simply put UCF monitors the regulatory and industry landscape, identifies emerging requirements/frameworks as well as modifications to those that already exist and conducts an analysis to identify how it relates to other frameworks.  This allows any organization to take their existing control framework and use UCF to map those controls across the entire compliance spectrum identifying where one control satisfies multiple frameworks.

Think about that for just a minute.  If for example you’ve designed a control for password rules as part of your SOX framework you can use UCF to quickly identify which of the other frameworks that control addresses (several, by the way).  If your company conducts business in states that have or are about to have their own data privacy laws with which you have to comply (Massachusetts is the most recent) it’s very likely that not only don’t you have to re-invent the wheel but already have one to use.  UCF makes it easy to identify points of intersection thus making the impossible possible.  Or rather, it allows you to kill two (or more) birds with one stone (so-to-speak).

I’ve been railing for years against the common approach most companies use in which they design one-off solutions to align with the myriad frameworks they operate under.  But it’s been a difficult argument to establish and until finding UCF I’ve had to struggle to make my case.  But not any longer.

To validate my take on UCF I showed it to a colleague who is in senior management at a Fortune 500 company and who is himself responsible for IT Governance.  He immediately saw its potential and wanted to know who else was using it and how so.  I fear I’ve opened up a can of worms though because when I mentioned that I was researching early adapters of UCF he asked if he could join in on the interviews so that he can pick their brains and leverage off of their success.  I was looking for validation and instead inherited a partner.  But I feel as if though I’m helping create a mini-wave of excietment  in the governance space and I’m OK with that.

I’ll have more to share with you over the next few months as I continue to dig into how UCF is being used in support of GRC initiatives.  But in the meantime I encourage you to
check them out for yourself.  If you’re someone who has a governance role, hopes to have a governance role or simply wants a glimpse into the future of GRC it’s well worth your time.