IT

Should bank examiners rely on audit and assessment reports?

A favorite cliché of mine is “if it wasn’t for the last minute nothing would ever get done.” Personally it’s sort of the way I’m wired and in my industry it’s an unwritten rule when it comes to many annual activities. There’s an appreciable uptick in services work each year beginning in early fourth quarter as banks and credit unions wake up to the realization that the audits and assessments they are committed to conduct have yet to be done. And examiners typically don’t pay much attention to the timing of the work; they only care that it’s done during the expected time frame, so oddly enough this approach works.

But this leads to another interesting quirk about how the examiners often operate. Generally speaking, if the reports are available, they don’t dig much (if at all) beyond the reports contents. And so the information security and IT components of many exams become more about inventorying recent reports and not much else. We see evidence of this all of the time when we conduct a first-year audit or assessment and discover gaps or issues that have been in place for years and which the exams never picked up on.

I’ve written in the past about how surprisingly few institutions maintain a current business continuity plan and even fewer properly test that plan. But what surprises me more is that these conditions have existed for years spanning many exam cycles. How is that even possible?

I’ll tell you how: There’s a documented plan that is provided upon request and by and large the examiner conducting the fieldwork checks off that they received it and voila, you have a non-issue. And because the people in the field are typically given too few hours to cover too much landscape, they don’t have enough time to dig in deeper. Sometimes it happens where an examiner happens to actually open the document and vet it for key details - every now and again we come across a DoR or an MoU where the absence of a recent business impact analysis was tagged - but that happens almost never.

I’m fond of advising clients that you conduct much of the required compliance work for one of two reasons: You do it because it’s the right way to manage your institution and reduce your risk or because you have to. Because of the approach taken by examiners, way too many institutions lean towards the latter and simply want to have a report available to hand over when asked. But is this really the right way to run a financial institution?

And when you consider that the value of the report is largely defined by its contents and the competency of the practitioners conducting the fieldwork behind it, isn’t there an increased likelihood that there are important issues that go undetected? If all you do is pay for the report (often issued by the firm submitting the lowest bid) and all the examiners do is check off that the report was available and issued during the appropriate time frame, is there any real value in even bothering with this process?

I’m a bit biased regarding the value of reports. My firm is on a constant hunt for real risk and not just simply working our way down a checklist to kick out a document and collect our money. We tend to examine our clients infrastructure as if though we have our own money deposited with them and tie what we see straight back to GLBA and NCUA requirements. The value in this approach is that we produce a report that the board of directors can relate to, not just the IT folks.

But again, if no one really even cares about the content of the report and only that it exists, why bother doing a good job?

Maybe our industry needs to adopt an approach similar to the PCI folks. Maybe the FDIC and NCUA should issue certifications to practitioners validating them as properly trained and educated experts with regards to GLBA. There would still be a variance from firm to firm to a certain degree, but at least there would be a recognized standard and an increased likelihood that if an examiner is going to rely on the competency and completeness of a report there’s some justification behind that decision.

Something’s going to have to change though and hopefully sometime soon. Because using the “last minute” logic is flawed and only serves to reinforce my own bad habits.

IT audits versus reviews

I had mentioned in my last post a recent conversation with my partner regarding a proposed IT general controls (ITGC) audit. My primary role in our practice is to head up regulatory compliance services which includes audits, assessments and program development; my partner’s primary role is head of sales and business strategy. However, there’s a significant amount of overlap between our two sides and I sometimes forget that I’m the compliance expert when we’re discussing the industry. His knowledge of the myriad regulations is impressive and there are times where I’ll vet ideas through him to validate my own thinking.

Anyway, he’d mentioned a conversation with the client around the proposed ITGC audit in which the project sponsor asked what the difference is between an audit and a review. I knew right away where the question stemmed from because of my experience in the industry. Many firms we compete with (and some I’ve even worked for in another time and place) don’t conduct audits, they conduct reviews. Sometimes they don’t even conduct a review or an audit but rather an assessment. I’ve struggled with this blurred use of terms because in my mind there are very clear delineations. The lifecycle of the governance, risk and compliance (GRC) domain is as such: identify and assess risk, design controls to mitigate those risks and test to validate that those controls are functioning as expected. And so, a risk assessment is conducted, governance elements are introduced in the form of policies and procedures, and regularly scheduled tests occur to make sure that the whole enchilada is actually getting the job done. And so when practitioners offer services that aren’t specific to one of the key areas of the GRC spectrum, it bothers me.

See, an audit is an audit is an audit; you determine the control objectives that are supposed to be supported by the entity that are in scope for the type of audit, identify the related control activities that either are or should be in place and then design a series of test steps to determine if those activities are occurring and tie back to the overall objective. The auditor has some leeway when it comes to offering an opinion as to what the results actually mean (one auditors pass is oft times another auditors finding) but fundamentally the audit results tend to be binary, you either pass or fail. It’s all fairly straight forward.

Now a risk assessment is not an audit; it’s a bit more arbitrary. Management generally is polled to determine what areas of their infrastructure (including finance, operations and technology) they are most concerned with, factor in regulatory and industry requirements and then come up with a plan for conducting the risk assessments. As these assessments occur, what they reveal would be factored into the overall audit plan to make certain that the areas presenting the greatest risk to the business are being examined closely and in a timely fashion.

So here’s my question: what exactly is a review? If you’re conducting tests and examining evidence you’re conducting an audit. If you’re interviewing stakeholders and determining what controls are in place but not testing their effectiveness then you’re conducting a risk assessment (assuming you’re asking the right sort of questions – a post for another day). Is there some odd dimension between the risk assessment and the audit I’m unaware of where this review occurs? And what exactly are you expecting from the results of this review? Because I’ll tell you this much, examiners only recognize risk assessments and audits. You can present them with a report that’s called a review in the title and tell them it’s actually an audit but you’d better be prepared to produce work papers because they’re going to ask you for them, trust me on this point. But my experience is that reviews don’t produce work papers because evidence is generally not formally collected in support of the report and requires significantly less effort to conduct. Which is why when we discussed the semantics of our industry, my partner is fond of saying that the difference between an audit and a review is about 50% of the proposed amount.

And as long as I’m ranting on the topic, how many of you out there include work papers as a deliverable when you contract with external entities to conduct audits? I’ve long been amazed at how many audit projects I’ve managed where work papers weren’t required or provided because it wasn’t included in the statement of work. I’m of the opinion that a summary audit report by itself is useless if you don’t have the supporting documentation because it’s the only true way to confirm that the testing occurred and support the findings. And of course it’s important to loop back to my earlier comment: Examiners will absolutely ask you for the work papers. I routinely receive phone calls from clients I’ve worked with at my previous firms asking if I still had access to their work papers because their examiner is asking for them. It’s awful for me because I have to tell them that while the evidence is still available (I keep everything for seven years for fieldwork I’ve conducted) there are no formal work papers to provide. Thus the reason that when we started our firm and developed the methodologies, I made certain that part of the scoping process included asking the client if work papers were to be included in the deliverables. It adds time to the project and so there’s a cost implication, but it’s the right way to run an audit and so really a need-to-have. You can take a short cut where applicable and only document findings but even so, how can you prove that a successful test was actually successful? For those practitioners looking for shortcuts it provides the wrong incentive.

If you want/need an audit, schedule an audit. If you want/need a risk assessment, schedule a risk assessment. If you’re considering a review or a generic assessment reconsider; decide which of the two proper categories your work fits into and than schedule accordingly. This isn’t rocket science; the FDIC and NCUA have been quite clear as to what you’re required to do so keep it simple. And if the resources you’re using aren’t speaking in the common audit and compliance vernacular force their hand and make them do so. Audit or risk assess, pick one.