assessment

Should bank examiners rely on audit and assessment reports?

A favorite cliché of mine is “if it wasn’t for the last minute nothing would ever get done.” Personally it’s sort of the way I’m wired and in my industry it’s an unwritten rule when it comes to many annual activities. There’s an appreciable uptick in services work each year beginning in early fourth quarter as banks and credit unions wake up to the realization that the audits and assessments they are committed to conduct have yet to be done. And examiners typically don’t pay much attention to the timing of the work; they only care that it’s done during the expected time frame, so oddly enough this approach works.

But this leads to another interesting quirk about how the examiners often operate. Generally speaking, if the reports are available, they don’t dig much (if at all) beyond the reports contents. And so the information security and IT components of many exams become more about inventorying recent reports and not much else. We see evidence of this all of the time when we conduct a first-year audit or assessment and discover gaps or issues that have been in place for years and which the exams never picked up on.

I’ve written in the past about how surprisingly few institutions maintain a current business continuity plan and even fewer properly test that plan. But what surprises me more is that these conditions have existed for years spanning many exam cycles. How is that even possible?

I’ll tell you how: There’s a documented plan that is provided upon request and by and large the examiner conducting the fieldwork checks off that they received it and voila, you have a non-issue. And because the people in the field are typically given too few hours to cover too much landscape, they don’t have enough time to dig in deeper. Sometimes it happens where an examiner happens to actually open the document and vet it for key details - every now and again we come across a DoR or an MoU where the absence of a recent business impact analysis was tagged - but that happens almost never.

I’m fond of advising clients that you conduct much of the required compliance work for one of two reasons: You do it because it’s the right way to manage your institution and reduce your risk or because you have to. Because of the approach taken by examiners, way too many institutions lean towards the latter and simply want to have a report available to hand over when asked. But is this really the right way to run a financial institution?

And when you consider that the value of the report is largely defined by its contents and the competency of the practitioners conducting the fieldwork behind it, isn’t there an increased likelihood that there are important issues that go undetected? If all you do is pay for the report (often issued by the firm submitting the lowest bid) and all the examiners do is check off that the report was available and issued during the appropriate time frame, is there any real value in even bothering with this process?

I’m a bit biased regarding the value of reports. My firm is on a constant hunt for real risk and not just simply working our way down a checklist to kick out a document and collect our money. We tend to examine our clients infrastructure as if though we have our own money deposited with them and tie what we see straight back to GLBA and NCUA requirements. The value in this approach is that we produce a report that the board of directors can relate to, not just the IT folks.

But again, if no one really even cares about the content of the report and only that it exists, why bother doing a good job?

Maybe our industry needs to adopt an approach similar to the PCI folks. Maybe the FDIC and NCUA should issue certifications to practitioners validating them as properly trained and educated experts with regards to GLBA. There would still be a variance from firm to firm to a certain degree, but at least there would be a recognized standard and an increased likelihood that if an examiner is going to rely on the competency and completeness of a report there’s some justification behind that decision.

Something’s going to have to change though and hopefully sometime soon. Because using the “last minute” logic is flawed and only serves to reinforce my own bad habits.

Test what makes sense, not headlines

The recent news about a social engineering exercise gone awry serves as a lesson on how not to conduct these kinds of tests. An information security firm had sent a credit union NCUA-branded media to install in order to test if the employees would react appropriately and first attempt to validate that the request was legitimate. The problem was that no one notified the NCUA about their role in the exercise and so when they were contacted by the institution, they assumed this was a legitimate threat and issued a warning to all of their credit union members.

At first blush, this story might appear amusing. Beyond some embarrassed people at the security firm conducting the social engineering exercise and some likely annoyed folks at the NCUA (a federal security alert is not a common or simple occurrence), it would appear you could chalk this up to “no harm, no foul.” From certain vantage points you might even consider this a wildly successful test. After all, the client reacted appropriately by contacting the NCUA and the NCUA reacted appropriately by identifying the actions as unsanctioned and potentially harmful and alerting their member institutions. But for those of us who practice in the industry, this is far from amusing and actually somewhat disturbing.

When one of our member practitioners or firms does something that brings negative attention to the industry or conducts themselves in a way the results in a black eye, it’s always extended to a certain to degree to all of us. At some point in the process, I would have thought that someone working for the offending firm would have reviewed a draft of the plan and flagged the part about including an uninvolved third party as unnecessary or inappropriate (and quite possibly illegal). And besides, grand and elaborate schemes aren’t really necessary. Most breaches that occur aren’t of the James Bond variety, so subtle tactics work best.

I’ve managed and conducted social engineering tests many times in the past and so I speak from experience. On one project, I had a renegade auditor who wanted to test data center physical security by trying to either force or talk his way into the facility. He was told in no uncertain terms that doing so was not authorized or acceptable and that if he did it and was arrested (a very likely scenario), we would not bail him out and he would be fired immediately. It was just a bad idea and not even remotely necessary to test the related controls. And then I was reminded of a story in which a well-known national security firm had their practitioners dress up in firefighting gear and arrive at a bank branch claiming there was a possible fire/smoke condition to see if they would be allowed access to private/protected areas of the bank. Of course they were granted the access and as a result were written up in an industry magazine and considered to be innovative and imaginative.

Social engineering is intended to examine how the human element reacts to a variety of scenarios designed to gain access to sensitive information or secured areas. There are many, many simpler and less obvious techniques available to poke and prod and test the effectiveness of related controls. So why was this test even necessary?

The short answer is that it wasn’t. It was a bad idea in design and execution.

At a basic level, I don’t understand is how this test was even conducted. A common element when executing any form of security work is to inform the key stakeholders of the plan so that things like this don’t happen. When the appropriate party was notified about the suspicious material received from the NCUA they should have known what to do (beyond escalating to the NCUA). We inform the primary security contact of our activities so that they know not to escalate outside of their own institution. We provide specific start and end times, all key details, and status updates along the way.

The wrong messages are sent as a result of these wayward tests. I’m thinking that credit unions will now require all sorts of crazy validation before trusting anything from the NCUA. I’m also concerned that for the bank involved in the firemen scenario, they may not properly evacuate the facility in the event of a real fire because they’ll wait for confirmation that it isn’t another test. Is that really the desired outcome of these exercises? Last year, I managed an exercised involving a phone based phishing test. Two days after we concluded the fieldwork, I received a message from our client sponsor asking if we were still executing the test. Turns out they were the target of a legitimate phishing attempt and because our activities had raised awareness, the situation was escalated appropriately. Doesn’t that make a bit more sense?

How to combat the insider threat

I was reading an article last week about how there’s been a recent increase in the number of reported security breaches caused by internal resources.  The insider threat is not a new one as corporate espionage is as old as civilization but it certainly is getting more press lately as patterns are shifting and criminal activity is adapting with the times.

But what was really eye opening for me was the conversation I had later that day while onsite at a client.  I was sharing some of the details of the story with an associate and the person sitting across the aisle was one of those freaky smart network people: The sort who speaks IP as well as English, who can read a network topology as if though it’s the morning paper and who are often consumed with getting every component plugged into his (HIS) network configured and secured perfectly much like Claude Monet wanted to get the tone and dimensions of every flower just right on the canvas.  When he heard what we were talking about, he perked up and joined in on the conversation.

I was sharing in my amazement of how technology has made it so much easier to collect sensitive information in unobtrusive ways.  One of my favorites was the availability of an audio recording device embedded within a network cable.  It has a transmitter built into it that broadcasts up to 160 feet away so someone can record boardroom conversations with relative ease.  And of course it not only looks like a network cable, it is a network cable and functions normally so there’d be no reason to be suspicious.  I’ve long advocated for outside-the-box thinking regarding how foreign devices can (and do) circumvent the very best security controls and here was a great example as to why.  So the network guy chimed in that there are so many ways to gain access to sensitive information without detection.  He described how there are many vulnerabilities that exist that none of the frameworks or regulations come close to addressing (I’m not offering examples; no need to give anyone any ideas).  But what struck me was how impassioned and concerned he was in discussing this subject.  And it occurred to me that he couldn’t just turn this off; that this was something he had on his mind somewhere close to all of the time.  And so I asked him if he ever shared this during any of the various risk assessment activities conducted during the year and he couldn’t recall being included or asked.

How do you support activities focused on protecting your infrastructure without including your experts in the dialogue?  How do you know where the risks really are if you’re not asking the people who are charged with the responsibility of mitigating them?

I’m a bit biased based on my own personal experience.  I came into the audit and compliance domain in the mid-90s in a somewhat less than flattering way.  I had been an application project manager with a well-earned reputation of breaking just about every change management control in place.  When I had something that needed to get moved into production, I lacked the patience to work through the required processes and figured out how to get around almost all of them.  The architect of many of those controls was himself an auditor (and still one of the very best I’ve worked with) and he and I forged a healthy respect for one another along the way.  When Y2K rose to prominence and the client was concerned about maintaining their remediated production environment, I was recruited by the same person to help guard the gate, sort of like hiring the bank robber to protect the bank.  But it turned out to be a great idea as I sniffed out a number of undocumented backdoors and workarounds that were being used and ultimately led to my then new and now current career path.  And it also proved to be an important lesson that’s served me well all these years: If you want to find out what you need to be worried about, ask the people who have the necessary skills to make you worry.

If you want to identify as many vulnerabilities and potential exposures to your infrastructure from the insider threat, start by engaging the people who can speak IP like a second language, who can read a network topology like it’s a treasure map and who know they can easily download an entire database without detection or insert devices in the data center that capture non-tokenized, unencrypted data streams as they travel through the pipelines.  I’ve always found that for the most part these people are eager to share what they know because it’s their best chance to affect any sort of change.  While conducting an IT general controls audit recently I had someone coax me into asking a question about the physical designs of their datacenter because there was a structural issue that deeply concerned him and he was desperate to see it show up on my report.

Times are tough, people are desperate and there’s no telling what some people are capable of or willing to do in order to survive.  Make sure you’re doing what you can to protect yourself and your customer’s sensitive information and make sure you’re including the right people in the conversation.  The best defense against the insider threat may very well be the people sitting alongside them.

Risk is at the heart of what matters most.

I had two great conversations this week regarding risk assessments (jeez, does that ever sound geeky).

The first conversation centered on what an associate was expecting  to accomplish via the risk assessment process and the second one was a general conversation about the proper approach to conducting one in support of PCI.  Fundamentally, at some higher level a risk assessment looks like a risk assessment regardless of its intended purpose.  But at the ground level, there are huge swings and variances between different types of assessments and how they’re conducted.

The first go-round centered on GLBA and was with a fellow practitioner I’ve worked with before on audits.  The conversation started because my associate was preparing to conduct an information security risk assessment later this month and was talking about all of the planning effort required before, during and after the fieldwork.  I commented that it sounded more like he was conducting an audit rather than an assessment; his reply “same difference.”  And I thought to myself “funny, it sounded like he just said that an audit and an assessment were the same thing.”  So I asked him to clarify and he did by stating that the only difference between the two was whether or not you needed to create work papers.  I was stunned by this shocking admission and obvious lack of understanding of our trade.  An audit tests in-place controls to determine if they’re working as designed and are sufficient to address the related risks those controls are intended to manage or mitigate.  An assessment examines objectives or assets, identifies risks to either and then looks to see if there are controls in-place to manage those risks.  I can get more granular than that but at its most basic level, that’s how it’s supposed to work.  Risk assessments identify risks and related control activities and audits test to see if the necessary controls are present and that they’re working properly.  He also shared with me that he uses the same general outline of questions and topics for both, that they’re both based on FFIEC guidance and that it’s exactly what the examiners are expecting.

OK, first of all, the examiners are not expecting that the same work is being done for both an audit and an assessment.  I’ve spoken with enough of them to know that plus I’ve heard several of them stand before large audiences and explain the nuances of both and so I know they get the differences.  Second of all, the entire purpose of a GLBA-based assessment is to ensure that at some acceptable frequency (which secretly means annually) each financial institution should conduct a thorough assessment of their infrastructure (not just IT, by the way) and identify and measure risks and threats to the customer data they manage and store.  If you move right past the assessment and conduct an audit you’ll never truly know if you’re missing something because you’re only testing what you can see.  I’ve participated in and conducted enough of these to know this to be true.  And if you conduct an assessment like it’s an audit, you have the same basic issue.  Thus the reason why you typically schedule an assessment first and an audit second.  I was about to explain this to my associate but decided against it; I somehow thought it would’ve fallen on deaf ears.

In the second conversation I participated in around risk assessments it was all about PCI.  This one was much easier because the team running the assessment had built a high-level approach based on guidance from the PCI DSS Council itself and displayed a clear understanding of what they were to do.  While they hadn’t developed any of the templates to be used (it’s not due to be conducted until next month) they were well into the planning stages.  My role was to simply listen and provide any meaningful feedback.   The one potential gap in their intended approach is that they were taking an asset-based approach to the assessment.  What that effectively means is that they only look for and attempt to measure risks around what they can see be it a piece of equipment or a documented process.  But if there’s something that doesn’t appear on an inventory, a network diagram or exist in a binder somewhere, they’ll potentially miss it.  A great PCI for-instance is what happens when a customer service rep writes down sensitive information on a pad because their system is hanging and they’re trying to keep the call within acceptable time frames?  It’s not supposed to happen but I’ve been in a dozen call centers over the past few years and have personally witnessed it happening in almost all of them.  If you were to rely exclusively on what’s documented, you would note that typically erasable whiteboards are to be used for such situations and so you would consider that to be low risk.  But the risk increases significantly if you move past the “thing” you’re looking at and poke around a bit.  Assuming that they do use a scratch pad on occasion, what happens to that piece of paper after the transaction is completed?   Is it thrown out and if so, is it placed in a secured bin or in the regular trash?  And so I advised the client to approach the assessment like it’s an Easter egg hunt.  Deviations and violations are always swirling about (we’re human, we make mistakes) and as part of the assessment process you should be looking for where that might be happening.  But the best part of the Easter egg approach is that it gets everyone involved in the conversation thinking a bit outside the box and that’s where the really neat information is found.  And really, in the end, that’s what a well-run risk assessment should be all about: seeking out and measuring risk wherever it may be hiding.

When the PCI conversation concluded I was feeling a bit energized because in brainstorming with the client I had the ideas flowing freely.   And than I recalled the first conversation and I could almost hear the screeching tires and smell the burning rubber where my creativity came to a complete halt.  It’s amazing to me how the term “risk assessment” can mean completely different things to different people.  I was as impressed with my client asking for guidance and wanting to get it right as I was disappointed in my colleague for having it all wrong.  This wasn’t an issue of principle but rather results; how can you properly manage this thing called risk if you don’t even know how to begin looking for it?

Speaking of risk, check back next week when I jump back onto the GRC train of thought and bring you up to speed with something I’m working on.