Posted by: David Schneier
assessment, Audit, compliance, GLBA, PCI, Regulatory Compliance, risk, risk assessment
I had two great conversations this week regarding risk assessments (jeez, does that ever sound geeky).
The first conversation centered on what an associate was expecting to accomplish via the risk assessment process and the second one was a general conversation about the proper approach to conducting one in support of PCI. Fundamentally, at some higher level a risk assessment looks like a risk assessment regardless of its intended purpose. But at the ground level, there are huge swings and variances between different types of assessments and how they’re conducted.
The first go-round centered on GLBA and was with a fellow practitioner I’ve worked with before on audits. The conversation started because my associate was preparing to conduct an information security risk assessment later this month and was talking about all of the planning effort required before, during and after the fieldwork. I commented that it sounded more like he was conducting an audit rather than an assessment; his reply “same difference.” And I thought to myself “funny, it sounded like he just said that an audit and an assessment were the same thing.” So I asked him to clarify and he did by stating that the only difference between the two was whether or not you needed to create work papers. I was stunned by this shocking admission and obvious lack of understanding of our trade. An audit tests in-place controls to determine if they’re working as designed and are sufficient to address the related risks those controls are intended to manage or mitigate. An assessment examines objectives or assets, identifies risks to either and then looks to see if there are controls in-place to manage those risks. I can get more granular than that but at its most basic level, that’s how it’s supposed to work. Risk assessments identify risks and related control activities and audits test to see if the necessary controls are present and that they’re working properly. He also shared with me that he uses the same general outline of questions and topics for both, that they’re both based on FFIEC guidance and that it’s exactly what the examiners are expecting.
OK, first of all, the examiners are not expecting that the same work is being done for both an audit and an assessment. I’ve spoken with enough of them to know that plus I’ve heard several of them stand before large audiences and explain the nuances of both and so I know they get the differences. Second of all, the entire purpose of a GLBA-based assessment is to ensure that at some acceptable frequency (which secretly means annually) each financial institution should conduct a thorough assessment of their infrastructure (not just IT, by the way) and identify and measure risks and threats to the customer data they manage and store. If you move right past the assessment and conduct an audit you’ll never truly know if you’re missing something because you’re only testing what you can see. I’ve participated in and conducted enough of these to know this to be true. And if you conduct an assessment like it’s an audit, you have the same basic issue. Thus the reason why you typically schedule an assessment first and an audit second. I was about to explain this to my associate but decided against it; I somehow thought it would’ve fallen on deaf ears.
In the second conversation I participated in around risk assessments it was all about PCI. This one was much easier because the team running the assessment had built a high-level approach based on guidance from the PCI DSS Council itself and displayed a clear understanding of what they were to do. While they hadn’t developed any of the templates to be used (it’s not due to be conducted until next month) they were well into the planning stages. My role was to simply listen and provide any meaningful feedback. The one potential gap in their intended approach is that they were taking an asset-based approach to the assessment. What that effectively means is that they only look for and attempt to measure risks around what they can see be it a piece of equipment or a documented process. But if there’s something that doesn’t appear on an inventory, a network diagram or exist in a binder somewhere, they’ll potentially miss it. A great PCI for-instance is what happens when a customer service rep writes down sensitive information on a pad because their system is hanging and they’re trying to keep the call within acceptable time frames? It’s not supposed to happen but I’ve been in a dozen call centers over the past few years and have personally witnessed it happening in almost all of them. If you were to rely exclusively on what’s documented, you would note that typically erasable whiteboards are to be used for such situations and so you would consider that to be low risk. But the risk increases significantly if you move past the “thing” you’re looking at and poke around a bit. Assuming that they do use a scratch pad on occasion, what happens to that piece of paper after the transaction is completed? Is it thrown out and if so, is it placed in a secured bin or in the regular trash? And so I advised the client to approach the assessment like it’s an Easter egg hunt. Deviations and violations are always swirling about (we’re human, we make mistakes) and as part of the assessment process you should be looking for where that might be happening. But the best part of the Easter egg approach is that it gets everyone involved in the conversation thinking a bit outside the box and that’s where the really neat information is found. And really, in the end, that’s what a well-run risk assessment should be all about: seeking out and measuring risk wherever it may be hiding.
When the PCI conversation concluded I was feeling a bit energized because in brainstorming with the client I had the ideas flowing freely. And than I recalled the first conversation and I could almost hear the screeching tires and smell the burning rubber where my creativity came to a complete halt. It’s amazing to me how the term “risk assessment” can mean completely different things to different people. I was as impressed with my client asking for guidance and wanting to get it right as I was disappointed in my colleague for having it all wrong. This wasn’t an issue of principle but rather results; how can you properly manage this thing called risk if you don’t even know how to begin looking for it?
Speaking of risk, check back next week when I jump back onto the GRC train of thought and bring you up to speed with something I’m working on.