Posted by: David Schneier
assessment, Audit, audit plan, audit program, CISA, CISSP, regulations, Regulatory Compliance, risk, risk management
It’s been a while since my last post as I’m in hunker-down mode as we prepare our next compliance software offering for release. But in the midst of my coding/testing insanity, a conversation occurred that brought up the value of certifications that I haven’t been able to completely let go of.
On occasion I receive phone calls from recruiters looking for resources to take on contract work. An important part of our practice is comprised of services work and so I’ll look into the opportunity; if it’s consistent with what we do and it’s a good fit for someone in our practice we’ll try and make it work. In this particular instance, the hiring client had some very specific requirements that presented as unusual. It wasn’t so much in what they were looking for from a definition perspective but rather their method of vetting the candidate. The recruiter told me right up front that any candidate needed to present proof of their certifications before being considered for the position. In more than a dozen years working in audit and compliance I can’t recall ever being asked right up front for such information and it caught me off guard.
The certification in question was the Certified Information Security Auditor (CISA) designation issued by ISACA. Generally speaking it’s the defacto standard when it comes to my professional space but only because it’s the only one available. While there are a number of IT auditors who also have the CIA designation it’s somewhat rare and unusual. But while it may be the standard cert for IT auditors, it’s certainly not a hard requirement and not something that all practitioners aspire too. I probably know more excellent IT auditors who don’t possess a CISA than I do those who do. I sat for the exam (and passed) back in 2005 because I was looking for a way to bookmark my audit experience; too many recruiters saw my resume and thought of me more as an IT practitioner than as an audit/compliance resource. I wanted to distinguish myself as an auditor and that seemed to be the best, most direct way to do so.
What I learned during the period of time while studying for the exam was that I already knew what was necessary to pass the test. There were a few disciplines covered during the exam that exceeded my knowledge (primarily around cryptography, encryption and key management) but I was okay with that because those were areas I would never pursue work in (we throw that stuff to our CISSP’s). Midway through the exam preparation experience, I questioned the validity of the certification. I genuinely believed that my previous eight years of experience spoke more to my expertise than any certification ever could. A year or so later I came to learn that when ISACA issued new certifications they also allowed for grandfathering – you could simply pay for the certification if you could prove that you already had the experience doing that sort of work. That cemented my opinion that experience was far more significant than the cert (and it also meant that a solid number of CISA’s I knew never had to pass the exam).
Within the first two years after I passed the exam I knew three people with almost no audit experience who studied for and passed the CISA exam because they believed audit and compliance work was their best way to stay employed. None of the three knew how to conduct a risk assessment, develop an audit plan, write an audit program or build work papers after taking the exam, yet all three were CISA’s. With some minor modifications to their resume they could present themselves as true audit professionals. That also cemented my opinion that the certification wasn’t as much of an indicator of ability as I once thought.
I recall a conversation with someone who was an IT audit instructor, but who at the time didn’t possess the CISA certification. His issue with the certification was that he didn’t believe multiple choice exams proved competency because you knew one of the provided answers was correct and so you just needed to be good at taking exams and making educated guesses. I don’t know if I completely agree but I have come to believe that the CISA certification would be that much more meaningful if the candidate had to display a basic ability in conducting the related work. Give them a set of criteria about an environment (e.g. software, networking, etc.), have them create a risk assessment to determine what should be assessed, develop an audit plan based on the identified risks and write the audit programs to test the necessary controls. A panel of reviewers could grade the material and decide if the candidate possesses the necessary competencies. At least with such an approach you would know that if you hire a CISA certified practitioner, they have the skills to do the job. By the way, of the three aforementioned practitioners who are CISA-certified, only one could actually pass such an exam today, three-plus years after having obtained the designation.
And so in an industry where you don’t need a certification to work (unlike the medical or legal professions), I’m not sure that a similar value should be placed on possessing one.