Information Security Finally Gains Objective Measure of Practical Skills and Knowledge Through Serious Assessment Tool
Posted by: Ed Tittel
I’m bemused. A long, long time ago in another life, I studied anthropology at Princeton and the University of Texas at Austin: at the latter school, I had a “near-PhD experience” in that subject, in fact. After talking with some principals at TeleCommunication Systems (TCS) in Maryland last week, I found myself recalling a long-forgotten encounter with Chairman Mao’s Little Red Book in the context of the relationship between theory and practice in understanding and evaluating human behavior and cultural beliefs. In particular, I was reminded of this statement of his: “Knowledge begins with practice, and theoretical knowledge which is acquired through practice must then return to practice. The active function of knowledge manifests itself not only in the active leap from perceptual to rational knowledge, but – and this is more important – it must manifest itself in the leap from rational knowledge to revolutionary practice.” Don’t get me wrong: I’m neither a closet Marxist-Leninist, nor am I am proponent of dialectical materialism. What draws me to the Chairman’s aphorism is the strange situation in the field of information security certification nowadays, where the vast majority of credentials – including highly-regarded certs like the CISSP and the CISM, for example – tend to focus on theory and to treat practice at arm’s length, particularly in their examinations of their certification candidates. And even those credentials which do include a performance-based component tend to do so more in the framework of specific scenarios (what you might called “canned security situations”) rather than more open-ended diagnosis and mitigation situations (like what CCIEs routinely encounter when taking that infamous and notoriously difficult lab exam, or the strenuous performance-based exams typical for more senior Red Hat credentials like the RHCE and RHCA).
Little Red Book to the left; TCS PerformanScore “symbolic gauge” to the right.
That’s why I was both intrigued and impressed to hear from TCS about their trademarked PerformanScore toolset to help address increasing needs – especially amongst those elements in our armed forces charged with engaging in cyber warfare – for qualified cybersecurity professionals who are prepared to engage in real-time information security encounters where lives, property, and critical infrastructure elements may hang in the balance of the resulting outcomes from those encounters. As I understand it, the PerformanScore approach is designed specifically to measure and assess critical information security skills and knowledge in a live environment. This approach takes candidates through three phases of measurement, in fact:
- Assessment: Provides metrics to enable the assessment of individuals or teams based on industry standards and specific organizational requirements (often characterized in the military as “missions”). The result is a competency benchmark that identifies strengths and weaknesses, with specific recommendations for training and mitigation when areas fall below certain thresholds. According to TCS, this testing instrument can even recommend specific training and/or additional performance-based testing to make sure that candidates reach or exceed required skills and knowledge thresholds to meet mission requirements.
- Learning: PerformanScore delivers specific and tailored feedback to both candidates and their instructors on a candidate’s strengths, and detailed feedback on areas where candidates need improvement. This approach lets instructors offer remediation or repetition where warranted, and gives them the opportunity to refocus and restructure training materials and exercises to meet a candidate’s or team’s specific needs.
- Testing: The testing facilities integrated into the PerformanScore environment provides managers in both technical and non-technical areas in-depth analyses of candidates’ skills and abilities. This offers ongoing insight into candidates’ suitability for inclusion in specific teams or on certain missions, and helps managers ensure the best fit between the individuals available to them and the missions that must be accomplished by them.
Because the PerformanScore methodology is vendor-neutral TCS can customize its coverage to match that provided by existing knowledge-based exams (including all the major and well-recognized information security certifications). In addition, however, PerformanScore is flexible enough to incorporate and accommodate additional mission-specific performance-based knowledge and skills requirements as well. In fact, TCS informs me that their approach is open-ended enough to also be applied outside the somewhat narrow (if extremely important) area of information security/information assurance, particularly in areas well-suited for training and testing based on use of learning labs or simulated environments based on both vendor-neutral and vendor-specific technologies.
Anybody who’s read my blog for more than a little while knows that I’m a strong proponent of and believer in performance-based testing as the strongest foundation for meaningful IT certifications. I’m incredibly intrigued by what TCS might have to offer here – enough so that I’ve begged my way into one of their classes in Maryland later this summer to experience their approach, implementation, and testing tools for myself. If their methodology and automation tools can deliver even half of what’s described in their product literature and information (see the PerformanScore pages for more info), it has the potential to remake the IT certification business as it currently stands. In particular – getting back to the Chairman and his Little Red Book– it has the potential to see that theory and practice are trained and tested in the right kind of balance to ensure that certified professionals not only “know their stuff” but that they can “do the job” or “handle the mission.” In the final analysis those latter qualifications are what really matters most, particularly in matters of war and commerce.