IT Career JumpStart

Oct 11 2010   12:57PM GMT

Check out Kryterion Online Secure Testing

Ed Tittel Ed Tittel Profile: Ed Tittel

The Kryterion online testing  environment adapts cert testing entirely to secure Web-based development and delivery

The Kryterion online testing environment adapts cert testing entirely to secure Web-based development and delivery

As I was pondering this morning’s blog, I stumbled across a reference to a company I’d previously known nothing about — namely, Kryterion Inc., a purveyor of “Online Secure Testing” (as per the banner on their Web site) — in checking out some specifics on the SANS GIAC Certified Forensic Analyst (GCFA) certification. It turns out that one of the principals behind the company is Dr. David Foster, a well-known pioneer in computer-based testing technologies (he ran cert testing development for Novell from 1990-1997, and helped create adaptive and performance based testing technologies still in broad industry use today, equivalent to a geological era later measured in “Internet years.” Dr. Foster also went on to found well-known psychometrics and test design firm Galton Technologies in 1997, and test security firm Caveon in 2003).

In a nutshell what Kryterion offers is a collection of online security technologies that permit the Internet to be used to deliver exams to certification or licensure candidates, without exposing content or exam answers to third-party sniffing or snooping. In addition, Kryterion bills itself as “…a full service provider of customizable assessment and certification products and services.” The company works with organizations of all kinds, including technology vendors, professional associations, educational institutions, and even governmental bodies to “…build and deliver everything from skills tests and simple online assessments to … comprehensive high-stakes worldwide certification program[s]” (I added an “s” to the end of this text snippet to make my quotation gramatically correct).

The really cool thing about the Kryterion technology is that it works for exam development, item creation and vetting, assembly and publishing of exams, and access to results reports using the same enviroment for exam creation, development, and refinement prior to public release that is later used to deliver (and proctor) exams for public delivery to authorized test-takers. The designers even include the ability for customers (technology vendors, professional associations, educational institutions, and so forth) to create customized Web sites that online test takers will use to handle registration, the test taking process (including pre-requisites management, test calendars and availability information), and even retake rules and conditions.  In terms of a different type of Web technology, Kryterion offers a secure content management system (CMS) for certification or licensure exam creation, delivery, maintenance, and reporting — even to the point of offering transcript access to test-takers who’ve registered with a site and who possess account and password information necessary to access their prior history data after taking an exam.

I’m pretty darn curious to learn who (besides SANS) is using Kryterion’s services in the world of IT certification. To that end, I’m forwarding this blog to the Kryterion sales department to see what kind of response it provokes in-house. Although this appears to be the tip of a looming iceberg of entirely Web-based certification and licensure technology adoption, I’m guessing that Prometric and VUE are keenly aware of these efforts, and probably concerned about the long-term implications. Even though many people are lucky enough to live within an hour’s drive of a testing center in the developed world, why bother driving somewhere else when your own PC and Web browser can bring the test center to you? And outside the developed world where access to a testing center may involve one or more days of tedious travel, the “bring it to you” model clearly wins hands-down. I think I see the wave of the future…

5  Comments on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • JodiK
    It seems that the major thrust of this column is that online testing from any location is a secure and convenient alternative that will at some point replace testing in traditional brick and mortar facilities. I think this opinion is somewhat naïve, and ignores many of the reasons that brick and mortar facilities exist in the first place. There are many reasons to physically visit a test center to test, but the glaring one, which will not “go away” with the spread of online testing, is the need for security. The author asks: “why bother driving somewhere else when your own PC and Web browser can bring the test center to you?” Well, the obvious answer to that question is: to ensure that the value of the test – the very reason you are taking it in the first place – is not compromised by wide scale cheating made possible through obvious vulnerabilities in a delivery model that fails to standardize and rigorously control the testing environment. You can possibly argue that online testing from any location using “webcams” for security may be more convenient for test takers, but there is just no way that you can argue it is more secure. Requiring a candidate to physically report to a test center accomplishes a number of things, but most importantly, it enforces security on multiple levels. • For one, the test center administrator validates the identity of the test-taker standing at the center by comparing the individual against a government issued, photo-bearing ID (to make sure that the test-taker is who they claim to be. How can you validate, with remote proctoring, that the person sitting in front of the webcam, hundreds or thousands of miles away, is actually the person whose name will appear on the score report and/or credential? In short, how do you validate that the “Bob Jones” who signed up for your test is the same “Bob Jones” that is sitting in front of the webcam? Or that the person sitting in front of the webcam is not “Bob Jones’” brother? Sure, you can see a guy sitting there taking the test, but how do you know what guy that is? • Next, even though there is a webcam at the candidate’s house mounted to the top of the computer – there are many places a camera can’t see. Many webcams used in remote proctoring are not 360 degree cameras – so there are many places in the room where a candidate could post notes or other materials that are not “picked up” by the camera. Alternatively, since the camera doesn’t pan the room, there could feasibly be another person in the room helping out with the answers. Even with a 360 degree camera, which – as we just said, many are not – it is possible for a candidate to hide books, notes or other materials. Panoramic cameras don’t look up and down, only around. imagine how many “doctors” there would be if we could all take licensure exams from our couch without a proctor roaming around to make sure we were not cheating. • Third, allowing candidates to test in their own environments introduces inconsistencies into the testing process. No two candidates will have the same experience, and this could potentially cause problems for test sponsors as some candidates claim “unfair” conditions or similar. A test center is truly the only way to ensure a stable and consistent – test experience for all candidates taking an exam. Another point the author makes is: “ … outside the developed world where access to a testing center may involve one or more days of tedious travel, the “bring it to you” model clearly wins hands-down.” I don’t think it is even close to “hands down.” In fact, many areas of the developing world, where collaborative learning and group test preparation are culturally engrained, could be adversely impacted by the introduction of remote proctoring and the potential devaluing of the tests delivered in this manner. The author also presumes that other providers are “concerned” about the long term implications of remote proctored testing. This is quite a presumption and one that is untrue. I strongly believe that there will always be a place for secure, proctored testing – especially for those exams that pose a life, health, safety or public security risk if cheating or dishonesty were prevalent. It is implicitly harder to ensure maximum security in an online environment when a candidate is sitting in his basement or in a coffee shop somewhere than if he was sitting in a secure test center, with proctors roaming the room, personal belongings safely stored in a lockers and content “locked down” to prevent theft. Is there a place for remote proctor testing in the future? Sure. But I don’t believe that it will ever be a sufficient replacement for the security and consistency of a brick and mortar center. Jodi K.
    0 pointsBadges:
    report
  • Ed Tittel
    Gee JodiK, it seems to me like you might have an axe to grind here. Will you please tell us where you work and what you do for a living? I have to guess you have some kind of strong ties to a training and test center operation, because they are the most likely to feel threatened by the Kryterion capabilities. In fact, that's why I speculated that Prometric and VUE must be aware of this stuff, and that they may be concerned about it, too. You sound concerned to me, too. And you do make valid points about security. That said, some cert program sponsors (both vendors and organizations) are prepared to shoulder some level of security risk, mostly by makng exams difficult and realistic enough to demand real skills and knowledge to complete them in the time alloted for an exam. In that light, it will also be interesting to see how proof of identity is established and maintained, so as to avoid suggestions (and indeed, the possibility) of impropriety by inviting somebody else to come sit in your chair and take the exam for you. I guess we'll see how this all turns out, eh? Thanks for sharing your opinions. Always nice to hear from somebody who has plenty of them, and isn't afraid to share them. Be nicer for me (and less painful, too) however, if you could avoid the ad hominen attacks and insinuations. All you need to do is state your point, and get on with it! Best wishes, --Ed--
    4,960 pointsBadges:
    report
  • Bradd858
    Hi all, I just finished taking an online test with Kryterion (I searched for this in Google because I was interested in other people's experiences). I'll be honest, my first thought was exactly as JodiK said - couldn't people just plaster notes out of the camera's view? Well, while I'm an honest enough person not to do that, I can tell you after taking this test - even if someone did, it's not easy to do. You first position the camera so it sees you, and both hands. It sees your eyes. A live person is watching you, hearing all sounds in the room. As I fumbled with the camera trying to get the right angle, I bet this was the time when they look most closely, to see if you have notes anywhere. The whole situation of knowing that someone is seeing and hearing everything you do doesn't bode well for cheaters. It's absolutely a different experience, and it's not comfortable that's for sure. I was afraid to look anywhere other than the monitor, thinking that they would think that I had notes of some sort. So that didn't help much, but in the end, it definitely beat finding parking and a test center, and I do feel they do a good job in making people honest. Thumbs up Kryterion!
    0 pointsBadges:
    report
  • Ed Tittel
    Dear Brad858: Thanks for sharing your experiences with us. I'm glad to hear from somebody's who's been through the Kryterion process, and more hopeful now that their model may be viable thanks to your careful reporting. It will be very interesting to see how this all pans out. There's no doubt in my mind that Kryterion has tried very hard to "get things right;" whether they actually have succeeded or not is for the market, and the certfication sponsoring organizations, to decide. Thanks again, --Ed--
    4,960 pointsBadges:
    report
  • Ed Tittel
    Bradd858, we appreciate your direct report based on personal experience, not theory. With all new technologies there is opposition to the status quo. Kryterion has addressing objections to online proctoring for years. Along the way we have acquired, and continue to acquire, savvy clients who won't do business any other way. There is a place for bricks and mortar testing facilities as well as a place for online proctoring. Kryterion offers both and is thus well acquainted with the benefit of both. Ed, feel free to check in.
    0 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: