There are as many definitions of pentest and penetration testing as there are google search results. (Some 10,700,00 or so). The problem is, there doesn’t seem to be a standard definition of what constitutes penetration testing.
As a result, there are hundreds of companies promoting their version of a “pentest,” and a wide variety of prices given for the proposed “service.” If you’re looking for “a penetration test,” you can spend hours reading about it on various vendor sites. But what are you really getting? It can vary. A LOT.
A couple of years ago one of our banking clients proudly informed us that he had commissioned a “penetration test” quarterly from the same company that managed their firewall. (Yes, I smelled a rat.)
I took a look at the contract, which did, indeed, provide a “penetration test” quarterly, and examined one of their previous reports.
I recognized the format of the report – it was output from a Nessus scan (back when Nessus used to be free). So this company was testing itself with a free product and charging the bank. Nice.
It was a nice report, and the client was happy with it. He was convinced he was going the extra mile to protect his bank. (Hopefully, he’s not still doing this.) I tried to explain to him the difference between penetration testing and a vulnerability scan, but it was hard going. Especially when he had been sold on the scan being the test.
It’s embarrassing when I see my own genre out to so blatantly make a buck. Right up there with “SAS 70 certification.” Then there’s the folks that come in with a lowball bid just to build business in a market they don’t have any traction in. They make us all look bad, don’t they?
So, what is a “penetration test?” Some of it depends on who is asking. The organization that is looking to acquire one really needs to know what they need to learn from the test. There is no passing grade, unfortunately.
Next: Let’s talk terms