Home > IT Blogs > Security Bytes > Web app security scanners not finding everything, study says
>>VIEW ALL POSTS  

Security Bytes

« Spike in failed SSH logins could be beginnings of a coordinated attack, ISC says
Alleged DDoS attack knocks Rockies’ ticket sales site offline »
Oct 23 2007   9:18AM GMT

Web app security scanners not finding everything, study says



Posted by: Bill Brenner
Application Security, Information Security Threats, Security Management

In this age of Web 2.0-based attacks, companies are turning to a variety of Web application security scanners to help them find and fix security holes. But according to a study conducted by independent security consultant Larry Suto, some of these scanners are overlooking quite a few vulnerabilities.

The report is accessible via the ha.ckers.org blog and looks at three tools in particular — NTOSpider, AppScan (IBM/Watchfire) and WebInspect (HP/Spi Dynamics). Of the three, he said:

– NTOSpider found 227 vulnerabilities with zero false positives.
– AppScan (IBM/Watchfire) found 27 vulnerabilities with five false positives.
– WebInspect (HP/Spi Dynamics) found 12 vulnerabilities with 13 false positives.

Now, to be fair, this is based on one man’s research and isn’t necessarily the ultimate verdict on how effective these tools are. I should also point out that the study was flagged by the vendor who fared best, NT OBJECTives Inc. CEO Matthew L. Cohen.

My purpose for flagging this is to get a discussion going among researchers and users alike as to which Web application scanning tools they use and which ones are the best or worse.

I want to pinpoint the common strengths and weaknesses of these tools and hopefully offer IT professionals some useful guidance as a result. This is a terribly important topic, given all the Web 2.0 threats we’ve been writing about of late.

So don’t be shy — let me know what you think.

  Bookmark and Share     Comment     RSS Feed     Email a friend

Comment on this Post


You must be logged-in to post a comment. Log-in/Register

Shagghie  |   Oct 23 2007   2:04PM GMT

What does it mean above when it says the study was ‘flagged’ by CEO Matt Cohen? Larry Suto, if I recall, is an independent researcher, yes? Just curious.
I think the study is very useful in certain contexts, and not in others. Certainly in my experience NTO Spider has always been able to crawl much deeper into websites. And I’ve only ever seen it have false positives a few times. One evaluation of Web Inspect had tons of false positives, which makes for a PITA for developers and can do a web dev team more harm than good tracking down ghosts. Have never used the third application so cannot comment.
Ultimately, running both NTO and SPI might be the best approach, although the SPI reporting is not nearly as developer-useful as NTO Spider’s reporting. Where mitigation is concerned, NTO all they way, imho. One thing’s for sure, any discussion out of this study will be useful to the community as a whole, and I’m glad to see Larry taking the time to do this and manually verify ‘found vulns’ and document his findings. We can all only benefit from that type of effort, and I encourage more researchers to attack this area of discipline!