Functional Testing of Web Services: Part III

After determining the requirements for functional testing of web services, it was necessary to examine some of the readily available test tools. My employer depends on the HP (formerly Mercury Interactive) suite for most test automation, and our performance test gurus recommended a LoadRunner component called VUGen — for Virtual User Generator. Among others, we also reviewed SOAPSonar from CrossCheck Networks. These two tools ranked the highest with our test-tool-testers for overall usability.

As we discussed previously, good web service test tools all share certain features:

• Scan WSDL (file or URL) to create request and response structure
• Contain user interface to XML for request and response editing
  • Response stored as a checkpoint
  • Regression test compares checkpoint to runtime values
• Allow parameterized values for request and corresponding response (data-driven)

The comparison tests were relatively simple: given a SOAP request provided by the web service developers, use each tool to create a simple data-driven automated test which submits the request to the web service and then validates the response against a previous known good run. A typical regression scenario!

The initial results were also straightforward. SOAPSonar actually ranked highest for ease of use, but does not offer integration (at least, not out of the box) with HP Quality Center or HP Performance Center.

Because of its relative ease of use and its integration (of course) with HP Quality Center and Performance Center, HP VUGen was selected as the preferred engine for our enterprise web service testing solution.

a caveat

Although VUGen can be installed and run standalone, HP does not sanction its standalone use for functional testing. So, to utilize the VUGen engine for functional testing of web services, HP has developed a Service Test tool as the licensed front-end. In addition to integration with Quality Center, the Service Test tool also allows the user to create a web services test as a Business Process Test (BPT). For those not familiar with Quality Center, this allows users not well-versed in test tool technology to create complex test plans by drag-n-drop of BPT components stored in the Business Process module database.

So your choice of a test tool depends on your current automation environment, your budget, and the expertise of your tool users. The same process for developing and executing SOAP test cases applies, regardless of the tool chosen.

In future postings, we will examine the capabilities of some of the other tools which were included in this investigation.

Keep automating!