Web services testing

Functional Testing of Web Services: Part VI

See: I didn’t make you wait too long for that SOAP submission tool. Lucky for all of us those gurus that write FireFox add-ons came up with one that does just what we want!

Called Poster, it’s described as “A developer tool for interacting with web services and other web resources that lets you make HTTP requests, set the entity body, and content type. This allows you to interact with web services and inspect the results.”

Sweet!

I won’t go into great detail here, since it’s a very straightforward concept and you can read all about it here on the FireFox add-ons page.

The UI is clean and works with various types of content:

Hmmm…I wonder if mega-humungo-giganticorp Microsoft has anything like this?

Functional Testing of Web Services: Part V

In my never-ending (or so it seems) quest for a cool set of tools for testing web services, I have come across another free(ware)bie, TCPMonitor. You can download a version from here.

First, define a Port to be used for intercepting requests/responses on your local test PC. Choose any port number that is not currently in use, like 8888:

Target Hostname is the name of the server hosting the web service to be tested.

Target Port # is the port where the web service is found.

This info can be easily found in the original web service URL. For example:

http://was-server-04:9085/services/MyWebService

When you use a tool (like VuGen) to import a WSDL or submit a SOAP request and you want to capture all the data that is exchanged between the client and the web service, you simply modify the URL to point instead to the TCPMonitor “interceptor” on your local PC. For example, modifying the URL above, we would get:

http://localhost:8888/services/MyWebService

The web service doesn’t care where the request comes from, and it sends its response right back to the local port. You then get a monitor console readout of your request (in the top pane, where it says “POST / “) and its response (in the bottom pane). For example:

For ease of viewing, make sure the ‘XML Format’ box is checked.

After you capture the information, click the ‘Save’ button to store the request and response in a text file. This file contains information about the port and server settings, the body of the request, and the body of the response. Here’s our example above, which you might use to trace the error:

Not too shabby for freeware!

Since TCPMonitor is a Java applet, you can even run it on your PC directly from the download website.

You might ask, “Soooo, if I don’t have VuGen, and I need a tool for actually submitting a SOAP request so I can use TCPMon to monitor the transaction, where might I find one that’s simple and free?” I’m glad you asked.

But that’s a subject for another blog!

Functional Testing of Web Services: Part IV

Two other web service test tools which deserve honorable mention in our comparison tests were: soapUI, available as Open Source or in a pro version distributed by eviware; and HP’s QuickTest Professional Web Services Add-In.

soapUI

This tool does not require a detailed knowledge of complicated technologies like .NET. Although the user interface is not as intuitive as that of some other tools, a tester who has some experience in programming can learn to create, organize and perform test activities fairly quickly.

soapUI allows structuring your test project into test suites that contain test cases, which can contain test steps. This structure is well-managed: you can add, modify, delete and change the order of every item in the structure. soapUI provides the tools to manage and run your test cases, and to view the SOAP responses to your test requests. You can even include limited load testing scenarios. For added flexibility, soapUI supports Groovy Script, a scripting language similar to Java.

QuickTest Pro Web-Services Add-In

QuickTest Pro with the Web Services Add-In offers a wizard to create a test object that represents the Web Service and port you want to test, and inserts the relevant steps directly into your test or component. This accelerates the process of designing a basic test that checks the operations that your Web service supports. You then update the wizard-generated steps of your test or component by replacing the generated argument values with known valid values, updating the expected values, and selecting the nodes you want to check in your checkpoints.

A QTP XML Warehouse setting is used to store request data, and a QTP Object Repository is used to store responses as checkpoints for verification.

The wizard generates a generic XML structure as a place holder for the expected XML return values. However, before you can actually run your test or component, you must replace the default values with the appropriate values for your test. A valid SOAP request can be imported into the XML Warehouse as a separate step, and a separate XML checkpoint would need to be created for each test case. Defining tests is a little cumbersome and the UI is a little clumsy, but if you’re already using QTP for functional testing, including web service regression tests in your test suites is an easy option.

Functional Testing of Web Services: Part III

After determining the requirements for functional testing of web services, it was necessary to examine some of the readily available test tools. My employer depends on the HP (formerly Mercury Interactive) suite for most test automation, and our performance test gurus recommended a LoadRunner component called VUGen — for Virtual User Generator. Among others, we also reviewed SOAPSonar from CrossCheck Networks. These two tools ranked the highest with our test-tool-testers for overall usability.

As we discussed previously, good web service test tools all share certain features:

• Scan WSDL (file or URL) to create request and response structure
• Contain user interface to XML for request and response editing
  • Response stored as a checkpoint
  • Regression test compares checkpoint to runtime values
• Allow parameterized values for request and corresponding response (data-driven)

The comparison tests were relatively simple: given a SOAP request provided by the web service developers, use each tool to create a simple data-driven automated test which submits the request to the web service and then validates the response against a previous known good run. A typical regression scenario!

The initial results were also straightforward. SOAPSonar actually ranked highest for ease of use, but does not offer integration (at least, not out of the box) with HP Quality Center or HP Performance Center.

Because of its relative ease of use and its integration (of course) with HP Quality Center and Performance Center, HP VUGen was selected as the preferred engine for our enterprise web service testing solution.

a caveat

Although VUGen can be installed and run standalone, HP does not sanction its standalone use for functional testing. So, to utilize the VUGen engine for functional testing of web services, HP has developed a Service Test tool as the licensed front-end. In addition to integration with Quality Center, the Service Test tool also allows the user to create a web services test as a Business Process Test (BPT). For those not familiar with Quality Center, this allows users not well-versed in test tool technology to create complex test plans by drag-n-drop of BPT components stored in the Business Process module database.

So your choice of a test tool depends on your current automation environment, your budget, and the expertise of your tool users. The same process for developing and executing SOAP test cases applies, regardless of the tool chosen.

In future postings, we will examine the capabilities of some of the other tools which were included in this investigation.

Keep automating!

Functional Testing of Web Services: Part II

In a previous posting, we examined the definition and basic functionality of a web service. Now let’s discuss the process behind testing web services. In order to perform any effective automated functional testing, two components are absolutely essential: good test data, and an adequate test tool.

Based on what we have already learned, the Data Requirements for testing web services should be fairly self-evident:

• The location of the WSDL file. This takes the form of a web URL, or the UNC path to a stored file on the network.
• The location of the web service itself, in the form of a URL.
• One or more valid SOAP requests in XML format. A SOAP request is used to exercise the operations presented by the web service.
• Identification of which data elements contain required values, and what those values are.
• Specification of the sequence in which operations must be called, if required.

The specifications which developers use to create the web service should contain all of this information, at least in the form of a template or simple unit test.

A good web service test tool allows us to easily manipulate the request data, submit the request to the web service, and then analyze the response. The Test Tool Requirements are summarized here:

• Allow import of the WSDL file to define the structure of the service.
• Expose the operations offered by the web service.
• Allow import of the XML SOAP request.
• Provide editing of XML structures (e.g., the imported SOAP request) for test data input.
• Support parameterization of element values for data-driven tests.
• Provide the ability to submit the SOAP request using a protocol (e.g., HTTP) supported by the web service.
• Include a method for capturing and storing the response.
• Support the creation of baseline checkpoints for regression testing. This allows the tool to verify a runtime response against a previously stored, known good response.

Regardless of the test tool selected, the process for creating and executing functional tests of web services is pretty much the same:

1. Import the web service WSDL file into the test tool

2. The test tool scans and parses the WSDL file to expose the operations

3. Choose the operation to call based on requirements of the test case

4. Provide input data using the test tool:

• Import the SOAP request into the test tool, and/or
• Edit the XML request generated by the test tool or SOAP import

5. Parameterize the input data

6. Execute the operation call(s):

• 1st Iteration: store response as baseline
• 2nd+ Iterations: capture response from web service for verification

7. Verify captured response against stored baseline (expected) response:

• Structure - text check of XML tags
• Content - XML checkpoint

Functional Testing of Web Services: Part I

Usually when you think of functional testing, you visualize some sort of User Interface (UI) for inputting data: a well-defined form or page with fields and arrows and boxes and other eye-candy.

And most modern functional test tools were designed around this object-action paradigm.

But what happens in today’s brave new world of Service Oriented Architecture (SOA)? Data inputs still exist, but they are no longer necessarily visual objects, and the web interface to this incomprehensible mechanism is now a mysterious black box called a “Web Service”.

This world was indeed new to me, but my job required that I decide how to incorporate web service testing into an automated functional regression suite. So I spent some time learning what I could about testing in this alien environment — in fact, the learning continues, and I’d like to share my ongoing adventures.

Please forgive any technical blunders: I am definitely a web service neophyte (in all meanings of that word).

I tend to think of programming in terms of analogies, and “service” suggests a restaurant to me (guess I’m hungry).

First, you find a restaurant by its address. Then you go in, sit down, and look at the menu. The menu tells you all the different culinary delights that the restaurant has to offer. You request the entree or meal that suits you, place your order, and the wait-person responds with a meal that is prepared and delivered for your consumption.

Analogously, a web service has an address in the form of a URL. There is a menu written in Web Services Definition Language (WSDL) which describes the items offered by the service, called methods or operations. The order is placed using the XML file format and usually takes the form of a SOAP (which apparently stands for nothing anymore) request (which kind of breaks the restaurant analogy since SOAP should NOT be a part of your meal but bear with me, please). Once this request is submitted, the web service prepares an XML response and delivers it back to the requestor for consumption.

Now, dragging this topic back into the realm of functional testing, the XML SOAP request contains the data inputs for the web service, in much the same way that a form or screen is populated with test data. The XML response contains all the data for an expected response and can be used to determine if a service request test case passes or fails.

That’s the big picture.

In coming installments, we will look at some tools used for testing web services, and processes needed for developing test cases.