How to use functional automated test client to perform better load testing?

455 pts.
Tags:
Functional testing
Software Break Points
Software load testing
Software Quality
Software testing
Software testing tools
We have an automation test client which is written for Functional Testing. This client takes inputs like what are the APIS to be called and the inputs, expected outputs, server ip, etc. This helped in our functional testing and is stable. Now to do load testing, We have added just 2 more parameters (Number of Threads, Loop count) to this functional test client. Number of threads greater than 1 implies all the APIs will be called in parallel my multiple threads. Loop count implies all threads will be in loop for this count. This logic presently working fine, we have found many issues (concurrency issues, memory leaks, etc) in the product/s while load testing, which were otherwise cannot be found in functional testing. This also gives the average timing info of each API call. Now my doubt is: Whether this approach is correct & complete or not for load testing? Since we found many issues which can be actually found under load, Can we say this functional test tool is also a complete load test tool? Do u see any presence of gaps in our load testing? What else can we do in improving our stable functional test client to perform better load testing ? Thanks inadvance.
ASKED: November 15, 2008  7:04 AM
UPDATED: November 19, 2008  6:46 PM

Answer Wiki

Thanks. We'll let you know when a new response is added.

This is a good approach, IMO.

First, I have one question: When a thread makes many API calls in a loop, it calls the same API at each iteration ? If so, I would configure the test client in order to call a different API in each iteration, randomly.

Second, since you have timing information, I would also configure the test client to make many api calls in parallel, with each thread calling the API which takes more time to complete (all threads calling that same api).

Third, I would configure some threads to make the api calls with incorrect parameters while some other threads continue calling APIs with correct parameters, so you can test if the handling of incorrect api calls does not affect the way it manages the correct ones.

———————————————

Discuss This Question: 2  Replies

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
  • Saimadhu
    Thanks carlos for your reply. Regarding your question: In each iteration, thread calls the same API set (which represents a work flow). This is to check the consistency of each API in functionality. Regarding calling different API in each iteration: To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow and another test client with another work flow. This is to trouble-shoot the issues in quick time, if any. Do u see any gap in this approach? Regarding calling with invalid parameters, valid parameters: To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow (say positive inputs) and another test client with another work flow (say negative inputs). This is again to trouble-shoot the issues in quick time, if any. Do u see any gap in this approach? Regarding all threads calling the same API (say creating user): Yes, we are testing with this setup. FYI, each test client can have API sets with multiple work flows (positive, negative etc) as inputs. This combination we will be using regulary in functional testing. Presently for load testing, we are separating the work-flows to trouble-shoot the issues. This is because, in the present setup itself (1 work flow at a time) we are seeing some issues under load run. So i think only after the stability of individual work flows (under load), we can move to multiple work flows with 1 test client. Do u see any gap in this approach? Measurements under load: Also, when we run the test client under normal & heavy load, we are taking the following measurements: memory leaks (using pmap) cpu utilization (using prstat) Thread count (using pstack) opened file descriptors (using lsof) connections established, released &leaked (using netstat -a)) context switches (using mpstat) In windows, we are using the perfmon to collect the above relevant info. Are we missing any other imporant measurement/s ?
    455 pointsBadges:
    report
  • carlosdl
    I don't really see any gap, and I think your test client is a good option for load testing also. Regards,
    65,110 pointsBadges:
    report

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following