Home > IT Answers > Development > How to use functional automated test client t...
Help

Question

Asked:
Asked By:
Nov 15 2008   7:04 AM GMT
Saimadhu   365 pts.

How to use functional automated test client to perform better load testing?


Software testing, Software testing tools, Software Quality, Functional testing, Software Break Points, Software load testing

We have an automation test client which is written for Functional Testing.
This client takes inputs like what are the APIS to be called and the inputs, expected outputs, server ip, etc. This helped in our functional testing and is stable.

Now to do load testing,
We have added just 2 more parameters (Number of Threads, Loop count) to this functional test client.

Number of threads greater than 1 implies all the APIs will be called in parallel my multiple threads.
Loop count implies all threads will be in loop for this count.

This logic presently working fine, we have found many issues (concurrency issues, memory leaks, etc) in the product/s while load testing, which were otherwise cannot be found in functional testing.
This also gives the average timing info of each API call.

Now my doubt is:
Whether this approach is correct & complete or not for load testing?

Since we found many issues which can be actually found under load, Can we say this functional test tool is also a complete load test tool?

Do u see any presence of gaps in our load testing? What else can we do in improving our stable functional test client to perform better load testing ?

Thanks inadvance.

Subscribe to Alerts! Get questions and answers delivered to your Inbox.


E-mail me updates on this question



   SUBSCRIBE

hidden modal window
Help

Answer Wiki (Improve, edit or add to this answer)

IMPROVE THIS ANSWER
View History
 RATE THIS ANSWER
+1
Click to Vote:
  •   1
  •  0
Last Answered: Nov 17 2008  3:37 PM GMT  by Carlosdl   21920 pts.




This is a good approach, IMO.

First, I have one question: When a thread makes many API calls in a loop, it calls the same API at each iteration ? If so, I would configure the test client in order to call a different API in each iteration, randomly.

Second, since you have timing information, I would also configure the test client to make many api calls in parallel, with each thread calling the API which takes more time to complete (all threads calling that same api).

Third, I would configure some threads to make the api calls with incorrect parameters while some other threads continue calling APIs with correct parameters, so you can test if the handling of incorrect api calls does not affect the way it manages the correct ones.

---------------------------------------------
  • AddThis Social Bookmark Button

Browse more Questions and Answers on Development.

Looking for relevant Development Whitepapers? Visit the SearchWinDevelopment.com Research Library.


RSS - Discussions Help

Discuss This Answer


You must be logged-in to discuss a question. Log-in/Register

Saimadhu   365 pts.  |   Nov 19 2008  6:23AM GMT

Thanks carlos for your reply.

Regarding your question:
In each iteration, thread calls the same API set (which represents a work flow).
This is to check the consistency of each API in functionality.

Regarding calling different API in each iteration:
To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow and another test client with another work flow.
This is to trouble-shoot the issues in quick time, if any.

Do u see any gap in this approach?

Regarding calling with invalid parameters, valid parameters:
To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow (say positive inputs) and another test client with another work flow (say negative inputs).
This is again to trouble-shoot the issues in quick time, if any.

Do u see any gap in this approach?

Regarding all threads calling the same API (say creating user):
Yes, we are testing with this setup.

FYI, each test client can have API sets with multiple work flows (positive, negative etc) as inputs. This combination we will be using regulary in functional testing.

Presently for load testing, we are separating the work-flows to trouble-shoot the issues.
This is because, in the present setup itself (1 work flow at a time) we are seeing some issues under load run. So i think only after the stability of individual work flows (under load), we can move to multiple work flows with 1 test client.

Do u see any gap in this approach?

Measurements under load:
Also, when we run the test client under normal & heavy load, we are taking the following measurements:

memory leaks (using pmap)
cpu utilization (using prstat)
Thread count (using pstack)
opened file descriptors (using lsof)
connections established, released &leaked (using netstat -a))
context switches (using mpstat)

In windows, we are using the perfmon to collect the above relevant info.

Are we missing any other imporant measurement/s ?

 

Carlosdl   21920 pts.  |   Nov 19 2008  6:46PM GMT

I don’t really see any gap, and I think your test client is a good option for load testing also.
Regards,