Comments on: How to use functional automated test client to perform better load testing? http://itknowledgeexchange.techtarget.com/itanswers/how-to-use-functional-automated-test-client-to-perform-better-load-testing/ Mon, 09 Nov 2009 13:30:36 +0000 http://wordpress.org/?v=2.6.2 By: Carlosdl http://itknowledgeexchange.techtarget.com/itanswers/how-to-use-functional-automated-test-client-to-perform-better-load-testing/#comment-57901 Carlosdl Wed, 19 Nov 2008 18:46:03 +0000 #comment-57901 I don't really see any gap, and I think your test client is a good option for load testing also. Regards, I don’t really see any gap, and I think your test client is a good option for load testing also.
Regards,

]]> By: Saimadhu http://itknowledgeexchange.techtarget.com/itanswers/how-to-use-functional-automated-test-client-to-perform-better-load-testing/#comment-57873 Saimadhu Wed, 19 Nov 2008 06:23:41 +0000 #comment-57873 Thanks carlos for your reply. [B]Regarding your question:[/B] In each iteration, thread calls the same API set (which represents a work flow). This is to check the consistency of each API in functionality. [B]Regarding calling different API in each iteration:[/B] To test different work flows in [B]parallel[/B], we are invoking multiple test clients. i.e. 1 test client with 1 work flow and another test client with another work flow. This is to trouble-shoot the issues in quick time, if any. Do u see any gap in this approach? [B]Regarding calling with invalid parameters, valid parameters:[/B] To test different work flows in [B]parallel[/B], we are invoking multiple test clients. i.e. 1 test client with 1 work flow (say positive inputs) and another test client with another work flow (say negative inputs). This is again to trouble-shoot the issues in quick time, if any. Do u see any gap in this approach? [B]Regarding all threads calling the same API (say creating user):[/B] Yes, we are testing with this setup. [B]FYI, [/B]each test client can have API sets with multiple work flows (positive, negative etc) as inputs. This combination we will be using regulary in functional testing. Presently for load testing, we are separating the work-flows to trouble-shoot the issues. This is because, in the present setup itself (1 work flow at a time) we are seeing some issues under load run. So i think only after the stability of individual work flows (under load), we can move to multiple work flows with 1 test client. Do u see any gap in this approach? [B]Measurements under load:[/B] Also, when we run the test client under normal & heavy load, we are taking the following measurements: memory leaks (using pmap) cpu utilization (using prstat) Thread count (using pstack) opened file descriptors (using lsof) connections established, released &leaked (using netstat -a)) context switches (using mpstat) In windows, we are using the perfmon to collect the above relevant info. Are we missing any other imporant measurement/s ? Thanks carlos for your reply.

Regarding your question:
In each iteration, thread calls the same API set (which represents a work flow).
This is to check the consistency of each API in functionality.

Regarding calling different API in each iteration:
To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow and another test client with another work flow.
This is to trouble-shoot the issues in quick time, if any.

Do u see any gap in this approach?

Regarding calling with invalid parameters, valid parameters:
To test different work flows in parallel, we are invoking multiple test clients. i.e. 1 test client with 1 work flow (say positive inputs) and another test client with another work flow (say negative inputs).
This is again to trouble-shoot the issues in quick time, if any.

Do u see any gap in this approach?

Regarding all threads calling the same API (say creating user):
Yes, we are testing with this setup.

FYI, each test client can have API sets with multiple work flows (positive, negative etc) as inputs. This combination we will be using regulary in functional testing.

Presently for load testing, we are separating the work-flows to trouble-shoot the issues.
This is because, in the present setup itself (1 work flow at a time) we are seeing some issues under load run. So i think only after the stability of individual work flows (under load), we can move to multiple work flows with 1 test client.

Do u see any gap in this approach?

Measurements under load:
Also, when we run the test client under normal & heavy load, we are taking the following measurements:

memory leaks (using pmap)
cpu utilization (using prstat)
Thread count (using pstack)
opened file descriptors (using lsof)
connections established, released &leaked (using netstat -a))
context switches (using mpstat)

In windows, we are using the perfmon to collect the above relevant info.

Are we missing any other imporant measurement/s ?

]]>