The amount of time it takes to test a web service really depends on it’s complexity. And the knowledge of the users using the service, if I am understanding your question correctly. Are you speaking of a web service from a Service Provider?
Understanding why you need this service and how well those users are being able to use the service would determine if it is a good investment. Most companies I know of will give you a “grace period” to try their service out to see if it meets your needs. Plus, I hope the company has spoken to the network administrator, management, and maybe a few key people that would be needing to use the service in the beginning.
But get feedback from those currently using the web service and see how they like/dislike the service and feedback from anyone else who should be consulted. And definitely check out that “grace period” and if something isn’t right, give the company a chance to make it better and asked for additional time to that “grace period” to evaluate the service again since your needs wasn’t met the first time.
I hope this helps. I tried to consider the who, the what, and the why.