By: Bernieh

Bernieh — Fri, 17 Apr 2009 16:50:52 +0000

I agree with the primary point you were trying to make here—customers need to do their own comparisons and not rely simply on the claims of a single vendor’s tests. I do competitive testing at Citrix with XenServer, sometimes vs. a competitor and sometimes not. When I post results in a Citrix white paper http://citrix.com/English/ps2/products/documents_onecat.asp?contentid=683148&cid=White+Papers, I always make the point that the customer shouldn’t just take our word for it. They should, nay must, do their own testing.
At the same time I don’t agree with your assertion, “Comparisons by vendors themselves are always biased”. Are they occasionally biased? Yes, sometimes they are. That happens especially when we seek to validate a specific feature or capability which we believe to be of competitive significance. And when we do, we’ll point that out. In other cases, it’s just not possible to do an exact apples:apples comparison. And finally, there are those vendors who’s EULAs limit or prevent altogether the ability to create a level playing field, resulting in the perception of bias.

I disagree; however, with your assertion that bias “always” exists or is even intentional. We go to great lengths to avoid bias in our testing at Citrix. We know that customers would see right through it anyway, so why bother? And, if the appearance of bias exists and is pointed out to us by a customer, we’ll do our best to explain our test rationale and if necessary, admit to any mistakes made along the way.

We all learn from comparison testing. As a vendor, we learn how to make better products. Customers, ideally, learn what’s right for them.

Bernie Hannon
Director, Certification & Testing
Citrix Systems Xen Products Division

Comments on: Virtualization vendor comparisons

By: Bernieh