Beta Testing

Beta Testing - Not for Everybody

It’s like the stock market - it can be risky - it can be frustrating and downright annoying - and it can be rewarding.  I’ve always liked participating in beta testing of products which I find of value, or as has happend to me before, in a product that I had to use - but (let’s just say to be nice), needed a lot of work — and by beta testing (with our live data of course) we got to evaluate the latest and greatest of the software.  Most of the time it was a win-win situation, but not always.

I wrote a few weeks ago about how my chiropractor didn’t realize what it meant to be a beta test site, and consequently just how much she “suffered” with the arrangement.

For me, as a custom software developer, I think I like participating in anothers beta test because I invevitably learn from the experience.  There are 2 major learning opportunites that I consider to provide the value for me by participating — they are:

1. The opportunity to really learn a product and understand it
2. The opportunity to find out just how the software provider works

Products and companies have made it and lost it based upon my experience with beta testing their product.  If a company isn’t easy to work with when you are providing beta testing, they probably aren’t a company you’ll want to work with long term.  Likewise with the product.

Beta Testing - A True Story

On a visit to my chiropractor today I heard a story that I found simply amazing to believe, yet I have to believe it as I was witness to some of the “events” a few years ago - and I remember them well!

It seems that some 6-8 years back my chiropractor (always on the lookout for a “good” deal) was in need of software to handle her business - appointment scheduling, billing - the “usual” stuff for a small practice.  I remember this period as I had been shaking my head on more than one occasion as her employee struggled with the system that was being used.  I also remember her excitement that she had ordered a new system that was really going to be much better than what she was using - and she couldn’t wait until it got installed so that she could address some of the “issues” she’d had with her existing system.

Well, making a long story short, let’s just say that it was a LONG time before improvements were noted by this loyal patient of hers.  She struggled seemingly at every turn - reports didn’t work properly, things didn’t balance - just an apparently endless array of “issues” with this new system that was supposed to solve so many of her problems.  “User friendly” was not how she described her new system at the time.

Interestingly enough, today after I commented about “…good thing your computer system is user friendly…”  (her new office person was struggling with some entry is what prompted my remark) — she asked me if I remembered all the problems she used to have with the system?  (I acknowledged that I indeed did!).  At that point in time she told me “…I really didn’t understand what being a beta tester really meant!  They gave me deep discounts!…”

Indeed they should have given her the software for free from my perspective — her beta testing was more like an alpha site - only she was running live!  I chose not to ask her if she would buy into a beta testing again.

However, what I took away from this today was that if deeply discounted pricing is provided to a customer in return for being a beta test site — the developer had better convey VERY clearly to the customer just what it means to be a beta test site for them.  It seems that in this case the expectations were not clear - the result appears to have been that the 5 years of high employee turnover, retraining, and resulting frustrations and inefficiencies probably cost a great deal more than paying for a proven product would have been.

Of course, I could be wrong!