Uncharted Waters

Mar 8 2017   3:33PM GMT

Technology Cost Trade-Offs

Justin Rohrman Justin Rohrman Profile: Justin Rohrman

Cost analysis
Cost management
Solutions architecture

I am working through a tough technical cost trade off decision with my current client.

Our nightly test automation has run from my local machine for the past year or so. There are some obvious challenges there — I can’t use my computer for other work in the evening, tests are subject to the whims of my local internet connection, and the occasional Windows update that makes my computer a zombie for an hour. We have three decent options of how to proceed.

Programmers sometimes like to pretend that there is a clear cut winner in our technical solutions. In my experience, everyone has a different cost and we pick the one that seems the least painful.

Option 1:

This is the easiest choice but not the cheapest. We have one license for the tool I am using to build a UI test automation suite for this client. Right now, that license is being used on my machine as a simultaneous development environment and also as a test runner. Option 1 would have us renew a support license to get the latest version of our automation tool to run on this other virtual machine. That would enable me to run the tests on another computer every night and to have consistent versions of the automation tool in all environments. This would cost a couple thousand dollars for a year. Once we have the installation on the machine, in theory we could let the support license lapse and continue using the tool there.

trade off

Option 2:

The automation tool I am using has a full client that is used for test development and as a test runner. There is also a standalone test runner that can be installed without the development environment. This standalone test runner is free and can be installed in as many places as we want. I could configure my local instance of the test automation tool to start test runs on these local machines. The trade off here is that I would have to leave my computer on the VPN every night so that it could see this remote virtual machine and start test runs there. If I go on vacation, or travel to a conference or to visit a client, this gets tricky. It is also just tedious to remember to check that I am on the VPN every night before I stop working.

Option 3:

Option three uses the same test runner. The trade off here comes in the form of technical infrastructure. For this option, I would configure our Continuous Integration system to kick off test a new test run after every build. This introduces a logistical problem. Each test run currently takes about 2.5 hours start to finish. To make this feasible, we would have to run tests in parallel shrinking the run time as much as possible. For this to work our test run time would have to shrink from 2.5 hours to sub 15 minutes. This could be done by installing the standalone test runner on several different virtual machines or by using a platform service like SauceLabs.

For most people working in modern software development, the choice seems like a no-brainer. Option 3 would be the choice, you get fast integration and fast results. My context is different though. We have a legacy product and getting new technology paradigms into that flow can be more difficult than the sting of a few thousand dollars.

Every technical choice has a cost trade off. That might be in terms of time, or money, or politicking to get what you think is the right solution, or all three. The answers are rarely clear once I have identified the problem and the choices, but being aware that there are choices and that each is a trade off on cost makes the process easier.

I’m not sure which of these choices we will go with yet, I’m still discussing this with the development lead. The solution we do choose though will be the one that seems to sting least.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: