Uncharted Waters

Jun 25 2014   2:33PM GMT

The Continuous Delivery Backlash

Matt Heusser Matt Heusser Profile: Matt Heusser

Continuous delivery

My Phone Wants to upgradeIt’s been two weeks since I have upgraded my computer. It has eighteen applications asking to upgrade — plus the operating system.

And that’s just my phone.

What’s worse is that I need these updates, or at least some of them. My current version of iTunes gets “stuck” and shows the cover art of Jackson Browne on every song occasionally, while twitpic crashes every time I launch it.

Meanwhile, on my WindowsXP machine, Adobe Acrobat is struck in an update loop. Every time you update Acrobat, it asks to update again.

Then there’s Turbotax, which asked to update, reloading a 50MB file, literally every single time I launched it this February, March, and April.

How did we get here?

Too many companies took delivery ideas from the web, like continuous delivery, (or “CD”) and tried to apply them to the desktop, without considering the context.

This may be leading to a backlash against continuous delivery, at least for applications. Before we get to that, though, let’s have a little history.

It all changed in 1998

Prior to 1998, most people didn’t have fast web connections, so a defect in a software product meant expensive and reputation-damaging recall and reissue of physical disks or CDs. That created an economic burden for test and release – testing took time, and we liked it!

Then Sierra released King’s Quest: Mark of Eternity in 1998. The game had a screen-freeze bug that happened within 15 minutes of opening gameplay … along with the web-based update that took five minutes to fix it.

Suddenly getting everything right up front didn’t matter as much anymore.

And that’s the argument. Customers want their applications as soon as possible, we’ll monitor use in production and rollback quickly if we have a problem. This solutions can work just fine.

On the web.

I tunes image is  Jackson Browne, but the song isn't!On physical devices, things are more complex. If we enable auto-update, we face the problem of updating to a bad version just before we lose wireless, or, on Windows, the occasional reboot without explanation. (Not to mention the java apps that sometimes really do want a specific version of Java.)

Manual updates, on the other hand, generally mean an annoying pop-up that we tell ‘later’ every day until finally giving in.

Meanwhile, the quick release cycles mean mistakes like the one at left, where “Hail Holy Queen” is playing but the image is the cover art from a recently-played classic rock song.

It seems to me there should be some sweet-spot between releasing too often and not often enough.

Economic Theory To The Rescue

Most companies don’t release every change to a single line of code — the transaction cost, or underly costs to test/deploy, would just be too high. The argument is the cheapest place to deploy is where transaction cost meets holding cost – the cost to hold onto the software and not release it – things like mental overhead, task switching, tracking a large number of changes, and so on. A lot of the work in the CD space is to decrease transaction cost, to make the two intersect earlier.

What we’ve forgotten about here is the cost to the customer – the mental cost of upgrading software, the perceived risk of new builds. Consider, for example,  the way I felt about launching turbotax on April 10th, a day after sending my paperwork to the government, when the software insisted on yet another update. Would that change impact my tax return? Did the changes matter? I didn’t know.

For applications that live on a computer, that cost to the customer is real, and something to consider. Companies that push their users to update too often may face a backlash, and those that position themselves in the sweet spot may profit.

In the mean time, I wonder if “doesn’t force you to update every 10 minutes” might actually become a selling feature for software. Time, as always, will tell.

What do you think?

5  Comments on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • parkie1982

    Hi Matt,

    Certainly a thought provoking piece but I don't know if any of the examples are actually Continuous Delivery. We just don't know if the examples embrace the whole Continuous Delivery concept through out their value stream, These could all be really bad upgrade mechanisms and shouldn’t be confused with Continuous Delivery as a set of software development patterns. I have heard the term update fatigue before and I think this is you what you are really driving at.

    Poor update implementation do cause problems for end users and they will vote with their feet to products that don't cause them the same issues. I think Google Chrome is a perfect example of this, silently updating every 6 weeks without causing the user problems. When Firefox switched to this high frequency release cycle the user was require restart and they had made issues with plug-in capability. This was a likely reason for a significant loss in market share for Firefox.

    Business evolution should drive out these bad implementations by users voting with their feet and another important aspect for teams to consider.



    20 pointsBadges:
  • Matt Heusser
    You make some solid points, Alan. Strictly speaking, I don't think it is the Humble/Farley definition of CD, nor the IMVU/Etsy/Twitter version either - it's more like Continuous Delivery Envy which is hurting the applications.

    Your Google Chrome update is interesting, but six weeks sounds much more like a traditional update cycle to me ...
    4,935 pointsBadges:
  • MGEakin
    A customers mental cost is part of the holding cost you mention. If an app doesn't launch that's a rather high holding cost for the company as customers might stop downloading and using the app, thus killing their revenue stream; if an app is just annoying (wrong album cover for a song) but still works, the holding cost might be minimal as you are still going to buy songs and the $$ will still flow. Good companies (defined as those staying in business) track these numbers and should know pretty quickly (based on daily bottom line) what their holding cost is. 

    In fact, I would argue they have a better handle on their holding cost than their transaction cost as most companies are paying more attention to revenue than expenses. If a company sees a 10% drop in revenue the day a change is deployed, I can almost guarantee you will see a change being deployed the following day to fix the previous change. Likewise, a 10% uptick will result in fewer releases going out as no one is going to want to touch a hot product.

    CD feeds this instant gratification loop by being able to deliver new features (potentially) every 2 weeks.
    50 pointsBadges:
  • parkie1982
    I'm glad you identified the Humble/Farley definition of CD, Matt. What other definitions have you seen?

    Interesting concept of Continuous Delivery envy. I've always thought the automated updates being around longer than the CD term, but that just my own opinion/observation. The proof in the pudding would to be look at the frequency of the updates and the content. CD practitioners would be delivering new features over bug fixes.

    The Chrome examples was for the update mechanism over being a CD process (I haven't looked at the complete process to make a judgement). I suspect they could deliver more frequently but the benefits aren't worth he effort.

    I find it interesting people making comparisons between release frequency and if you are doing CD. The difference between Continuous Delivery and Continuous Deployment is the business is in control of the release in the former. They choose when you release, so they could decide to wait 6 weeks or a few hours but the software must always but in a releasable state.
    20 pointsBadges:
  • Matt Heusser
    @Parkie1982 - It seems that everyone has their own definition of continuous delivery! I go by humble/farley because that at least gives us an agree-on, canonical place to start. Noah Sussman, formerly of etsy, has some interesting ideas, though I don't know if he has released his own term for CD. I think the twitter/etsy/imvu conversation is more around 'devops.' Here's a line from NewRelic - http://refcardz.dzone.com/refcardz/continuous-delivery-patterns

    Re: Continuous Delivery Envy - yes, updates have been around a long time. What I see with Quickbooks and Adobe is an attempt to do something like CD, possibly even continuous deployment on the desktop, where there are costs to the consumer for such releases.
    4,935 pointsBadges:

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: