It’s been a week of mea culpas. Following Google’s admission that it didn’t handle the launch of Google Buzz very well, on Thursday WordPress, a leading provider of blogging platforms to individuals and businesses alike, experienced an outage of approximately 110 minutes. The WordPress problems affected 10.2 million blogs, depriving those bloggers of about 5.5 million page views.
According to the official WordPress blog (which I assume was also unavailable during the downtime), the problems were likely the result of an “unscheduled change to a core router by one of our data center providers [that] messed up our network in a way we haven’t experienced before, and it broke the site.” Worse, the outage broke all of the company’s mechanisms for failover in San Antonio and Chicago.
I’ll be interested to hear about WordPress’ stopgap solutions in the case of blog failure. In the meantime, I’m left to ponder — and point out to our readers — that having a disaster recovery and business continuity plan isn’t necessary only for such catastrophes as hurricanes — or a massive data breach that leaves you scrambling to explain to irate customers what went wrong (I gather that WordPress did an admirable job updating users via Twitter during the outage). But how do you prove this to the business so you can get the resources you need? Consider piggybacking disaster recovery efforts on other projects or mapping availability risk.
The WordPress problems also underscore the importance of properly vetting your providers, whose data center outage could negatively affect your business and its customers. For more information about assessing potential sourcing partners, check out our FAQ on getting started with IT outsourcing.