Posted by: Mark Fontecchio
Container Data Center, Google data center, Uninterruptible Power Supply
Over the past week, there has been a lot of discussion online regarding the Google data center energy summit held at the company’s Mountain View facility last week. In particular, there has been a flurry of activity pointing to a video tour inside a Google container-based data center. Let’s step back and take a look at what information was actually new, however.
Container-based data center
Google confirmed what everyone already knew — that the company has a container-based data center. Robert X. Cringely reported this back in November 2005, so it’s not exactly news. But it’s the first time Google actually confirmed the rumors and showed a sneak peek inside. You can take a look at the video below, taken by Data Center Knowledge:
[kml_flashembed movie="http://www.youtube.com/v/bs3Et540-_s" width="425" height="350" wmode="transparent" /]
Pretty cool stuff. But as James Hamilton, an engineer at Amazon, wrote in a recent blog post, it’s interesting to note that Google built this container-based data center, but then never returned to that design. In fact, Hamilton thinks that the data center design isn’t optimized for shipping containers:
The design chosen essentially built a well executed but otherwise conventional data center shell using standard power distribution systems and standard mechanical systems. No part of the building itself optimized for containers. Even though it was a two level design, rather than just stacking containers, a two floor shell was built. A 220 ton gantry crane further drove up costs but the crane was not fully exploited by packing the containers in tight and stacking them.
For a containerized model to work economically, the attributes of the container need to be exploited rather than merely installing them in a standard data center shell. Rather than building an entire facility with multiple floors, we would need to use a much cheaper shell if any at all. The ideal would be a design where just enough concrete is poured to mount four container mounting bolts so they can be tied down to avoid wind damage. I believe the combination of not building a full shell, the use of free air cooling, and the elimination of the central mechanical system would allow containerized designs to be very cost effective. What we learn from the Google experiment is that a the combination of a conventional data center shell and mechanical systems with containers works well (their efficiency data shows it to be very good) but isn’t notably better than similar design techniques used with non-containerized designs.
Adding to that, there has been talk for years (starting with Cringely) that Google took the containerized data center idea from Internet Archive, the company that runs the Wayback Machine so you can see what web pages looked like a decade ago. The story goes that Brewster Kahle, Internet Archive’s founder, presented the idea of a container-based data center in 2003 and that Larry Page heard the presentation. A couple months later, Google filed a patent application for a modular data center, which the patent office granted them in 2007.
Needless to say, Kahle doesn’t seem like a big Google fan now.When I talked to him last week, he said that “with Google, there’s a giant sucking sound. They tend to take from the commons and don’t tend to give back. The way the world generally works is you give acknowledgement to where things came from. It doesn’t look like they referenced where their ideas came from, but that is like Google.”
When I asked for a response from Google, a spokesperson sent me links to the research pages of Urz Holzle and Luis Andre Barroso, two Google engineers, along with this statement: “Google publishes many papers about our work on data centers and are hoping to have similar sessions to foster openness among the industry.”
Some might call Kahle’s comments sour grapes; others might make a comparison to Thomas Edison patenting the light bulb even though others claimed to have thought up the idea first. Either way, it’s clear that Google doesn’t have a monopoly on the idea, as all the major IT vendors — Sun Microsystems, IBM, HP and others — have their own version of a modular data center, and as far as I can tell, Google hasn’t sued them over it.
Server-based uninterruptible power supply
Let’s be clear. The fact that Google has 12-volt power supplies is nothing new. SearchDataCenter reported on it back in 2007, not long after Holzle and another Google engineer, Bill Weihl, wrote about it in a research report about high-efficiency power supplies for home computers and servers. What is new is that Google is building its own servers to include an uninterruptible power supply (UPS) right on the server, in the form of a 12-volt valve-regulated lead acid (VRLA) battery. Most servers rely on a centralized UPS system that requires a big room filled with big boxes that hold dozens and dozens (or hundreds and hundreds) of batteries. The UPS is there to keep power on to the servers while they transition to a secondary power supply, usually a generator, or primary power from the utility kicks back in.
By building the UPS right on the server board, Google can avoid the centralized UPS system and get much better efficiencies for its backup power supplies (over 99% compared to centralized systems, the best of which are about 96% efficient and the worst of which can dip below 80%). It’s very cool, and very different, and you can expect other leading-edge data centers to start following suit.
River-based data center
Perhaps the most underreported part of the show was Google showing the design for its river-cooled data center in Belgium. Some in the data center industry are saying that the next crisis to hit data centers is water — not necessarily the cost of it, which is still relatively cheap compared to power — but its availability. In Belgium, Google bypassed having to get water from the utility at all, instead sucking up water from a nearby canal, treating it, and using it to cool the data center. Though Hamilton thought the data center could be cost prohibitive because Google had to build its own water treatment plant, it is still a new idea, and one data centers out there can consider.
From talking to several people who attended the event, it seemed to be positive overall. One attendee I spoke to did say he was surprised there was any coverage of the event because all attendees had to sign a non-disclosure agreement before entering. Why make everyone sign an NDA and then have the press in to cover it on the record? I don’t know. Sometimes Google works in mysterious ways, I guess.
This same attendee added that he felt Google was most focused on proving that its super-low power usage effectiveness (PUE) numbers were legitimate. Why would they do this? Perhaps they can get some positive press on the matter, which they did. But they can also claim that their super-efficient server-based UPS is a big part of those low PUE numbers. And Google did say during the event that it would be willing to license the idea, which it is also trying to patent. So perhaps Google could profit from the event as well.