PUE

Data center earns nearly $1 million in utility rebates from PG&E

Hosting and wholesale data center provider Fortune Data Centers walked away with a $900,000 check from Northern California energy utility Pacific Gas & Electric for making energy efficiency investments in its new San Jose data center.

Fortune opened the new data center for business in April, 2009, reclaiming a former manufacturing site. After phase I of construction, the 78,000 square foot facility has 43,000 square feet of IT space, and the company is claiming a PUE of 1.37, which if correct would mean that the facility’s data center mechanical infrastructure is highly efficient compared to the industry average of 2.0. See Mark Fontecchio’s article for more on evaluating public data center PUE claims.

The utility rebate came from PG&E’s High Tech-IT Facility program, which offers incentives for California data centers to reduce energy demand.

According to Fortune Data Centers CEO John Sheputis, the energy savings came from optimizing the cooling system and purchasing the most efficient uninterruptible power supply systems possible. Sheputis said the PG&E payments offset about half the cost of the premium for implementing high efficiency infrastructure.

PG&E engineers were very involved in the design and commissioning of the project. “PG&E has a commissioning agent to make sure you’ve installed the energy efficient devices and to make sure that it’s working,” Sheputis said. “They have to see the equipment operating. We get payments as the measures are brought online. They’re not allowed to just go around handing out checks.”

While $900,000 is a lot of cash, it’s not the largest data center utility rebate so far. NetApp scored $1.4 million from PG&E for its data center efficiency measures. Sheputis golfs with NetApp data center execs, and joked that “They’re dissing me for not getting as big a check as they did.”

Google on data center efficiency: Stop making excuses

NEW YORK — Bill Weihl, Google’s green energy czar, told a group of data center operators here that some of them need to stop making excuses for not improving their facilities’ energy efficiency.

After years of secrecy around how its data centers operate, Google has now drawn the curtain to show how efficient its data center facilities are. But during a panel discussion at The Uptime Institute’s conference in New York today, some questioned whether all data centers should be cut in the same mold.

In particular, the question was whether the data center power usage effectiveness (PUE) of some businesses — financial institutions, for example — should be compared to those of search engines such as Google.

“Should a bank have the same PUE as a search engine?” Ken Brill, Uptime founder and executive director. “The answer is no.”

The reasoning behind it is that bank and financial applications require a higher level of uptime than search queries, and thus need more redundancy, which leads to lower efficiency. But Weihl questioned the logic.

“We actually have some Sarbanes Oxley requirements,” he said. “We’re not just a search engine company. We also run very reliable data centers that I think any data center operator here would be proud to run.”

Weihl later added that the discussion sounded like “people making excuses for why the EPA or DOE should not push hard for a standard because, hey, we’re different.”

“To me,  not to be too combative, but that sounds like an excuse for not doing better.”

Currently the federal government is working on developing an Energy Star rating for data centers. Michael Zatz, the manager of the Energy Star commercial buildings program, sees the potential for different categories of data centers, but would prefer that those categories be defined by what kind of work the data centers perform, and not necessarily by what industry they’re in or how they identify themselves.

Data center location and the cost of power

What costs more in your data center, server hardware or power? Well, it depends on your zip code.

James Hamilton, an architect at Microsoft, recently wrote about the cost of power compared to servers in large-scale data centers. He concluded that servers cost more than power in cooling — in Hamilton’s scenario, the server hardware costs three times more than power. Take a look:

Hamilton made available the spreadsheet on overall data center costs that he created to make the calculations.

Two assumptions caught my eye: one on the cost of power, which Hamilton put at $0.07 per kilowatt-hour (kwh); the other on the power usage effectiveness (PUE) being 1.7.

According to the Uptime Institute, the average PUE is 2.5. This average was calculated using data from Uptime’s data center user members, many of which are large companies running large data centers. So I plugged that PUE number in instead. A worse (higher) PUE number leads to higher power costs, because there is more power being wasted.

On the cost of power, I wasn’t sure whether $0.07/kwh was cheap or not. It seemed cheap to me, but I’m not that familiar with power rates. I went to a Web site run by the Energy Information Administration, a division of the U.S. Department of Energy. The EIA publishes a regular newsletter called Electric Power Monthly that has these rates. The data I looked at was in a spreadsheet called Average Retail Price of Electricity to Ultimate Customers by End-Use Sector, by State.

As it turns out, location matters a lot. The cheapest location for power is West Virginia, where the industrial rate is 4.34 cents per kwh. If you build a data center there with a PUE of 2.5 and use Hamilton’s spreadsheet,  server hardware costs three times more than power. If you build it in Hawaii, where power costs 29.51 cents per kwh, then power costs more than twice as much as server hardware.

The national industrial rate in August was 7.61 cents per kwh. Using that assumption to build a data center with a PUE of 2.5, server hardware costs about 80% more than power.

Green Grid publishes data center utilization tool

The Green Grid has another tool for data centers to measure their companies’ efficiency, saying it could be a valuable way to communicate with upper management on the state of the data center.

The IT efficiency-focused group has published a new paper on the “productivity indicator.” Christian Belady, the principal power and cooling architect at Microsoft who was the driving force behind the PUE/DCIE metric, edited the paper and said it should be used as “a communication tool” between various members of a company – IT workers, data center facility folks and company executives.

“What this does is give you a quick visual of how you’re doing, especially if you’re communicating up to executives,” he said.

The paper suggests building a radial graph with five “spines,” with each spine representing a metric:

• DCIE
• Server utilization: The activity of the server processors relative to its maximum ability in the highest frequency state.
• Data center utilization: The amount of power drawn by the IT equipment relative to the actual capacity of the data center.
• Network utilization: Percentage of bandwidth used compared to bandwidth capacity.
• Storage utilization: The percentage of storage used compared to the overall storage capacity.

The paper doesn’t say how to come up with each of these numbers, but there are tools and software out there to get the data points for each of them (see the definition of storage utilization, for example). And if for some reason you are still trying to figure out how to measure network utilization, for example, you can still plot a graph using the productivity indicator, but with fewer spines. Here is a sample picture of a productivity indicator radial graph:

Belady and John Tuccillo, a Green Grid member from APC, said businesses can add target lines if they want as well. They could have targets for six months out, a year, and 18 months out. Companies can use it for whatever data center metrics they’re actually using. So if it’s not a pentagon, it might be a square or a triangle with four or three data points, respectively. In which case it might look like this:

Companies can also break down one of those categories, such as data center utilization, into a more detailed radial graph all its own, such as this:

They emphasized that this isn’t something that companies should use to compare to other companies. Instead, it’s a way for businesses to realize their existing energy situations and set target goals for themselves down the road.

“Different companies have different risk thresholds. A business may say, ‘You know what? My storage utilization, because of my business plan, should only be at 50%,’” Tucillo said. “One of the strengths of this tool is that it allows for the end user to weigh the spines to what their business practice is.”

Key points to consider in the evolution of the PUE data center metric

Yesterday I blogged about the Uptime Institute’s criticisms of PUE, specifically that the Green Grid metrics whitepaper does not explicitly discuss the need to gather PUE data over time. Microsoft’s Christian Belady (one of the key developers of the PUE metric) responded with the following:

“Ken [Brill of The Uptime Institute] has some valid points, clearly there needs to be more clarity and refinement in the definition to make it a rock solid benchmark. PUE is a “living metric” that the industry and in particular the Green Grid is working. But with all of the issues that are in the process of being resolved, here are three basic facts:
1) All metrics can and will be gamed regardless of the crispness of their definition. Show me a metric and I can come up with a way to game it.
2) Companies that are measuring PUE are improving their PUE over time. So they are improving their efficiency
3) Companies that are not measuring, are likely not improving. So these companies will be at a competitive disadvantage.

Microsoft is a company that is measuring (since 2004) and improving our PUE benchmarking against ourselves. There is no reason any other company cannot do this. Comparison with other companies is useful but less important to us as long as we demonstrate continuous improvement in our PUE. We hope that the issues people have with PUE for external benchmarking will be cleaned up in time but we do not plan on waiting until then for continuous improvement in our own operations.”

Here are a few key points to consider in the ongoing evolution of PUE:

Gaming PUE is going to happen
A lot of data center providers have included PUE ratios in press releases lately, many of them incredibly low. Rich Miller at Data Center Knowledge says he’s seen it before. “That’s pretty much what happened with the Uptime Tier System, which set forth a four-tier rating system for data center reliability. Data centers began describing themselves as equivalent to ‘tier three-plus’ or even ‘tier five.’”

PUE will need to evolve into a dynamic quality control metric
Dave Ohara at GreenM3 has a great explanation of how data center pros should use PUE in a dynamic way. “What helped me to think of PUE as a dynamic number is to think of it as quality control metric. The quality of the electrical and mechanical systems and their operations over time are inputs into PUE. As load changes and servers will be turned off the variability of the power and cooling systems influence your PUE. So, PUE can now have a statistical range of operation given the conditions. This sounds familiar. It’s statistical process control.”

Standards and training needed on how and when to measure PUE
Data center managers getting started with a PUE measurement program need some guidance — where, when and how do you take the most meaningful measurements? Microsoft’s Mike Manos and Belady have put together an excellent PUE Strategy post on their blog, The Power of Software. This checklist takes PUE newbies from measuring by walking around with a clipboard to data center chargeback. The Uptime Institute’s Pitt Turner has a great webcast on how to measure PUE on UPS and PDU equipment. The next step will be to get everybody doing this in the same way — which is where ASHRAE TC 9.9 comes in. The organization supports PUE and announced plans to develop a publication that would standardize PUE measurement methodology in November 2007, but no word so far on the progress of that project.