According to IDC’s 2010 Digital Universe report, digital data grew 62% last year as 800,000 PB were added. IDC says 1.2 million PB (1.2 zettabytes) will be added this year, and that will increase to 35 ZB in 10 years.
While those numbers may look staggering on a page, they probably don’t shock anybody charged with managing data storage. The real shocking – and frightening – number is that IDC says the amount of IT staff to manage all this data will only grow by a factor of 1.4 by 2020. If IDC is correct, than the dreaded “do more with less” mantra will become a long-term way of life.
So how will this all change the way we manage data? Chuck Hollis, global marketing CTO of EMC – which sponsored the IDC study – says the data growth will push a lot more of it to the cloud this year. Hollis says the IT staffs at large enterprises that he talks to are ready to set up private clouds to manage data.
“For tech guys, this is the year of putting your cloud strategy together,” Hollis says. “We’re way beyond the ‘What is the cloud?’ discussion, and it’s a very mature discussion with the IT guys I talk to.
“The larger enterprises say, ‘We’re big, we can do this ourselves. We can build a private cloud behind the firewall and get comfortable with it.’ They’re saying, ‘We pay the same price for this stuff – the processors, server, storage – there’s no reason I can’t do what Amazon does.’”
Hollis says as long as organizations feel they can control their data in the cloud, they’re willing to move it there.
“The cloud works when enterprise guys can be in control,” he said. “Ask them to give up control, and it’s not that attractive a proposition for them. You can’t outsource responsibility and accountability. In financial services, a trillion dollars a day floats around the global economy over the cloud. Most days we’re OK with that. Clouds, schmouds, it doesn’t matter as long as enterprise guys feel they’re in control.”
Other emerging methods of managing growth aren’t quite as mature, Hollis says. That includes data deduplication for primary data. While EMC is now the leader in backup dedupe, Hollis says the success of primary deduplication “has a lot to do with processors being fast enough to do it without impacting performance. If you have a SAP application with 10,000 demanding users, maybe it [deduplication]’s a false savings. The concern is, at what cost? The technology gets better year over year, but some are of the opinion this is just a temporary fix, you’re just buying yourself some time. A lot of information is not compressible, like JPEGs. You can’t compress something that’s already compressed.”
Flash solid state storage is another area where EMC has been out front, but it’s another technology where the greatest benefits are still down the road. “If you take what processors have done in the last 10 years as far as density, price and performance, then start with flash in 2010 and forecast it out in 10 years, it could actually get cheaper than disk,” Hollis said. “That would be an interesting world.”