IDC today released the results of its annual EMC-sponsored Digital Universe study, which confirms what storage professionals see first-hand every day: data keeps growing unchecked and resources to manage it aren’t growing nearly as fast.
IDC forecasts that 1.8 zettabytes of data will be created and replicated this year – enough to fill 200 billion two-hour high-definition movies, 57.5 billion 32GB Apple iPads or the amount of storage required for 215 million high-resolution MRI scans per person per day.
In other words, a really lot of data, and it’s doubling every two years according to IDC’s numbers. And metadata is growing twice as fast as the digital universe.
Looking farther out, IDC forecasts that by 2020 IT departments will have 10 times as many virtual and physical servers, 50 times as much information, and 75 times the number of files or containers that encapsulate information than they do today.
And there will be 1.5 times the number of IT professionals to manage it all.
As you would expect, EMC global marketing CTO Chuck Hollis hit on the “big data” theme in discussing the results, but also suggested the findings could serve as a wakeup call to change the way people manage data.
“I would use this as evidence to go to senior management and say ‘We need a new game plan here,’” Hollis said. “Simply expanding five percent year-over-year on storage costs, taking the machines they have and tuning them up – that’s not going to keep up. I meet a lot of storage people who think they’re like the people with their fingers in the dikes, the water keeps coming and they’re running out of fingers and toes. Maybe it’s time to think about this problem differently.”
Hollis said “a lot of people are looking at this as an opportunity instead of a problem,” and those people are what EMC refers to as the “big data crowd.” They consist largely of media and entertainment companies and researchers who use data to make money for their employers.
“There are actually two kinds of IT organizations we see often in a big company,” Hollis said. “One is the traditional IT guys who deal with shared services, e-mail, Oracle and things like that. The big data crowd is usually a separate IT structure, usually researchers or business guys who have an idea and they handcraft the environment in such as way that makes the money or provides the value they want. The technology is different, the organization is different, and the thinking is different. At what point does this big data IT start looking like mainstream IT? Certainly not this year, but if this data growth keeps going, in three or four years it will be a lot more complex.”
IDC group vice president for storage Dave Reinsel said data growth is fueled partly by the low cost of disk. But he agrees with Hollis that organizations need to take a different look at how they deal with the data.
“We’ve made it dirt cheap to store,” he said. “If costs were going up like gasoline, people might change their behavior. But storage cost per gig is going down every year, so people have more. But data centers aren’t cheap to run. You have to justify building another data center. We’re getting to the point where we need to enable companies to extract the value out of that information.”
So far, Reinsel said, cloud storage isn’t playing much of a role in storing that information. Today, all cloud computing accounts for less than 2% of IT spending.
“Only 20% of information will be touching the public cloud by 2015,” Reinsel said. “People aren’t just jumping to public clouds. Hybrid clouds are out there and social networks are driving growth to public clouds, but there are still security concerns.”