Eye on Oracle

Apr 9 2007   3:07PM GMT

Petabytes of data: The new normal?

Ken Cline Profile: Clinek

As enterprise Oracle DBAs know first-hand, data growth these days is exponential. Gartner calculates that the average site doubles its data under management every 6 to 18 months. Terabyte-sized data stores were a novelty not too long ago but you’ll soon be hearing more and more about petabtyes. That’s 1,000 terabytes. One million gigabytes. A lot of data.

A petabyte is a difficult concept to comprehend. Consider this:

  • A petabyte of data is the equivalent of 250 billion pages of text, enough to fill 20 million four-drawer filing cabinets. Or imagine a 2,000-mile-high tower of 1 billion diskettes.
  • For comparison, the U.S. Library of Congress, with 130 million items on about 530 miles of bookshelves — including 29 million books,  2.7 million recordings, 12 million photographs,
    4.8 million maps and 58 million manuscripts — can be stored on only 10 terabytes.
  • A single sequential scan through a megabyte takes less than a second, but at 10 MB/s it would take over a year to complete such a scan through a petabyte.
  • It would be faster to send a petabyte of data from San Francisco to Hong Kong by sailboat than over a megabit per second internet connection — indeed, it would take over 250 years!

For this post, I began compiling what I thought would be a short list of petabyte-sized data collections (not necessarily single databases) to show what was on the horizon for DBAs. To my amazement, the list went on and on, for example:

I soon gave up compiling an exhaustive list (email me at tdichiara@techtarget.com with your suggestions). Are petabyte data stores really the new normal? Not quite, but we’re getting suprisingly close

In all, the world has seen the amount of data grow from 5 exabytes in 2003 to 161 exabytes in 2006, according to IDC. The world’s storage systems can no longer store all of the data being created. This year,
the amount of information created and replicated (255 exabytes) will surpass, for the first time, the storage capacity available (246 exabytes).

Database and storage admins: your jobs are safe!  (Edit: Or maybe not. MySpace has NO storage admins for its petabytes.)

1  Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • Elisa Gabbert
    For those of us who aren't *quite* up to petabytes yet, terabyte hard drives for desktop PCs are now available!(!)
    0 pointsBadges:

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: