Yottabytes: Storage and Disaster Recovery

Aug 16 2012   12:57PM GMT

Storage No Longer a Barrier to Ubiquitous Government Surveillance

Sharon Fisher Sharon Fisher Profile: Sharon Fisher

Big Brother image via Shutterstock

Big Brother image via Shutterstock

“Within the next few years an important threshold will be crossed: For the first time ever, it will become technologically and financially feasible for authoritarian governments to record nearly everything that is said or done within their borders – every phone conversation, electronic message, social media interaction, the movements of nearly every person and vehicle, and video from every street corner.”

This is due to a 1-million-times improvement in the ability to store information since 1985, according to John Villasenor, an electrical engineer at the University of California, Los Angeles, as well as a senior fellow at the Brookings Institution.

As an example, it would cost just $50 for an entire year of location data to 15-foot accuracy for 1 million people, updated every five minutes, 24 hours a day, Villasenor said in a seminar earlier this year. Similarly, storing the audio from telephone calls made by an average person in the course of a year would require about 3.3 gigabytes and cost just 17 cents to store, a price that is expected to fall to 2 cents by 2015, he said.

Scott Shane in the New York Times blog called attention to Villasenor’s work, which was published in a Brookings Institution paper date last December called Recording Everything: Digital Storage as an Enabler of Authoritarian Governments.

“In the 1960s, the National Security Agency used rail cars to store magnetic tapes containing audio recordings and other material that the agency had collected but had never managed to examine, said James Bamford, an author of three books on the agency,” reported Shane. “In those days, the agency used the I.B.M. 350 disk storage unit, bigger than a full-size refrigerator but with a capacity of 4.4 megabytes of data. Today, some flash drives that are small enough to put on a keychain hold a terabyte of data, about 227,000 times as much.”

Civil liberties organizations have increasingly been concerned about the amount of government surveillance that has been permitted, ranging from data that can be obtained from cellphones with no warrant required (though Rep. Markey (D-Mass.) has put forth a bill to limit that) to location data that the Sixth Circuit Court has ruled doesn’t require a warrant.

Moreover, Villasenor notes, individual people are providing a great deal of such data themselves, through the use of social media, mobile location apps, and so on.

But it is the rapidly declining cost of storage that makes such surveillance possible, Villasenor says. “Over the past three decades, storage costs have declined by a factor of 10 approximately every 4 years, reducing the per-gigabyte cost from approximately $85,000 (in 2011 dollars) in mid-1984 to about five cents today,” he writes. “In other words, storage costs have dropped by a factor of well over one million since 1984 [My note: an ironic benchmark to use]. Not surprisingly, that fundamentally changes the scale of what can be stored.”

These technological improvements put it within the reach of a country to store all the data it can obtain, Villasenor says. “For a country like Syria, which has a population of 15 million people over the age of 14, the current cost to purchase storage sufficient to hold one year’s worth of phone calls for the entire country would be about $2.5 million – a high number but certainly not beyond governmental reach,” he writes. “If historical cost trends continue, the annual cost in 2011 dollars to purchase enough storage for Syria’s government to record all calls made in that country will fall to about $250,000 by 2016 and to about $25,000 by 2020.”

While video data takes up much more space, limited video data –such as recording license plate numbers — is becoming increasingly prevalent in various states throughout the U.S., most recently in Minnesota. “Over the course of a full year, a system of 1,000 roadside license plate reading cameras each producing 1 megabit per second would generate image data that could be held in storage costing about $200,000,” Villasenor writes. “The resulting database of license plate numbers (as opposed to the images used to obtain the numbers) could be stored for a small fraction of this cost.”

The so-called “Peaceful Chongqing” universal video surveillance project in China — ostensibly for public safety — could cost as little as 25 cents per person per year by 2020, Villasenor writes.

Villasenor’s paper was focused on what he called authoritarian governments. Extrapolating costs to the U.S. was presumably left as an exercise for the reader.


1  Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.
  • Storage No Longer a Barrier to Ubiquitous Government Surveillance – Yottabytes: Storage and Disaster Recovery « Alucardv's Blog
    [...] Storage No Longer a Barrier to Ubiquitous Government Surveillance – Yottabytes: Storage and Di.... Wow… Like this:LikeBe the first to like this. [...]
    0 pointsBadges:

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: