Yottabytes: Storage and Disaster Recovery

Dec 11 2016   11:05PM GMT

Amazon Snowmobile Issues Need Solving

Sharon Fisher Sharon Fisher Profile: Sharon Fisher


Amazon has upped the ante on that old question of the bandwidth of a station wagon full of backup tapes hurtling down the highway: For one thing, instead of a station wagon, it’s a semi.

The data storage device hauled by the semi is a 45-foot shipping container called Snowmobile, and it could let people send up to 100 petabytes of data to Amazon Web Services – much, much faster than it would take to upload it. A petabyte is 1 million gigabytes. The data can be stored in either Amazon’s regular S3 service, or its “Glacier” cold storage service, which is less expensive.

The notion of doing the initial upload of data to a cloud service by shipping a physical hard drive isn’t new; the major cloud vendors have all supported this for a while. This is, in fact, considered an upgrade to Amazon’s “Snowball” service, which uses a mere 80TB suitcase.

But a semi? That’s new.

Amazon plans to drive Snowmobiles to its customers’ offices, extract their data, then cruise to an Amazon facility where the information can be transferred to the cloud-computing network in far less time than it would for so much data to travel over the web,” write Jay Greene and Laura Stevens in the Wall Street Journal. “Ten Snowmobiles would reduce the time it takes to move an exabyte from on-premises storage to Amazon’s cloud to a little less than six months, from about 26 years using a high-speed internet connection, by the company’s calculations.”

Amazon announced the new service at its annual customer conference. It is actually already available and costs half a cent per gigabyte per month of use, or about $500,000 a month to use its full capacity, Greene and Stevens write.

Physically, Snowmobile attaches to your network and appears as a local, NFS-mounted volume, writes Jeff Barr, AWS chief evangelist, in a blog post (which includes all sorts of awesome Legos showing how it works). “Snowmobile is a ruggedized, tamper-resistant shipping container 45 feet long, 9.6 feet high, and 8 feet wide,” he adds. “It is water-resistant, climate-controlled, and can be parked in a covered or uncovered area adjacent to your existing data center. Each Snowmobile consumes about 350 kW of AC power; if you don’t have sufficient capacity on site we can arrange for a generator.”

So how do you attach it to your network? “Each Snowmobile includes a [fiber] network cable connected to a high-speed switch capable of supporting 1 Tb/second of data transfer spread across multiple 40 Gb/second connections,” Barr writes, adding that you can use your company’s existing backup or archiving tools. “Assuming that your existing network can transfer data at that rate, you can fill a Snowmobile in about 10 days.”

Of course, the question of security comes up. You don’t want your company’s entire data record to be hijacked by some guy with a CDL. “Snowmobile uses multiple layers of security designed to protect your data including dedicated security personnel, GPS tracking, alarm monitoring, 24/7 video surveillance, and an optional escort security vehicle while in transit,” writes Amazon. “All data is encrypted with 256-bit encryption keys managed through the AWS Key Management Service (KMS) and designed to ensure both security and full chain-of-custody of your data.”

Exactly when and where this encryption is done isn’t said; can you load encrypted data onto Snowmobile, so that even Amazon doesn’t know what it is, or is it Amazon that encrypts the data, meaning the company has access to the data at some point?

There’s also the matter of securing the trucks themselves while in transit, writes Daniel Stoller in Bloomberg. “Researchers recently said that trucks, much like the ones Amazon would use, are prone to the same kind of hacking attacks that have disabled some connected cars,” he writes. “The researchers showed that there is a real possibility of ‘safety critical attacks that include the ability to accelerate a truck in motion, disable the driver’s ability to accelerate and disable the vehicle’s engine brake.’”

That also raises the question of what happens to the Snowmobiles full of data after it’s uploaded. How do you wipe a Snowmobile, how long does it take, and what assurance do you have that Amazon actually does this? Assuming it does; Amazon doesn’t talk about it.

The question also arises, why bother transferring the data at all? Why not just truck it to Amazon, plug it in, and leave it? Though having all the data on a physical device would sort of defeat the purpose of having it in the cloud.

Getting the data back out again may be more of an issue (a problem Amazon has also had with Glacier. “The initial launch is aimed at data import (on-premises to AWS),” Barr writes, though he adds, “We do know that some of our customers are interested in data export, with a particular focus on disaster recovery (DR) use cases.”

It will be interesting to see whether other cloud vendors, such as Google and Microsoft, follow suit, or if Amazon will have the long-haul data-trucking field to itself.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: