Replicate Amazon EBS instances to S3

366125 pts.
Tags:
Amazon EC2
Cloud Computing
EBS
My company has a website where users upload files (and many of them are pretty large). We have a few EC2 instances so we're looking to load balance them and they're currently stored on an EBS volume (because they're easy and fast to access). Is there a way to replicate these files so we can upload them on S3? If so, what's the process? We already know snapshots won't work because they would replicate the entire volume. Much appreciated.

Answer Wiki

Thanks. We'll let you know when a new response is added.

When you need to replicate Amazon EBS volumes to S3, start by writing a shell script that will sync your local s3cmd files with the S3 bucket each time the file is updated or a new file is added or deleted. Just use a short piece of code such as this:
s3cmd sync ./ s3://your-bucket-name/.

If you’re running a Linnux machine, the process is also simple. Just use the FUSE file-system and rsync commands to sync and replicate your volumes to S3. This command allows you to use different machines for your file editing, deletion and creation activities. On Windows machines, try Distributed File System (DFS), which is a part of Windows Server and will sync your files automatically.

Discuss This Question:  

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when members answer or reply to this question.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network:

To follow this tag...

There was an error processing your information. Please try again later.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Thanks! We'll email you when relevant content is added and updated.

Following