Clouds in the Open: The Operations Evolution of Open Source & Public Clouds

Jun 24 2013   9:09PM GMT

CCC13 Live Blog: OpenStack Swift with CloudStack

Aaron Delp Aaron Delp Profile: Aaron Delp

Disclaimer – This is a live blog from the CloudStack Collab Conference. Might have a bunch of errors in formatting, etc.  I’m just typing as fast as I can.  Also, I work for Citrix and I focus on CloudPlatform, the commercial version of CloudStack.  Just want to be up front with everyone.

Title: OpenStack Swift Introduction – Technical Overview & Use with CloudStack by John Dickinson from SwiftStack

  • What is Swift – It is an Object Storage System – this isn’t block, not a filesystem, designed to be:
  • Highly concurrent, open source, running on cheap commodity hardware and is very developer friendly
  • Swift has a large production user base in both service providers as well as private clouds
  • Swift uses a hash ring for data placement, this allows you to add and remove capacity on the fly (no downtime)
  • John is now going into the hashing algorithm used for placement within the ring – this hash is used to both place and recall data from the “ring”
  • The idea behind this is to make all of the data as unique as possible, when data comes in and is placed throughout the cluster. Each replica for instance is placed on a different server, rack, etc.  The goal here is to handle any failure and still have a copy of the data
  • The hierarchy of this is regions -> zones -> replicas
  • Swift Failure Handling -> How does this work?
  • What if a disk fails? – Swift will auto replace this data without operator interaction, data is rebuilt (think RAID arrays) and the operator can  remove/replace the drive later
  • What if a server fails? -> This is more availability (can’t get to data) vs. durability (data is GONE). In this case it will route around the requests until the server is brought online or taken out of the cluster by the operator manually
  • What if bit rot? (file system corruption) -> Swift scrubs data in the backgrund to make sure the data is consistent and will rewrite data bits if needed
  • Swift’s Design – Proxy Server – Deals with client integration (interface to outside world)
  • Swift Account, Container, Object servers are in the background
  • Objects are stored in containers, a container has a list of objects and associated metadata
  • Accounts are how “who has access to what” happens
  • Last piece – consistency processes – runs in the background and performs care and feeding of the environment
  • There is no shared state in between the services, this allows for fast scaling by adding more nodes at any time
  • This shared nothing design allows it to support a massive amount of concurrency
  • This makes cloud-era applications a perfect use case
  • Recent Features: Global Clusters, Static Large Object manifests, Bulk Requests, Quotas, CORS, Multi-range requests
  • 120 total contributors, 32% growth so far this year
  • What does this mean for CloudStack?
  • Swift has been supported with CloudStack since 3.0
  • For Secondary storage (CloudStack VM snapshots and templates in Swift) – To steal a term from my SAN days, this is the equivalent of SATA “cheap and deep” storage
  • What about Application Storage? – What about Availability Zones? Affinity? etc…
  • John talked a bit about this but no big details at this time
  • Example: User (not disclosed) that uses Swift and CloudStack
  • Application storage for user data – Massive compute cluster supporting a large user base, concurrent access across the data sets
  • Rapid provisioning of the new vm’s – spin up hundreds of vm’s in minutes

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: