What does this have to do with the just concluded April 2013 OpenStack Summit? Everything! The Summit was a big turning point for the just under three year old Open Source Cloud Platform Project. Never before has the Summit felt more like a real business conference with a clear-eyed agenda to architect a stable and viable enterprise ready cloud platform, rather than just a bunch of wild developers on a mission to save the world. After several years of intense development and shakedown, OpenStack is finally more than ready to take on the unique challenges of enterprises that want to deploy private clouds.

There were plenty of great examples of respected organizations, public and private that have crossed the innovation chasm by deploying OpenStack clouds into their production environments. The keynote sessions all highlighted large enterprise case studies to emphasis the message that OpenStack is a strong and very viable option. Here are just a few to whet your appetite:

• • Bloomberg has deployed a large scale internal cloud, for a variety of applications including analytics and support of their backend systems.
• • BestBuy built an OpenStack cloud to support their extremely spiky holiday rush traffic. In just eight months, 25% of their systems ran on their new cloud in time for the 2012 Christmas buying season. They found the new platform, combined with a move to a PaaS push button development environment and a server side delivery architecture, allowed them to speed delivery of their catalog content pages down to under 2 seconds, while saving over $500K per platform upgrade. How is that for some serious discounting!
• • Comcast Cable and NBC/Universal built an OpenStack cloud to move content delivery out of proprietary limited function set-top boxes and into backend servers, giving them more flexibility and capabilities for the development of new interactive features such as live streaming and gaming.
• • The NSA, yes the spies in DC, are using OpenStack. Of course, what, how and why they are using it were not shared, but clearly they are not overly concerned with reports of security deficiencies.
• • Samsung is rolling out a production grade platform that will support millions of users across the globe. If the success of the new Android based smartphones and tablets are any indication, Apple should be looking over its shoulder very carefully.

In conclusion, if your company is contemplating a public or private cloud deployment, you should be putting OpenStack at the top of your shortlist. The rich ecosystem of big names, such as Dell, IBM, RedHat and Cisco is a clear signal that OpenStack has arrived!

About the Author
Beth Cohen is a senior cloud architect for Cloud Technology Partners, Inc., focused on delivering solutions to help enterprises leverage the efficiencies of cloud architectures and technologies. Previously, Ms. Cohen was the director of engineering IT for BBN Corporation, where she was involved with the initial development of the Internet, working on some of the hottest networking and web technology protocols in their infancy.

Enterprise SaaS takes off – With $14.5 billion in SaaS sales (an increase of 17.9% from 2011), lots of providers are getting it right. The biggest news is the boom in enterprise SaaS application adoption. Originally touted as the great leveler, allowing small businesses to take advantage of sophisticated systems hereto only affordable by big companies, the enterprise is rapidly dumping their high maintenance in-house systems and deploying a variety of SaaS services instead. Salesforce of course has long been a big player in this space, but the Workday’s spectacular IPO in October indicates a bright future. Look for lots of interest in Microsoft cloud products such as Office 365 and Azure as companies realize that these are cheaper and more flexible alternatives to traditional desktop tools. The rapid adoption of mobile devices in the workplace and its demand for more business customized apps is only going to accelerate this trend.

OpenStack grows up – While it remains to be seen if OpenStack wins the cloud infrastructure wars against VMware, CloudStack and Eucalyptus, given the number of cloud service providers lining up behind HP and Rackspace to roll out Openstack commercial services, it is not difficult to predict that 2013 will be another banner year for OpenStack. Practically the every major tech company in the world with the notable exception of Amazon is throwing their support behind OpenStack. On the technology side, more tools and functionality than ever makes the future of the largest cloud Open Source project ever definitely rosy.

Cloud Hardware Architectures get real – Over the last year or so, several vendors including VCE, the uneasy coalition of EMC, Cisco and VMware, Dell, and NetApp, have announced prepackaged cloud hardware stacks. On the surface the idea is appealing to enterprise IT infrastructure teams unprepared for the cloud revolution. However, as companies quickly found out, there is a big difference between dropping in a rack of hardware and building a productive enterprise cloud infrastructure. Since a primary cloud objective is hardware and software abstraction, more vendors will be developing infrastructure architectures tolerant of commodity hardware and supportive of transparent upgrades.

VMware Cloud gets it right – All indications are that 2013 will be the year that VMware finally gets it right after years of passing virtualization off as cloud. Enterprises that have been patiently waiting for a full suite of cloud features and tools will be rewarded with a system that will be expensive (what else is new), but actually delivers the goods.

Cloud Tools mature – With more offerings than ever from startups and mature companies alike, the market for sophisticated tools will be heating up as companies realize that they need orchestration, brokering, PaaS and cloud management suites. There will be lots of activity, new offerings, acquisitions, and of course, the inevitable hype.

About the Author
Beth Cohen is a senior cloud architect for Cloud Technology Partners, Inc., focused on delivering solutions to help enterprises leverage the efficiencies of cloud architectures and technologies. Previously, Ms. Cohen was the director of engineering IT for BBN Corporation, where she was involved with the initial development of the Internet, working on some of the hottest networking and web technology protocols in their infancy.

As I mentioned my previous post on the recently concluded OpenStack Summit held in San Diego October 2012, OpenStack needs to be taken seriously by anyone who is interested in building a public or private cloud.  For the more technically inclined the latest Folsom release has several new modules and features of interest to the enterprise and cloud service provider alike:

Quantum – Software Defined Networking (SDN), the latest darling of the cloud world is moving forward quickly with some really valuable features including L2 to L3 tunneling and a network API.  Expect to see lots of new development here.  On a side note, there was a great panel on the future of SDN moderated by Ken Pepple from Cloud Technology Partners, with people from Midokura, Big Switch and HP Cloud Services talking about their vision of the future for SDN.

Cinder – Now that Cinder has been spun out as its own named project, new features include intelligent location of Virtual Machine image storage, real snapshots, live migration, and more capability for large scale simultaneous VM initiations.

Keystone – New features includes support for role-based identify management (RBAC), PKI functionality, tools to add integration capability with enterprise grade account management systems such as Active Directory and other LDAP based systems.  Rumors of Kerberos support coming were floating around.

Documentation and security are both finally being taken seriously, with a half day documentation track on Monday morning and a full day of security related sessions on Thursday.  There was recognition among the lead techs that the developers who are creating OpenStack are not the users.  This resulted in much discussion regarding new features to make it easier to do deployments, upgrades (finally!) and manage it by operations folks.  There even was some discussion of IaaS/PaaS integration and how the VM’s work with the platform, a long overdue recognition that the IaaS and PaaS layers of the cloud stack are intimately related.

On the subject of operations, the tools to manage OpenStack are still weak, but they are no longer non-existent.  Ceilometer and heat among others look promising, but the better operations management tools are still mostly part of separate distributions such as Cloudscaling, Nebula, StackOps and others rather than being part of the product core.  While I do see the reasoning behind this thinking, at the very least we really need to have some standardized installation tools.  There is little benefit for every instance and distribution to reinvent the wheel.

One final note, one thing that was quite noticeable was that the Grizzly design sessions had a very different vibe from previous summits.  While some were packed with lots of discussions about new features, more often the Technical Project Lead and a few others ran the show with little input from the other participants in the room.  As the number contributors grow, the community is going to need to work hard to avoid having the immediacy of the Summit be diluted.

That doesn’t mean there isn’t plenty of meat already, but as can be seen by this tiny sampling of the current hot projects, there is still much work to be done.  The good news is the OpenStack community is ready willing and capable of delivering the goods in the coming months and years.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

From CTP’s perspective, since the majority of our customers are the big enterprise, we need to evaluate IaaS providers against a very different set of criteria. The enterprise is going to be more cautious – partially because they need to be and partially because they can afford to be — about putting anything beyond dev/test in these types of public IaaS services.  Gartner only touched on the IaaS requirements that are really important to the enterprise, regulatory compliance, true high availability SLA’s, support for large scale global deployments and a cost structure that isn’t essentially linear.  From that perspective, the Magic Quadrant would look very different: CSC, IBM, HP, Savvis and Terremark would all be top tier players, with Rackspace and Amazon in the niche quadrant.

On a side note, it is quite noticeable just how many of the providers on the list have at least some OpenStack built into their infrastructure.  HP Cloud Services is 100% based on OpenStack, while RackSpace is migrating from their legacy infrastructure as fast as they can convert their customers.  Internap, AT&T and others have either already stood up OpenStack services or have announced that they are in the works.  Definitely validates the OpenStack approach.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

After four extraordinary days of immersion into all things OpenStack at the just concluded OpenStack Summit in San Diego, I can heartily say without reservations that OpenStack is not only very real, it is truly a game changer for the IT industry.  With that being said, yes, there is certainly plenty of hype about this Open Source Infrastructure as a Service (IaaS) offering – one wag even quipped that OpenStack was at the peak of the Gardner hype cycle, so it must be real.  Here are just a few highlights for the terminally busy:

• The technology has come a long way since the project’s August 2010 inception.  With 13 major corporate foundation contributors, it seems like the only major technology companies not involved are Amazon and Apple.
• There was much discussion about the newly created OpenStack Foundation as an organization and its long-term viability.  However, compared to where the Apache Foundation was at the same time in its lifecycle, the OpenStack Foundation is far ahead of the curve.
• OpenStack is definitely gaining market traction.  There were plenty of academic, enterprise and service provider IT folks kicking the tires.  To encourage more market adoption, the website is featuring user stories.  A new one from WebEx was inspired by a visit to the Boston conference where the engineers saw the Mercardo Libra success story.
• While there was still a preponderance of developers in attendence (it was a design summit after all), there were more operations and business types than ever.  The vendor area was busier than ever.
• The biggest technical buzz was around Quantum and virtualized Software Defined Networking (SDN) — standing room only at the technical update and all the design sessions, but Cinder, and Keystone were not far behind in incorporating new features and functionality.
• Every time I turned around there seemed to be a new distribution available from both the usual suspects, such as Piston and Nebula, and the big players such as Cisco, SuSe and ???.  Niki Acosta’ Vapor Trail blog has a nice summary of the many vendor announcements.  More on the case of the proliferating distributions coming…
• And finally, the parties this time were for the most part more subdued.  HP’s soirée at the New Children’s Art Museum was cool, but the endless Techno Musak at all of them just gave me a pulsating (pun intended) headache.

As can be seen from this tiny sampling of the Summit activities, an enormous amount of energy and momentum has built up in just over two years.  For a project that will likely eventually be as important as Apache and Linux, every organization and person who has contributed to it should be justifiably proud of their accomplishments so far.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

You would think that cloud technology would have standardized long time ago.  While network standards that shape the Internet have been widely accepted throughout the industry, cloud standards have had a much slower adoption path.  Sadly the current state of cloud standards is, after 15 years, still far less mature than it ought to be.  As cloud infrastructure technology matures, increased interest by the enterprise and emerging vendors is driving a renewed effort to create viable standards that will benefit everyone.

For companies looking for integration and the capability to build hybrid clouds, various standards and proprietary and open APIs have been proposed to provide interoperability up and down the three layers of the cloud stack.  The first and so far only, cloud-oriented standard that has been ratified is the Open Virtualization Format (OVF), which was approved in September 2010 after three years of processing by the DMTF.  OVF’s open packaging and distribution format offers some platform independence by allowing migration between some platforms, but it does not provide all the tools needed for full cloud interoperability.

I think we can all agree that everyone benefits from technology standards in the long-term – the operative word is long-term.  You would think that everyone would agree that cloud infrastructure technology standards should be given high priority, but creating compelling proprietary systems and discouraging standards gives early adopter companies competitive advantage in the short-term.  Think about Amazon and VMware’s vast technology and market leads in the cloud services and enterprise infrastructure systems respectively.  They have little motivation to support any standards that have the potential to undercut their monopoly market positions.

Users of cloud are asking when will cloud computing standards mature enough so that more companies will feel comfortable implementing cloud architectures and using cloud services without feeling locked in.  Ironically, while the commercial cloud offerings have been growing, built on the very standards that created the Internet itself, Amazon and others have been reluctant to publish their architectures.  Application Programming Interfaces (API), which hide the underlying architecture, are all well and good, but they do not guaranty true interoperability.  Downstream vendors quickly find that they need to build API interfaces for all the different services they need to support; adding significantly to the development and maintenance costs.  To address this and transparently transfer workloads among the different vendor based on predefined business rules, there needs to be much more comprehensive standards.

One obvious question to ask is if there is an opportunity for the commercial cloud systems to become standards.  After all, there have been precedents where formerly proprietary formats, such as VMDK (Virtual Machine Disk) which was developed by VMware, have become de facto standards simply by being widely adopted by the industry.  One could even argue that platforms such as Amazon’s AWS are already standards.  However, as many companies have found to their chagrin, while it has a vast variety of services, easy to use tools, and a significant technological head start on the competition, it is more of a cloud world roach motel.  Lots of companies have found it easy to get applications running quickly, but changing providers or taking the applications back in-house as requirements change is fraught with unexpected perils.  Amazon’s backing of Eucalyptus does not address that problem directly, but it does offer a viable option for companies that want to build what Amazon euphemistically calls, on-premise services.

In conclusion, the good news is that the cloud industry is finally reaching consensus that the time to build cloud interoperability standards is long overdue.   The biggest need remains for interoperability standards to allow virtual machines to be migrated between clouds transparently and for more robust hybrid cloud solutions.  For the moment companies that want to use multiple platforms or a mix of public and private options are stuck with complex architectures and emerging orchestration tools such as enStratus and Rightscale to bridge the gap.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

In a previous discussion of cloud high availability, I covered in general terms what are some of the principals and approaches that make sense in a cloud environment. This time we will dive into some details of how this can be achieved in an Openstack environment.

The average published MTBF on switches seems to be between 100,000 and 200,000 hours. This number is dependent on the ambient temperature of the switch in the data center. I am assuming that most modern data centers are properly cooled for maximum switch life. This translates to between 11 and 22 years. Even in the worst case of poor ventilation and high ambient temperatures in the data center, the MTBF is still 2-3 years based on research found at http://www.garrettcom.com/techsupport/papers/ethernet_switch_reliability.pdf.

The mean time to replacement (MTTR) for a switch is going to be dependent on how exactly how the data center is staffed and what processes are used for replacing switches. Assuming that you would keep a few spares in the data center and that it is fully staffed 24 hours/day, the average time to replace a switch including configuration is going to be under 2 hours. Most modern switches are auto-configured so the actually provisioning time after the switch is powered up in the rack is under 5 minutes.

Let me walk through what will happen in the case of a top of rack (ToR) switch failure in the Swift cluster. Swift by its nature is fault tolerant at the rack level. That means that the system will continue to operate without data loss if an entire rack goes off-line. The cluster would detect the rack being off-line and send out a notification that the NOC staff would see within 5 minutes. In the case of rack going off-line Swift does not automatically move any data. The reason for this is that in fact, the NOC staff needs to make a decision about the cause of the rack going off-line and how long it will take for it to come back on line. In the case of a switch failure, the data in the rack is still intact, so it is far more efficient to just replace the switch then bring the rack back on-line without having to move the data. Even if the NOC staff decides to move data around, which they would only do if the fault is in the servers not the switch, the network overhead that it adds to the cluster is in the range of 3-5% for a large cluster with properly tuned ring rebuild cycle. Clearly taking a rack off-line is not considered a problem. I would argue that you should expect to be able to take racks off-line with no impact to the system as a whole as a matter of course for maintenance, upgrades and other reasons.

Nova behaves is slightly differently in the case of a rack failure. Unlike Swift the architecture does not have an assumed base unit of failure at the rack level. It does have the concept of a availability zone, which is quite different from a Swift zone just to confuse things. That doesn’t mean that you cannot create an equally fault tolerant Nova architecture, it just requires more development of high availability at the application level of the system combined with the use of the availability zone as a mechanism for balancing the applications in different locations. The assumption is that it is the responsibility of the application to build in fault tolerance, not the underlying infrastructure to keep track of the individual VM instances. Nova zones can be used to achieve this level of fault tolerance in combination. Combining this with the a live migration functionality and HA application design will allow you to build support for rack level failure. Again, the metrics for determining the next steps (replacement of switch only or rebuilding of entire rack) will be based on the specific component failure. See the recent discussion of this at http://lists.us.dell.com/pipermail/crowbar/2012-January/000643.html for more ideas on how to architect such a system.

Another approach would be to create high availability through redundant hardware. In this case you could provision the racks with two switches. However this is an expensive option in a large data center with hundreds of racks. It is clearly orders of magnitude more expensive to take this approach. From a risk perspective, you have substantially increased your per rack costs with little or no reduction in risks since the rate of failure is so low to begin with and the architected unit of failure for a cloud infrastructure should be at the rack level to begin with.

About the Author

Beth Cohen, Cloud Technology Partners, Inc. Transforming Businesses with Cloud Solutions

In our work, clients often ask for a cloud reference architecture.  They see it as a holy grail that will provide clear vision and help define their cloud strategies.  In response to a recent client inquiry about a cloud hardware reference architecture that could be used to define a joint venture with another company, I realized that defining a cloud reference architecture as the same hardware platforms was the wrong way to look at the question.  While I do agree a reference architecture is certainly a good starting point, it not something I would build an entire corporate cloud strategy around.

Reference architectures are supposed to be by their nature somewhat theoretical.  Like the OSI model and the cloud stack model, they are artificial constructs or frameworks that can be used to create real systems.  By themselves they tend to be so generic and broad that building an enterprise cloud based on a reference architecture without considerable work defining business objectives and system requirements would only be an exercise in frustration.  For example, an enterprise cloud designed to be used for internal use as a more flexible development platform is a far different animal than an enterprise cloud planned to be used to support massively scalable customer facing applications.

After looking at a few available cloud reference models, a reference architecture means very different things depending on the agenda of the creator.  A useful way to compare the perspectives of the various cloud reference architectures is to map them to the older and simpler Cloud Service Stack Model: IaaS, PaaS and SaaS.  For example: both the NIST and its closely related IBM architecture are relatively generic and high level, but they both have an operational focus which closely matches the IaaS layer.  The recently published RackSpace Private Cloud Reference Architecture while specific to OpenStack, also primarily has an operational/IaaS bias.  Microsoft has defined the cloud not surprisingly more from a development platform/PaaS view, but it also has an IaaS flavored version based on Hyper-V.  The HP and VMWare versions are more appropriate for companies building end to end applications, SaaS or otherwise.

In the end, a good cloud reference architecture should be robust enough to allow it to be used from a variety of different perspectives: business, operations, development and consumer.  However, any company that is foolish enough to try to use a cloud reference architecture to create its strategy without the application of a rigorous amount of common sense and basic good business practices, is likely to be disappointed with the results.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Moving companies’ IT services into the cloud the right way, the first time!

Question: When will cloud computing have standards so that more companies will feel comfortable implementing cloud architectures and using cloud services without feeling locked in?

Cloud computing is finally maturing as a technology.  After many false starts and the proven success of proprietary products and solutions such as Amazon and VMWare, the Open Source community is finally coalescing to devote its considerable resources and energy to the long recognized need for cloud computing standards.  OpenStack, with close to 6000 contributors and over 100 companies backing the project, including such giants such as Dell, Rackspace, Citrix, HP and NetApp, it is one of the fastest growing Open Source initiatives ever undertaken.  The project just celebrated its first year (well actually 14 months, but who’s counting) anniversary.  At the recent OpenStack Conference it was announced that the initiative has been spun out of Rackspace and officially organized as an industry standards organization similar to the Linux Foundation.  What that means is still open to definition; see Andy Oram’s recent blog at OpenStack Foundation requires further definition for more on that.  However, it is a positive sign that the OpenStack community is serious about keeping the technology available to any and all who want to create software and tools that support real cross platform IaaS cloud integration.

For a project that is just over a year old, there is already plenty of exciting work to show for the effort.  Following the Diablo release, the Essex Release Developers Summit held in Boston October 3-7, 2011, was where 250 hardcore developers, product managers and movers and shakers in the OpenStack community met to brainstorm the roadmap for the Essex release scheduled for April 2012.  There are plenty of opportunities to create the architecture and the tools to make OpenStack enterprise ready, but it will not be an easy or trivial task to do it.  There are already some shiny new companies who have the attention of the venture community.  Piston Cloud Computing, founded by several luminaries from the OpenStack Project, including former NASA Nebula Chief Technical Architect Joshua McKenty, and former Rackspace technologist Christopher MacGown, was launched with $4.5M in funding with no customers or product.  While these folks have solid credentials, can we say there might be more than a hint of another Dotcom bubble here?

From the technology perspective, there is considerably less hype and more meat to the project.  Led by Rackspace, with considerable support from plenty of others, the development of the two main OpenStack components, the compute engine code named Nova and the object store code named Swift are moving along quickly.  Integration of the Keystone authentication engine, the Glance image library management tool and other necessary elements, such as a more professional GUI Dashboard are planned for the upcoming release.  There seems to be general agreement in the community that Essex development will primarily be focused on making the components robust enough for enterprise production environments.

Speaking of production deployments, sitting in on the OpenStack deployment panel at the conference was interesting.  Yes, there are already some OpenStack deployments in full production, most notably at Rackspace.  For companies without a deep bench of very senior Linux systems administrators and DevOPS engineers who are happy mucking with syslogs to draw on, OpenStack does not yet have the tools and ecosystem in place to support a deployment managed by mere mortals.  That does not mean that companies should not be looking at it seriously now, because the tools are coming very fast.  There are already offerings from Rightscale, Cloudscaling and StackOPS, to name just a few.  With the solid Swift and Nova components that are already at least partially built, the foundation is already there for entrepreneurs to build the tools needed to make the technology production ready by early next year.  The time to start planning for a production deployment is now.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Moving companies’ IT services into the cloud the right way, the first time!