From the Enterprise perspective, now has never been a better time to invest in cloud services. Enterprises are broadly adopting all types of cloud services at multiple levels in the organization. Initial predictions were for the enterprise to favor private and community clouds over public services, but the hottest trends in Cloud adoption has been Software as a Service (SaaS) and cloud-based business applications which are projected to grow from $13.4 billion in 2011 to $32.2 billion in 2016, a 19.1% five-year CAGR! The rapid adoption of SaaS applications of all flavors by the enterprise has been a surprise to many, but the vastly reduced costs due to the pay as you go pricing models, and high degree of service delivery flexibility has overcome any perceptions of needing to trade price for reduced feature sets. Unsurprisingly, the most often added cloud-based application services are Customer Relationship Management (CRM), Supply Chain Management (SCM) and Enterprise Content Management (ECM), with Web Conferencing, teaming platforms and social software suites nipping at their heels.

Infrastructure as a Service (IaaS) continues to appeal to companies that want to replace in-house and traditional data center models for the cloud hardware abstraction approach. Gartner is predicting Infrastructure-as-a-Service (IaaS), cloud management & security devices, and Platform-as-a-Service (PaaS) are growing from $7.6B in 2011 to $35.5B in 2016, a CAGR of 36%. A word of warning for companies that think just because the Amazon cloud is easy to use, it is easy to build and manage. Building a private cloud still requires a degree of expertise that few enterprises have in-house. In the next year or so expect more companies to see the value of using the services of cloud consultants to avoid painful and expensive mistakes when building private clouds; or alternatively using emerging enterprise focused public cloud services such as Bluelock or Terremark.
Platform as a Service (PaaS) which still does not quite know what it wants to be when it grows up, is lagging with only about $1B in revenue in 2012, but as the market matures, expect to see rapid uptake as companies recognize the value of standardized tools that can ease the pain of Cloud application deployments. Some of the newer PaaS tools like the ServiceMesh Agility Platform combine the features of a SDLC workflow engine, production support and orchestration across different cloud platforms.

New Cloud Technology Directions

Ultimately, innovation is the marriage of technology and organization change. The dilemma is how to pull innovation into IT core functionality without disrupting the flow of new ideas, when the modern enterprise understanding of IT is focused on operational excellence and cost control. The CIO should be leading the cultural change to the new flat organization by leveraging cloud and mobile application in new and interesting ways. The trick is to create tools for people to quickly test and validate their ideas so they can implement new ideas that work within the enterprise framework. Cloud tools can be used to deliver on that promise, but is the enterprise up to that challenge?

This is not the first time wrenching technology innovation has changed how companies do business, and it certainly will not be the last. Companies able to use disruptive technologies, such as cloud computing, effectively will leave companies who do not have that ability in the dust. By recognizing the seeds of change and embracing them, the smart company can leverage cloud services by using the efficiencies of pooled IT resources at the same time allowing greater flexibility to meet the challenges of the global economy. This combination of combining commodity utilities with innovation allows companies to compete effectively using the efficiency and flexibility strategies simultaneously.

Challenges, Always Challenges…

Consumers intuitively get cloud and have been more than willing to embrace it warts and all as it has matured. They are highly price sensitive and will easily sacrifice features in exchange for low cost. On the other side of the market spectrum, the more conservative enterprise market is still struggling with the basic model. Some companies worry that the emerging cloud companies are too small to do business with, while others are concerned with how to incorporate new technologies into existing technology portfolio investments. To address their concerns, at the same time as Cloud technology continues to mature and standards develop , innovation around service delivery and breakthroughs in storage technologies are making it ever more enterprise ready. The pace of cloud vendor consolidation has already picked up as the traditional enterprise vendors such as HP and IBM has rushed to add enterprise ready cloud services to their portfolios. This should alleviate the fears of even the most technology adverse companies.

A final word for any remaining cloud technology skeptics, as with any innovative force, the cloud is just a tool to drive change, not an embodiment of change itself. Enterprise cloud consulting leaders, have firsthand experience with how companies willing to ride the Cloud revolution will not only survive in today’s hyper-competitive world, but thrive. I cannot wait to see where the next wave takes us!

About the Author
Beth Cohen, Beth Cohen is a senior cloud architect for Cloud Technology Partners, Inc., focused on delivering solutions to help enterprises leverage the efficiencies of cloud architectures and technologies. Previously, Ms. Cohen was the director of engineering IT for BBN Corporation, where she was involved with the initial development of the Internet, working on some of the hottest networking and web technology protocols in their infancy.

As I mentioned my previous post on the recently concluded OpenStack Summit held in San Diego October 2012, OpenStack needs to be taken seriously by anyone who is interested in building a public or private cloud.  For the more technically inclined the latest Folsom release has several new modules and features of interest to the enterprise and cloud service provider alike:

Quantum – Software Defined Networking (SDN), the latest darling of the cloud world is moving forward quickly with some really valuable features including L2 to L3 tunneling and a network API.  Expect to see lots of new development here.  On a side note, there was a great panel on the future of SDN moderated by Ken Pepple from Cloud Technology Partners, with people from Midokura, Big Switch and HP Cloud Services talking about their vision of the future for SDN.

Cinder – Now that Cinder has been spun out as its own named project, new features include intelligent location of Virtual Machine image storage, real snapshots, live migration, and more capability for large scale simultaneous VM initiations.

Keystone – New features includes support for role-based identify management (RBAC), PKI functionality, tools to add integration capability with enterprise grade account management systems such as Active Directory and other LDAP based systems.  Rumors of Kerberos support coming were floating around.

Documentation and security are both finally being taken seriously, with a half day documentation track on Monday morning and a full day of security related sessions on Thursday.  There was recognition among the lead techs that the developers who are creating OpenStack are not the users.  This resulted in much discussion regarding new features to make it easier to do deployments, upgrades (finally!) and manage it by operations folks.  There even was some discussion of IaaS/PaaS integration and how the VM’s work with the platform, a long overdue recognition that the IaaS and PaaS layers of the cloud stack are intimately related.

On the subject of operations, the tools to manage OpenStack are still weak, but they are no longer non-existent.  Ceilometer and heat among others look promising, but the better operations management tools are still mostly part of separate distributions such as Cloudscaling, Nebula, StackOps and others rather than being part of the product core.  While I do see the reasoning behind this thinking, at the very least we really need to have some standardized installation tools.  There is little benefit for every instance and distribution to reinvent the wheel.

One final note, one thing that was quite noticeable was that the Grizzly design sessions had a very different vibe from previous summits.  While some were packed with lots of discussions about new features, more often the Technical Project Lead and a few others ran the show with little input from the other participants in the room.  As the number contributors grow, the community is going to need to work hard to avoid having the immediacy of the Summit be diluted.

That doesn’t mean there isn’t plenty of meat already, but as can be seen by this tiny sampling of the current hot projects, there is still much work to be done.  The good news is the OpenStack community is ready willing and capable of delivering the goods in the coming months and years.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

On paper, the concept of the development and operations part of an IT organization working in concert is wonderful.  In fairness when a company makes the effort to encourage the organizational changes that need to be done, the productivity gains, velocity and quality improvements of the IT systems more than make up for the pain of the organizational changes.  However the effort of making that transformation is often far more fraught with technical obstacles and political stumbling blocks than many over-stretched IT organizations is capable of handling.  Fortunately with proper guidance there are Agile and organizational transformational techniques that can be used to overcome the issues.

Often the reasons why these transformations are not as successful as they could be fall into three categories:

Organization, Political and Business Issues – Anyone who has ever worked knows that organizational issues are the most deep-seated and difficult to overcome.  The old boys’ networks and entrenched ways of thinking are sometimes insurmountable roadblocks.  There are several methods to address these issues, but in all cases strong sponsorship and support from the executive ranks is essential for a successful creation of a DevOps mindset in an organization.  One way to influence the change is to take a page out of the old organizational change playbook; create an organization that aligns with the desired outcome.  In one large enterprise the solution to the resistance to changing from the old IT operational mindset was to reorganize the entire IT organization so that the development and operations groups reported into the same structure.  The manager of both groups was a strong believer in the need for DevOps to support the company’s multi-million dollar cloud initiative.  By forcing the groups to work together in Agile Swarms, they were able to build the trust and skills needed to bring about the organizational transformations.

Outsourced IT Business Model – Yup, you read that correctly.  For all the touted benefits of outsourcing, and I will agree that there are many, agility and organizational transformation is not one of them.  The outsource vendor is motivated to manage risk by eliminating as many unknowns as possible.  Like traditional IT operations the easiest, but no always the best way to accomplish this goal is by minimizing changes in the environment.  This encourages rigid thinking and highly delimitated roles.  It isn’t always that the work is off-shored that causes problems.  Often the issues are related to poor project management, unclear articulation of the team goals, and a disconnect between the purchaser of the services and the vendor.  One company is mostly using American labor, but they are so stuck in their ways that they are just starting to think about breaking down the barriers between the functional groups.  Another is having an IT meltdown on the operations side of the house which is causing them major headaches.  Communications issues across the organization and between the vendor and the customer in this mixed shop are behind much of the conflict.

Staff Skills – This is an often overlooked aspect behind the failure of DevOps initiatives, the skills and working habits of development and operations people are typically wildly divergent.  Developers often work on new projects, so are typically more willing to take risks, work with underdeveloped and/or buggy code and generally are not very concerned if things break along the way.  They know that breaking things is just a routine part of the development process.  On the other hand, IT operations people are paid to keep systems running as smoothly as possible.  They are often held to strict SLA’s so they take five-nines seriously.  One way to keep an environment stable is to introduce change as little as possible.  IT operations are famously resistant to changing code on the fly or introducing other things that might cause them to be paged in the middle of the night when the systems have a hiccup.  Who can blame them really?  This defensive mindset can be addressed by applying newer architectures and technologies that are tolerant of component failure – the automated deployment and designed to fail architectures.  Once it is demonstrated that a failure of a component is not a catastrophe, just one more expected event that has no immediate impact on the systems, the next logical step is to build processes that allow continuous incremental changes to the systems.  Encouraging a set of shared standards is a good way to develop DevOps processes across the organization.  For one enterprise this meant building a service catalog of standard images for the developers use with all the commonly used tools.  The developers given the option of using either a pre-built applications tools platform or building a custom one took the easier path.  Standard tools problem solved.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

To understand how to manage your high availability options, it is important to have a discussion about how high availability, risk and component failure work in a cloud environment.  High availability is very important for cloud environments; often the cloud provider is required to meet strict service level agreements for 99.99% or 99.999% (the so called 5-nines) availability.  In theory, that is the very reason that customers are interested in using cloud services.  It should be noted that most of the public cloud service providers have lots of methods to measuring availability that is in their favor, so even in the case of catastrophic systems failure, they are rarely accountable for the downtime.  This has been one of many sources of caution for enterprises that have wanted to leverage public cloud services.

Before going into the details of how to quantify the cost of risk mitigation for a cloud, a short discussion of the science of risk management will help with understanding how it all works.  The goal of business risk management is to detail what kinds of risks exist in your specific business and determine how to prevent them entirely or minimize their impact on the business as a whole. Business risk management is essentially quantifying the risk that a given system will fail multiplied by the cost.  Cost is further broken into two more categories.  Out of pocket costs, also referred to as sunk costs, and lost opportunity costs.  Sunk costs are costs that you will need to pay out to fix the problem, while lost opportunity costs are revenue lost due to the system unavailability.  For example, the risk that there will be regular earthquakes in Japan is high.  The Japanese have responded to this threat by having some of the strongest earthquake resistant building codes in the world.  However, as last year’s 9.0 tremor and following tsunami so dramatically demonstrated, it is impossible to prepare for such extreme and rare events.

High availability is best addressed by redundancy.  However, redundancy can be achieved at several levels of the IT infrastructure: hardware, software, network, or a combination.  Traditional IT organizations have reduced the risk of downtime by concentrating almost exclusively on hardware redundancy.  The scale of the cloud, where there are already thousands or hundreds of thousands of systems, hardware redundancy at the component level quickly becomes unsustainably expensive.   A telling scholarly article that looked at the reported hardware failures from several large data centers shows that by far the most likely failure at data center scales is as would be expected the components that have moving parts, such as hard drives and power supplies.

In practice, the enterprise needs to balance the cost of duplicating hardware throughout the cloud ecosystem, which is the traditional approach to solving the risk management problem, against the need for keeping operating expenses low. Another consideration is that duplicating everything at the hardware level does not automatically guaranty that you do not still have a single point of failure in the environment.  For example, you might have remembered to contract with multiple carriers to spread the risk of a network outage, but if they all come into the data center at a single location, the data center is still prone to a catastrophic “backhoe failure”, which is what happens when a backhoe has severed all the up-link cables in one fell swoop.  It is an expensive and time consuming repair that leaves many unhappy customers in its wake.  Yes, there are ways to mitigate this risk, but they are expensive and need to be balanced against the relative probability of such an event.

The best approach is to look at the probability of failure of each component in context of the entire ecosystem.  Since hard drives fail at such a high rate, the hardware approach is to mirror or RAID the drives across thousands of systems.  This translates to data redundancy and added costs that far exceed the optimum for availability and cost reduction.  At scale, building a storage system that handles the data redundancy at the software level is far more efficient.  Another examples, is planning for power supply (or more precisely fan) failures.  Again, since they are generally the next most common component to fail, instead of filling the cloud data center with thousands of extra power supplies and fans, it is better to build the cloud to be resistant to downtime if a server node fails.  In the end, addressing cloud high availability is not only about determining the MTTF of hard drives, cables or switch ports, it is also balancing it against the likelihood of a given failure at the data center macro level.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions