Enterprise SaaS takes off – With $14.5 billion in SaaS sales (an increase of 17.9% from 2011), lots of providers are getting it right. The biggest news is the boom in enterprise SaaS application adoption. Originally touted as the great leveler, allowing small businesses to take advantage of sophisticated systems hereto only affordable by big companies, the enterprise is rapidly dumping their high maintenance in-house systems and deploying a variety of SaaS services instead. Salesforce of course has long been a big player in this space, but the Workday’s spectacular IPO in October indicates a bright future. Look for lots of interest in Microsoft cloud products such as Office 365 and Azure as companies realize that these are cheaper and more flexible alternatives to traditional desktop tools. The rapid adoption of mobile devices in the workplace and its demand for more business customized apps is only going to accelerate this trend.

OpenStack grows up – While it remains to be seen if OpenStack wins the cloud infrastructure wars against VMware, CloudStack and Eucalyptus, given the number of cloud service providers lining up behind HP and Rackspace to roll out Openstack commercial services, it is not difficult to predict that 2013 will be another banner year for OpenStack. Practically the every major tech company in the world with the notable exception of Amazon is throwing their support behind OpenStack. On the technology side, more tools and functionality than ever makes the future of the largest cloud Open Source project ever definitely rosy.

Cloud Hardware Architectures get real – Over the last year or so, several vendors including VCE, the uneasy coalition of EMC, Cisco and VMware, Dell, and NetApp, have announced prepackaged cloud hardware stacks. On the surface the idea is appealing to enterprise IT infrastructure teams unprepared for the cloud revolution. However, as companies quickly found out, there is a big difference between dropping in a rack of hardware and building a productive enterprise cloud infrastructure. Since a primary cloud objective is hardware and software abstraction, more vendors will be developing infrastructure architectures tolerant of commodity hardware and supportive of transparent upgrades.

VMware Cloud gets it right – All indications are that 2013 will be the year that VMware finally gets it right after years of passing virtualization off as cloud. Enterprises that have been patiently waiting for a full suite of cloud features and tools will be rewarded with a system that will be expensive (what else is new), but actually delivers the goods.

Cloud Tools mature – With more offerings than ever from startups and mature companies alike, the market for sophisticated tools will be heating up as companies realize that they need orchestration, brokering, PaaS and cloud management suites. There will be lots of activity, new offerings, acquisitions, and of course, the inevitable hype.

About the Author
Beth Cohen is a senior cloud architect for Cloud Technology Partners, Inc., focused on delivering solutions to help enterprises leverage the efficiencies of cloud architectures and technologies. Previously, Ms. Cohen was the director of engineering IT for BBN Corporation, where she was involved with the initial development of the Internet, working on some of the hottest networking and web technology protocols in their infancy.

As I mentioned my previous post on the recently concluded OpenStack Summit held in San Diego October 2012, OpenStack needs to be taken seriously by anyone who is interested in building a public or private cloud.  For the more technically inclined the latest Folsom release has several new modules and features of interest to the enterprise and cloud service provider alike:

Quantum – Software Defined Networking (SDN), the latest darling of the cloud world is moving forward quickly with some really valuable features including L2 to L3 tunneling and a network API.  Expect to see lots of new development here.  On a side note, there was a great panel on the future of SDN moderated by Ken Pepple from Cloud Technology Partners, with people from Midokura, Big Switch and HP Cloud Services talking about their vision of the future for SDN.

Cinder – Now that Cinder has been spun out as its own named project, new features include intelligent location of Virtual Machine image storage, real snapshots, live migration, and more capability for large scale simultaneous VM initiations.

Keystone – New features includes support for role-based identify management (RBAC), PKI functionality, tools to add integration capability with enterprise grade account management systems such as Active Directory and other LDAP based systems.  Rumors of Kerberos support coming were floating around.

Documentation and security are both finally being taken seriously, with a half day documentation track on Monday morning and a full day of security related sessions on Thursday.  There was recognition among the lead techs that the developers who are creating OpenStack are not the users.  This resulted in much discussion regarding new features to make it easier to do deployments, upgrades (finally!) and manage it by operations folks.  There even was some discussion of IaaS/PaaS integration and how the VM’s work with the platform, a long overdue recognition that the IaaS and PaaS layers of the cloud stack are intimately related.

On the subject of operations, the tools to manage OpenStack are still weak, but they are no longer non-existent.  Ceilometer and heat among others look promising, but the better operations management tools are still mostly part of separate distributions such as Cloudscaling, Nebula, StackOps and others rather than being part of the product core.  While I do see the reasoning behind this thinking, at the very least we really need to have some standardized installation tools.  There is little benefit for every instance and distribution to reinvent the wheel.

One final note, one thing that was quite noticeable was that the Grizzly design sessions had a very different vibe from previous summits.  While some were packed with lots of discussions about new features, more often the Technical Project Lead and a few others ran the show with little input from the other participants in the room.  As the number contributors grow, the community is going to need to work hard to avoid having the immediacy of the Summit be diluted.

That doesn’t mean there isn’t plenty of meat already, but as can be seen by this tiny sampling of the current hot projects, there is still much work to be done.  The good news is the OpenStack community is ready willing and capable of delivering the goods in the coming months and years.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

From CTP’s perspective, since the majority of our customers are the big enterprise, we need to evaluate IaaS providers against a very different set of criteria. The enterprise is going to be more cautious – partially because they need to be and partially because they can afford to be — about putting anything beyond dev/test in these types of public IaaS services.  Gartner only touched on the IaaS requirements that are really important to the enterprise, regulatory compliance, true high availability SLA’s, support for large scale global deployments and a cost structure that isn’t essentially linear.  From that perspective, the Magic Quadrant would look very different: CSC, IBM, HP, Savvis and Terremark would all be top tier players, with Rackspace and Amazon in the niche quadrant.

On a side note, it is quite noticeable just how many of the providers on the list have at least some OpenStack built into their infrastructure.  HP Cloud Services is 100% based on OpenStack, while RackSpace is migrating from their legacy infrastructure as fast as they can convert their customers.  Internap, AT&T and others have either already stood up OpenStack services or have announced that they are in the works.  Definitely validates the OpenStack approach.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

Not long ago I spent a week in mid-April 2012 at the Folsom Openstack Summit/Conference in San Francisco.  There was much enthusiasm for the project among the 450 developers who attended the development summit portion of the program and even more among the 900 or so developers, product and business development folks who stayed for the two day general conference that followed.  Both from the business and technology perspectives, Openstack has come a long way in just the past six months.  Where it continues to lag was the noticeable lack of real users attending the conference to contribute their voices and valuable guidance.

On the vendor front, Rackspace, of course, continues to be the biggest major supporter overall; but more marquee names have announced their intention to join the fledgling Openstack Foundation including AT&T, HP, Redhat and IBM.  HP’s motivations are clear; it is in the process of standing up one of the largest Openstack public IaaS offerings with the public beta scheduled to go live in the middle of May 2012.  It certainly has the deep pockets and technical wherewithal to successfully go up against Amazon and Rackspace.  IBM’s incentives for supporting the project are less clear, but the company does have a long history of backing open source projects and using their support for their own ends – Linux is a prime example that immediately comes to mind.  The jury seems to be still out for early supporters, Citrix and Microsoft, both of which have not officially committed to the Foundation to date.

Looking at Openstack from the technical perspective, in the 18 months since the project’s inception, it has come a long way towards becoming a viable system that could be used in a production environment.   I would still argue as I did six months ago, that you still need a cadre of systems engineers and dev/ops people to build and support it, but the attendees at the technical sessions I joined recognized the need for better user documentation in general and more ways to engage the operations people who are stuck supporting something that is still pretty rough around the edges.  The frustration from the few end users who were in attendance was abundantly clear.  These people are not developers and don’t have time to figure out the missing pieces.  As one person put it so succinctly, at 2AM trolling through the code is not an acceptable method of troubleshooting a down system.

At the end of the day, Openstack will only successfully be adopted by the enterprise if the real end users, the operations people from service providers such as HP and Rackspace or enterprise customers, step forward and join the technical conversation. Really, who better to set the direction of such an ambitious open source project than the users?  To rectify this situation, I encourage enterprises that are considering an Openstack deployment project to start contributing to the community today and plan on sending their systems operations managers and architects to the next Openstack Design Summit in the fall.  The future is literally in your hands.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

Since the dawn of the Internet there has been an on-going debate over whether to use Layer 2 (Ethernet) or Layer 3 (IP) networking inside the data center.  In the beginning, yes, we are talking about the 1990′s, for the most part networks were built on Layer 3 protocols.  L2 switches were only used for internal LAN’s or very small installations.  Does anyone really fondly remember NetBIOS or SPX/IPX?  While Layer 2 switches were easy to deploy – one brand was appropriately named Black Box — they were impossibly slow and unreliable once you scaled past more than a couple hundred machines.  Before the development of public IP address sparing protocols such as CIDR, DCHP and NAT, if you wanted to have an internet connection you had to assign each system a public IP address anyway.

Fast forward 20 years and many network protocols later, data centers are now typically architected to use L2 switches rather than L3 routers.  The reasoning seems to be that Ethernet is faster because you don’t have the overhead of the IP hierarchy, and you don’t have to worry about reconfiguring IP addresses as systems get moved around.  For me the second argument doesn’t hold up, since that is exactly what DHCP and DNS are designed for!  The simplicity of Layer 2 protocols might work well in a data center with hundreds of physical machines, but cloud data centers have the additional burden of needing to keep track of all the virtual machine addresses and networks as well.  It is not uncommon for one physical node to support 30-40 VM’s.  Layer 2 switching protocols have improved mostly by adding “bolt-ons” such as VLAN’s, RBridges, or Cisco’s L2MP.  I would argue that these are all proprietary patches to the fundamental scale and complexity problem. They still don’t have the built-in hierarchy and resiliency of a fully routed IP network.

A better paradigm is to think of cloud data centers as miniature (or in the case of Amazon, not so miniature) versions of the Internet.  Thus applying the inherent scalability and flexibility of the IP address based Internet to a cloud data center network architecture makes perfect sense.

Cloud Network Architecture Basic Principles

• The cloud means, “Any server, any service, any time”
• Scalability through hierarchy
• Simplified network management
• Maximum network traffic flexibility
• Flattened traffic flow over the entire network mesh
• Minimize amount of state information maintained in network by keeping VM state (VM MACs and IPs) out of core network
• Reduce number of protocols to manage

Layer 2 Architecture Limitations

• Number of VLANs is limited to 4096
• Number of MACs stored in switch tables is limited
• Need to maintain a set of Layer 4 devices to handle traffic control
• MLAG (which is used for switch redundancy) is a proprietary solution that doesn’t scale beyond two devices and forces vendor lock-in
• Difficult to troubleshoot network without IP addresses and ICMP
• Configuring ARP is tricky on large L2 networks
• All network devices need to be aware of all MACs, even VM MAC’s, so there is constant churn in MAC tables and network state changes as VM’s are started or stopped
• Migrating MACs (VM migration) to different physical locations could be a problem if ARP table timeouts aren’t set properly

Layer 3 Architecture Advantages

• Provides the same level of resiliency and scalability as the Internet
• Easy to control traffic with routing metrics
• Can use BGP confederation for scalability, so core routers have state proportional to number of racks, not to number of servers or VMs
• It keeps per VM state (VM MACs and IPs) out of the network core, to reduce state churn. The only routing state changes are in case of a ToR failure or backbone link failure
• Uses ICMP to monitor and manage traffic

In future articles, there will be further dives into some of the ways that Layer 3 networks, virtual networks, and the most exciting new networking development, software only networks can be used to successfully address the needs of a cloud data center network.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

The magic of the cloud is that it can do anything.  It is both robust and flexible, the best of both worlds.  Ok, I admit that I have been spending far too much time reading cloud marketing materials lately.  Now back to reality.  Yes, the cloud is highly flexible and it can do almost anything, but if you want to get the most out of your private cloud investment, you need to pay attention to the underlying hardware as I discussed previously, and you need to define what you are planning on using it for by creating and testing use cases.

Use case planning seems counter-intuitive.  After all you can sign up for a web server with Amazon in about 5 minutes.  Amazon does not know what you are planning on doing with it.  Wrong.  Amazon’s product management department spends plenty of time figuring out exactly what would be attractive to their typical customer and honing the service to deliver it.  For the enterprise, the planning process is no different, but instead of planning for an external paying customer, for example, the use could be for internal application developers or a web portal.

To give you an idea of how this works, let us say, you are planning on using the cloud for the company’s E-commerce website.  This means that you will need to plan for applications that will support thousands of sessions per second, variable workloads and lots of complex and changing data.  By identifying the key metrics such as number of concurrent transactions per second, size of database, etc. you can then build a method for testing your assumptions.

To get the conversation started here is a short list of possible use cases for a private cloud.  Over the next few weeks I will be digging deeper into how to leverage the cloud model in the enterprise.

Archive storage — Many companies have moved to keeping their archives on line instead of on backup tape for many excellent reasons.  Using SAN or near-line storage is still expensive.  Cloud object or block storage is an attractive alternative because of its optimized approach to high availability.  It also scales nicely as archives grow over time.

Federated hypervisor/VM management – This is one of the main reasons that the enterprise is interested in the cloud in the first place – any server, any service, any time.  Adding self-service, charge back and transparent delivery of the right resources from a federated pool can be very cost effective.  Look for a cloud that provides cross platform hypervisor support and robust VM management tools.

Development and test – One of the best use cases for an enterprise cloud is a shared development and test environment.  Self-service is essential, but the private version allows much more control on resource use by using a rules based delivery model to optimize IT investments.  Creating an enterprise PaaS environment is also desirable because it allows better integration across applications and more standardized application development.

Application spaghetti rationalization – An enterprise cloud delivers better application portfolio management and more efficient deployment by leveraging self-service features, rules for deployments based on types of use.

Web services, portals and e-commerce – Web services of all sorts are a natural for the enterprise cloud.  They are well suited to being able to take advantage of the inherent elasticity and automated workload based provisioning and deployment capabilities.

VDI Support – VDI is another natural for an enterprise cloud.  VDI is often used to better maintain control over workers’ compute environments, but the workloads are inherently highly variable, which is an excellent reason for implementing such systems on the cloud.  An obvious extension is mobile application support which is a growing part of the enterprise service portfolio.

Disaster Recovery/Business Continuity — Again the cheap storage and VM management makes a good case for using the cloud as a secondary site.  The public cloud is already heavily used for these purposes, but moving the function in-house could be cost effective for a very large enterprise.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Transforming Businesses with Cloud Solutions

Question: When will cloud computing have standards so that more companies will feel comfortable implementing cloud architectures and using cloud services without feeling locked in?

Cloud computing is finally maturing as a technology.  After many false starts and the proven success of proprietary products and solutions such as Amazon and VMWare, the Open Source community is finally coalescing to devote its considerable resources and energy to the long recognized need for cloud computing standards.  OpenStack, with close to 6000 contributors and over 100 companies backing the project, including such giants such as Dell, Rackspace, Citrix, HP and NetApp, it is one of the fastest growing Open Source initiatives ever undertaken.  The project just celebrated its first year (well actually 14 months, but who’s counting) anniversary.  At the recent OpenStack Conference it was announced that the initiative has been spun out of Rackspace and officially organized as an industry standards organization similar to the Linux Foundation.  What that means is still open to definition; see Andy Oram’s recent blog at OpenStack Foundation requires further definition for more on that.  However, it is a positive sign that the OpenStack community is serious about keeping the technology available to any and all who want to create software and tools that support real cross platform IaaS cloud integration.

For a project that is just over a year old, there is already plenty of exciting work to show for the effort.  Following the Diablo release, the Essex Release Developers Summit held in Boston October 3-7, 2011, was where 250 hardcore developers, product managers and movers and shakers in the OpenStack community met to brainstorm the roadmap for the Essex release scheduled for April 2012.  There are plenty of opportunities to create the architecture and the tools to make OpenStack enterprise ready, but it will not be an easy or trivial task to do it.  There are already some shiny new companies who have the attention of the venture community.  Piston Cloud Computing, founded by several luminaries from the OpenStack Project, including former NASA Nebula Chief Technical Architect Joshua McKenty, and former Rackspace technologist Christopher MacGown, was launched with $4.5M in funding with no customers or product.  While these folks have solid credentials, can we say there might be more than a hint of another Dotcom bubble here?

From the technology perspective, there is considerably less hype and more meat to the project.  Led by Rackspace, with considerable support from plenty of others, the development of the two main OpenStack components, the compute engine code named Nova and the object store code named Swift are moving along quickly.  Integration of the Keystone authentication engine, the Glance image library management tool and other necessary elements, such as a more professional GUI Dashboard are planned for the upcoming release.  There seems to be general agreement in the community that Essex development will primarily be focused on making the components robust enough for enterprise production environments.

Speaking of production deployments, sitting in on the OpenStack deployment panel at the conference was interesting.  Yes, there are already some OpenStack deployments in full production, most notably at Rackspace.  For companies without a deep bench of very senior Linux systems administrators and DevOPS engineers who are happy mucking with syslogs to draw on, OpenStack does not yet have the tools and ecosystem in place to support a deployment managed by mere mortals.  That does not mean that companies should not be looking at it seriously now, because the tools are coming very fast.  There are already offerings from Rightscale, Cloudscaling and StackOPS, to name just a few.  With the solid Swift and Nova components that are already at least partially built, the foundation is already there for entrepreneurs to build the tools needed to make the technology production ready by early next year.  The time to start planning for a production deployment is now.

About the Author

Beth Cohen, Cloud Technology Partners, Inc.  Moving companies’ IT services into the cloud the right way, the first time!

As we discussed in part 1, cloud services can be classified into five broad categories:

• Consumer Grade
• Small Business Grade
• Mid-sized Business Grade
• Enterprise Grade
• Private Cloud

For the consumer and the small business, cloud services are widely available, cheap and fairly robust if you are willing to put up with the limited services and self help support.  But for the enterprise, hoping that Amazon will not accidentally compromise your data, is just not going to be enough to satisfy your auditors or risk analysts.  Part 2 will discuss products that are more suited to the more sophisticated requirements of mid-sized and larger enterprise customers.  Part 3 will cover the cloud architectures and services that appeal to the needs of the large enterprise.

Mid-sized Business Grade – This is the classification that is the hardest to characterize.  Many of the newest entries into the cloud space fall into this category as vendors ramp up products that will appeal to companies that cannot afford the risk of implementing the consumer grade services available to small business customers, but who are unwilling or unable to pay for enterprise grade products or private cloud implementations.  In general the mid-sized enterprise is not going to be satisfied with the minimal customization available to the small business market.  The value that cloud services bring to mid-sized companies is access to applications with features and capabilities long available to enterprise customers at a fraction of the cost.  This market is full of niche products that cater to narrow specialties, such as facilities maintenance and asset management solutions (MaintenanceConnection and WebWork), and more sophisticated products with broader appeal, such ERP (NetSuite).  These is also a growing set of products being marketed as cloud portfolio management tools, such as Cloudswitch, a migration and virtual cloud data center management tool.  Many of these offerings have the capability for greater customization and better support services, which make them attractive to the mid-sized market, but the vendors tend to be relatively small so there is a risk of vendor lock-in and less robust back-end architectures than might appear on the glossy website.  Expect to see the most growth in these types of products as mid-sized enterprise customers become more comfortable with moving their IT applications to the cloud.

About the Author

Beth Cohen, Luth Computer Specialists, Inc.

OK, we get it.  Cloud computing is big – really big.  In the interest of clearing away the massive amounts of vendor hype, here is a simple formula for sorting out the vast array of available — over 3000 at last count — and emerging cloud services.  In general, cloud services can be classified into five broad categories:

• Consumer Grade
• Small Business Grade
• Mid-sized Business Grade
• Enterprise Grade
• Private Cloud

Part 1 will cover consumer and small business grade products.  Part 2 will discuss products that are more suited to the more sophisticated requirements of the mid-sized and larger enterprise customer.

Consumer Grade – This represents by far the largest segment of the currently available services in terms of usage.  Close to 100% of all consumers are using some kind of cloud services, if you count email.  These products can easily be characterized by their extremely low price, little or no support, and limited customization.  They are designed for automated delivery of relatively simple vertical services with broad appeal to the mass market; the fast food of the IT world if you will.  Some examples of consumer grades products are webmail (Gmail, Hotmail), on-line backup (Mozy, Carbonite), photo sharing ( Flickr, iStock), and some web-based office productivity products (Google Docs, Office Live).

Small Business Grade – This category is growing by leaps and bounds.  However, the quality of the services, support and price vary wildly.  Again these services are pitched as low price alternatives, but they are focused on the business market rather than the consumer.   Many of the thousands of offerings are SaaS enabled applications that were formerly sold as in-house standalone server based systems that are being converted to cloud offerings to capitalize on the attractive subscription pricing model.  While some available services have broad appeal to any small business, like virtual file servers (Nasuni), database systems (Oracle), web hosting (Rackspace), accounting services (Intuit), payroll (ADP), HR management (Capterra), and sales force automation (Salesforce.som, Sales Metric), many of the products are extremely focused on their particular narrow vertical niche, anything from organizational membership management software (Wild Apricot, NetForum), facilities management applications (Maintenance Connection), to medical billing collections services (Athena Health, AllScripts).  If there is a small business application, then more than likely someone has made it available on the cloud.  While the price is attractive for a small business, the per seat model doesn’t scale well.  The biggest issue with many of these services is the lack of any customization and the mostly self help support.

About the Author

Beth Cohen, Luth Computer Specialists, Inc.