It’s that time again, OpenStack Summit. The semi-annual gathering of the OpenStack masses to engineer the next set of projects (and “Kilo” release), and marketeers to tell us how OpenStack is real, has lots of Enterprise customers and will overtake the world soon…eventually…or already has.
And if we step back and look at how OpenStack has evolved over just the last four years, it’s been an “interesting” journey.
It started with Rackspace and NASA deciding to collaborate on Compute (Nova) and Storage (Swift) projects. And they would open-source the work. And while open-source projects we by no means new, the fast that this was called “OpenStack” threw lots of people into a tizzy – especially those that sold competitive non-open-source projects.
For at least the first 18 months of public OpenStack existence (there was “secret” stuff happening behind the scenes well before this went public), you couldn’t attend an event or meetup without having to hear the Rackspace/NASA history. And of course people tried to explain this new model, which used commodity hardware was best aligned to these magical applications that handled failures in a new way.
Over time, more programs were added, and opinions varied about how OpenStack would survive in the real world. Should it innovate or clone? Should it create a compatible ecosystem of commodity providers, or work to create unique business opportunities?
And of course there was the question of who should drive the ship. Will OpenStack be driven by a community (opinion, opinion), or should there be a benevolent dictator (eg. Linus Torvalds in LInux)? What occurred was the creation of the OpenStack Foundation, to mute some of the influence that Rackspace had over the programs and bring new levels of transparency and governance into the community. And then all the major vendors jumped on the bandwagon to be sponsors and attempt to influence the project in various ways.
And as 2014 comes to a close, the landscape is far different than it was just a couple years ago. The small companies are being acquired (MetaCoud/Cisco, Cloudscaling/EMC, Inktank/RedHat, eNovance/RedHat) by the large vendors. Some of the early evangelists (Joshua McKenty, Jessie Andrews, Randy Bias, etc.) now work at other companies or are out of the OpenStack ecosystem. And while VC’s are still investing in the space (see Infographic), the exits have mostly been sub-$100M.
Eucalyptus has been acquired by HP, to power their OpenStack strategy. Apache CloudStack still has a strong community, but recent changes at Citrix have led some to question it’s future. The talks of open-source cloud wars have most definitely died down as Microsoft Azure, Google Compute Platform and AWS continue to grow and add new functionality, without being powered by OpenStack. Cisco is pushing the OpenStack for SP’s agenda with their InterCloud strategy, and Mirantis seems to be behind many of the successful projects (in all markets).
People weren’t very happy when I asked if Paris would be the last major OpenStack Summit. While events in Vancouver and Tokyo have been planned for 2015, I still think it’s a valid question in the context of broad community involvement vs. vendor-specific efforts and activities. I believe there is still quite a bit of consolidations to happen in 2015.
OpenStack has come a long way since 2010. We no longer talk about the Rackspace/NASA history and the grandeur of disruptive movements. Now we talk about vendor strategies and if wide-scale deployments will happen. The market has changed and the largest players (both clouds and vendors) are placing heavy bets on the future. OpenStack will be part of those bets, but whether it’s a direct factor or indirect in deciding $$ winners and losers is still TBD.
About six months ago, we decided to switch the focus of The Cloudcast podcast from being about “cloud computing” to being more focused on DevOps, SaaS (the AWS ecosystem) and trends for developers. In particular, the focus on the SaaS ecosystem that enables services around AWS has been very interesting to watch evolve. They have broken up the mindset that Ops needs a “single-pane-of-glass” approach to tools. They allow customers to create the Ops models that works best for them, but creating tons of native API-based integrations with other services.
The consumption model of these SaaS applications is different than you’re used to in traditional IT environments. They charge based on usage, whether it’s in hours or in capacity of data analyzed, thus eliminating huge bills for management software that often become shelfware. And they allow the Ops environment to be flexible and “customized” because most integrate with a massive amount of other 3rd-party SaaS services via APIs (example)
Some companies such as Cloudability (podcast), New Relic, DataDog (podcast), Loggly (podcast), PagerDuty (podcast), Evident (podcast), StackStorm (podcast) and BigPanda (podcast) focus primarily on AWS environment. Other such as CloudPhysics, Platform9, BlueBox take a broader view of the clouds and applications they support. But in every case, they are collecting tons of information about customer usage and gaining insight and experience about building out massively scalable infrastructures. In other words, they are creating learning curves that are orders of magnitude faster than any individual IT/DevOps group could by themselves.
As a customer, being able to take advantage of that learning curve is incredibly powerful. It’s analogous to being able to hire rockstar engineers, except that it doesn’t matter where your business is based or what industry you’re in – since not everyone lives in Silicon Valley or wants to pay those rents/mortgages. You’re renting outstanding talent, and only paying for the software for as long as you want to use it. And when companies like Spanning (recently acquired by EMC) offer the ability to backup data from SaaS applications into your data center or another clouds, you begin to have data recovery or portability if a service goes out of business or you find another one you like better.
To me, the next step is figuring out how to gain insight and knowledge into those learning curves. As I’ve spoken with these SaaS vendors, the feedback has been mixed. Some host events and discussions with select customers to share insight. Others publish learnings to their blogs, or speak about their experience at meetups/events. Still others are taking a page out of AWS’ playbook and turning those customer trends and actions into new features or guidance to customers – such as what Cloudability does to help customers spend less or more intelligently.
As these SaaS services begin to offer links into more cloud environments (Azure, GCE, VMware, OpenStack, etc.). the possibilities to integrate them into your cloud environment will only expand in the near-term future. I believe they are worth exploring, especially if you have challenging areas that are complex to acquire talent, or your current management software isn’t giving you the insight that you need. You can only benefit from the learning curves of these SaaS providers.
At least a couple times a week, colleagues or people within the industry will ask for career advice. What should I do next? Should I work for this company? Where do you think the industry is going next? What’s the next cool technology to learn? I’ve written about this a couple times before. It’s never a one answer fits-all conversation. There are always critical factors to take into consideration – What’s the opportunity? What skills do you have today? What skills are you learning? Where do you live, and does this matter? What’s the next step going to be after this one?
Before I get into the discussion I’ve been having with myself lately, I thought I’d share a story from many years ago. I went to college to study finance and marketing. When I graduated, the jobs in technology were more interesting than cold-calling for stock brokers, so I threw away an education (or so I thought) and jumped into technology. That was scary. I didn’t know the 7-layer OSI model from the Maslow’s Hierarchy of Needs, but I studied like crazy and loved the pace of change and competition. After a couple years of doing sales and consulting, my boss came in my office on a Friday afternoon. He said that I had three choices:  move to Massachusetts for a corporate job (burr…cold!!),  be fired, or  as a long-shot, take a couple engineering classes and be a field engineer installing networking equipment. I had 15 minutes to decide. Sometimes life is funny and complicated. I chose option #3. That was scary. For the next 6 months I flew almost every day of every week, reading manuals on the flights and learning by fire about the technology. It was painful, but I learned how to learn. This was the greatest experience I’ve ever had and I’m grateful to have stumbled into it. It was 20 years ago and I had no planning that it was coming.
Fast forward 20 years and quite a lot have changed. I’ve been lucky to have been able to use that “ability to learn” to transition back and forth between technical, marketing and “other” jobs, across multiple technology companies. During that time, life changed and priorities changed. Learning became easier, but planning became more complicated. 20 years ago, technology transitions happened over 10-20 years. Mainframe to Mini to PC to Web. I now believe that similar transitions happen 2x as fast, taking 5-10 years. The economics and supply chains have been radically impacted by things like Open-Source Software (OSS) and Public Cloud Computing. [Tip: Download a copy of “The New Kingmakers” by Stephen O’Grady from Redmonk to get a better appreciation of that change.] Continued »
While it’s interesting to watch Oracle OpenWorld keynotes, it’s often more interesting to watch the commentary on Twitter from people that are directly or indirectly impacted by Oracle announcements. They have a (sort of) new CTO, a track record of acquisitions and unlimited amounts of cash for M&A, so it’s fun to consider who they might buy next.
Still, no word from Oracle on Docker during the keynotes. Even VMware, which could logically be fearful of containers replacing VMs (or not), mentioned Docker at VMworld. Docker is sort of important to infrastructure teams and sort of important to application teams, and Oracle cares about applications and sort of announced a Platform-as-a-Service (hint: WebLogic), but still no mention of Docker.
But what if CTO Larry Ellison decided that his last chapter was going to be filled with modern acquisitions in order to preserve his legacy and set up his company for the next decade or more? What if Oracle decided to buy Docker? Besides all the initial apocalyptic fury, it would create some very interesting questions and scenarios. Oracle does have a history of buying open-source technologies (directly or indirectly), such as Java (via Sun) and MySQL. Continued »
If you’re into DevOps, you’re probably already familiar with Docker, the container management technology that came from dotCloud – back when it used to be a PaaS company. If you’re more of a traditional IT person, you might have just recently heard of Docker if you follow VMware and saw them discuss it at the recent VMworld 2014 event in San Francisco. If you’re curious about the technology, here’s some places to get started:
NOTE: There is a difference between Docker (the company) and Docker (the technology). The company is a VC-backed entity that created the technology and is responsible for commercializing it in multiple ways. The technology is available via open-source, as well as through commercial offerings – both from Docker and other companies (see the list below). I know that’s a little bit confusing, but welcome to the open-source, open-core, new tech from the Cloud moving to the Enterprise world of modern IT.
Ready to dig in? Here’s some good starting places:
- Docker Weekly Newsletter – https://www.docker.com/subscribe_newsletter/
- CenturyLink Labs Newsletter – tons of good stuff related to Docker (independent) – http://panamax.io/# (see bottom for sign-up)
- Read the Code – as pointed out before, here’s the source code: https://github.com/docker/docker
- Not into Reading the Code – here’s a book by the Docker creators – http://www.dockerbook.com/
- Want to Experiment (online) – https://www.docker.com/tryit/
- Want to Install it (laptop) – Mac, Windows, Linux
- Try it Live: Several Cloud Providers now offer Docker, including Google Cloud Platform, Digital Ocean, Tutum and several others
- Tutorial on Docker 101 – http://slides.com/kennycoleman/introdocker#
- But what about Security? – http://www.slideshare.net/jpetazzo/linux-containers-lxc-docker-and-security
Hanging with the Community
Would rather learn from others? Here’s where to go:
- Find Docker-ized Files & Applications on Docker Hub – centralized repository for Docker images
- Go to DockerCon – it was in June, so plan for 2015
- Watch the Videos from DockerCon or recent meetups – https://www.youtube.com/user/dockerrun
- Attend a Docker Meetup (just about everywhere) – https://www.docker.com/community/meetups/
Using a PaaS Continued »
What a strange, weird and often times confusing week in San Francisco. As always, it’s great to be able to reconnect to such a broad community. But it definitely highlighted that our industry as at a crossroads in many dimensions.
Before the show there was a massive earthquake outside of San Francisco which destroyed a lot of good grapes (and some real estate in Napa). And we had a really interesting Twitter discussion about how geeks like to learn new stuff.
Heading into the week, the excitement level felt somewhat muted. Usually we know about the majority of planned announcements, but there’s always a level of anticipation about those few projects or announcements that have been well-kept secrets.
VMware – The Infrastructure Company
There is a part of the VMware Software-Defined Data Center (SDDC) story that leads to the conclusion that IT will evolve to deliver IT-as-a-Service, which has a bunch of benefits for driving cost-savings and agility for the business. But I’ve noticed an interest nuance in how they tell their story which has me a little bit confused. In order to get to that IT-as-a-Service state, the critical elements are around Standardized Application Services, delivered via a Self-Service Catalog, with automated deployments going on behind the scenes. It’s about the consumption model for end-users and the management model for IT. But in the current SDDC story, that stuff all comes last. It comes after they talk about turning Compute, Storage and Networking into software. Software-Defined Plumbing. This is how infrastructure companies talk – infrastructure, infrastructure, infrastructure – and then sprinkle in some security and management at the end.
Everybody knew about “Project Marvin”, now called EVO:Rail. For a show focused on Software-Defined Enterprise, it was a small set of hardware that seemed to steal the buzz from the show. They also previewed the concept of EVO:Rack, but didn’t provide any timelines, so I’m just going to assume that it’s at least 12 months out (if not more). The technology to seamlessly manage multi-rack scale is very complex and not immediately visible within the VMware portfolio today
Some people seemed to get hung-up on the terminology between Converged Infrastructure and Hyper-Converged. I think it might be simpler to call this new trend 2nd-generation Converged Infrastructure (1st-gen being products like VCE Vblock, and reference architectures like FlexPod and VSPEX) . At the end of the day, it’s still primarily technology that you could buy before, with some pre-built installation scripts. It’s slightly improved in terms of having an element manager that looks at the entire system, but the previous device-level tools are still available. And those tools aren’t cheap or simple, so there are still areas for improvement. Continued »
It dawned on me recently that I’ve been part of this crazy IT community for 20 years now. My first job was in sales for a small reseller. When they asked me if I knew the 7-layer OSI model, I confidently said “yes” and later realized that I had confused that with Maslow’s Hierarchy of Needs, which I learned about in some psychology course. Stack’s a stack, right? Since then I’ve had the opportunity to manage a consulting team, be a systems engineer, handle support calls, be a product manager, be a marketeer, work in corporate M&A, start a few small “companies” and a bunch of other fun stuff. I’ve been lucky that people and companies have let me bounce around and explore different interests.
And after 20 years, sometimes I feel like I’ve been doing this forever. People in our industry like to joke that an “Internet Year” is like 7 years in the real world, so I suppose it makes sense that I feel like I’ve been doing this forever. And then I look at how old my kids are, and my mortgage, and my driver’s license and realize that AARP doesn’t kick in for a while and “retirement” is at least ANOTHER 20 years away. Crap! Now what?
I’ve written before that I tend to have a bunch of conversations with colleagues that fall into the 35-45 yrs age-range. Even did a podcast about it. Everybody is watching the crazy pace of change in the IT industry and they are trying to figure out what to do next, what to learn next, where is the path forward.
I always try and stress two things to anyone that asks me for guidance/advice on those questions:
- Whatever you decide to do, make sure it’s something moving you towards next steps and bigger goals, as you have 10-15-25 more years left to work – unless you have a rich relative or are really awesome at winning the lottery.
- Expect that you’re probably going to need to gain the experience you need for that next role without anyone paying you to do it (eg. learning for “free”)
The other advice I’d probably give someone today is that we’re nearing the end of a long-run of how the IT industry has been modeled. The model of vendor > distribution > channel/SI > customer, now has serious competition from public Cloud Computing and various forms of Open-Source. OPEX is replacing CAPEX and Software is replacing Hardware. Knowing how to write code will be important no matter where you end up. There is going to be quite a bit of chaos over the next 5 years, and then some new equilibrium will most likely shake out. Go read Simon Wardley’s blog if you want some more guidance on a model for mapping out the future – just beware, he’s super smart and your head will probably hurt after reading a few articles. Continued »
I saw the image for Pizza-as-a-Service floating around from many of my colleagues over the last few weeks. It came from an article written by Albert Barron (IBM). And while I would love to live in a world where Pizza was the only food available, I have a couple issues with this article. Actually, I don’t have any issues with the main premise of the article, which appears to be educational to help explain the difference between the concepts of Infrastructure-as-a-a-Service (IaaS), Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS). Where I have some issues is how our industry tends to take analogies like this and stop at the *-as-a-Service definitions.
My issues tend to fall into three categories:
- These definitions are quickly blurring. For example, how do you classify something like Relational Database Service (RDS) from Amazon Web Services (AWS)? It delivers a Database via IaaS infrastructure, but several Database Admin (DBA) functions are automated behind the scenes, like a PaaS.
- Since many companies sell a mix of on-premises and off-premises capabilities, it’s somewhat misleading to claim “vendor manages”, especially if this is for a Private Cloud environment. The vendor may be creating the automation to enable certain functions as-a-Service, but they aren’t actually operating them for the customer (unless this is an outsourcing or SaaS-delivered service). So while the vendor may have helped you automate the provisioning, backup and protection of a Database, this isn’t an apples-to-apples Database-as-a-Service with a public cloud offering. I’ve seen many customers misunderstand that language and set the wrong expectations for their IT operations or developers.
- Even if people grasp the differences in definitions, they often don’t grasp the differences in real-world use-cases. For example, I find many people that don’t understand what happens to an application once it starts to leverage “services” from a cloud. I’ll go back to the IaaS + Database-as-a-Service example. If a company leverages those services, which make a lot of sense if they don’t want to focus on DBA tasks, then they are going to have to figure out how to replace those DBA-as-a-Service capabilities if they ever choose to move that application to another Cloud provider. I suspect that this is because infrastructure people understand things up to the VM-level, and application people understand stuff above the VM (or container). Either way, it’s always interesting to see people want to make a claim about “lock-in” when they really don’t understand what locks a customer or a specific application into a specific cloud. It’s not always just the cloud provider.
Sometimes I’m a complete blowhard, and other times my soapboxes start to materialize in the marketplace. I’ve been on my soapbox about Cloud Management as-a-Service a couple times in the past year (here, here). My premise goes something like this:
- Cloud management software is either overly expensive to buy upfront, or often too complex to install and operate.
- The number of people that have the skills to setup and operate a Cloud environment is still fairly small
- The companies that run Public Cloud environments for many customers have a learning curve that will grow exponentially faster than any set of individual cloud on-premises cloud deployments.
- Unless you’re a Public Cloud provider and have built your own intellectual property (eg. Cloud Management software), there really isn’t any differentiation that an IT organization gains from going through the effort of learning to install and operate their own Cloud Management infrastructure. If it works, it efficiently spins up resources for application teams. If it fails, then life is pretty miserable for everyone involved.
And when asked about use-cases, I typically point to three initial starting points:
- Customer that wants to allow their developers to access Public Cloud resources, but they would like to continue to have some amount of visibility (and potentially policy-based control) of that usage. Sometime this is called “Cloud Brokering”.
- Customer that wants to own the underlying infrastructure (in their Data Center or at a CoLo facility), but doesn’t want to to operate the underlying Cloud Management infrastructure. Customer would like Cloud Management to be an OPEX model, and pay for it as they evolve their IT operations to use the new system.
- Customer that wants to build a Hybrid Cloud environment, combing #1 and #2 into a single system.
The greatest trick the devil ever pulled was making mainstream media believe that everyone in IT was an experienced Linux SysAdmin.
Or so I thought as I traveled to Portland, Oregon for OSCON 2014 (all content and videos included). I wasn’t exactly sure what to expect, although I’ve been to several open-source events in the past (LinuxCon, OpenStack Summit, dozens of local meetups for Docker, DevOps, etc.). I went because I wanted to be uncomfortable and learn some new things. I went because I wanted to meet people in new segments of the community. I went because I wanted to learn how open-source communities act and interact, from the perspective of developers, community leaders, evangelists, operations, and all the groups somehow involved. I went because I wanted to get some hands-on experience and ask questions.
Stuff I confirmed:
- Even if you’re attending the event, it doesn’t automatically mean you’re a Linux guru. Lots of people still trying to figure it out.
- Everything is on GitHub. It’s there, or it doesn’t really exist to this community.
- DevOps skills are in high demand
- Docker is the new hotness, or at least the new “gotta see how much world hunger this solves”
- No two developers or admins use the same tools.
- There are lots of companies doing the cool stuff, they just hide it behind how you interact with them on the web/mobile.
- Not all companies are comfortable with using FOSS software, but many of them keep exploring because of the possibilities.
I took a tutorial on Git and GitHub. Even with the small examples that we worked on, with approximately 200 people collaborating on the same project, I’m still trying to wrap my brain around how some sites and applications get updated so frequently without issues (then I saw this picture and I don’t feel so bad. Fork code bases, update, merge, pull requests, reviews, accepts, push code. The notes from the tutorial were shared on this Google Doc, as well as some Q&A. Most of the tutorial can be done in your own time, content here. LinkedIn might be what people consider their “formal” resume/CV, but GitHub is becoming the place where people go to see what you’ve actually accomplished.
I took a tutorial on Docker (slides here; here’s another good set of tutorials), because you couldn’t walk 5 feet without running into someone doing something interesting with Docker (see: Kubernetes, Mesos, Cloud Foundry, Clocker, etc.). The layering, nesting and portability elements of Docker are very interesting, and community support and associated projects are very strong (Cloudcast podcasts on Docker – Eps.97, Eps.139, Eps.143, Eps.150). Some people associate Docker with virtualization, like VMware or KVM. Other people associate Docker with application virtualization. Others consider it to be an IaaS or PaaS replacement. Many of the Docker experts will tell you that each of those is partially right, and partially wrong.
I spent a morning in Simon Wardley’s strategy and value-chain mapping course, “Playing Chess with Companies“. It was an excellent opportunity to understand and practice the concept of value-chain mapping (eventually there will be an app for WardleyMaps) and how it can be used to not only analyze existing strategy, but also view competitors and attempt to predict how certain actions could impact the competitive landscape. 3-4hrs isn’t enough time to grasp all the concepts, so I suggest spending some time on Simon’s blog to get more details and examples of how it applies in real life. In particular, spend some time trying to understand the evolutionary model of concepts > products > commodity > utility. Also spend some time understanding how the evolution not only creates disruption to existing actors, but also potentially opens new opportunities for other actors.
I spent time listening to how Facebook rolls out updates. Their design is modular, so updates only impact small areas of what customers see for any given change. But they measure the updates meticulously, and apply them to small sample sizes, to determine if the update accomplished it’s goal (faster response times, longer viewing times, usage metrics, reduced CPU usage, etc.).
I sat through a session which described how Instragram builds their web and mobile apps, using a team of 5-6 people. I got exposed to the React Mobile development framework and the types of tradeoffs being made between Traditional Server-Side, Rendered AJAX and Single-Page applications (bandwidth, how much code executes, static content vs dynamic content, etc.)
I was exposed to some new concepts – Kubernetes (which runs Google Compute Platform) and Mesos. These can be looked at as separate concepts and projects, or linked together to help scale and model micro-service based architectures.
I saw a mix of companies/groups/organizations highlighting on-premises offerings (Ceph, MesosSphere, Docker, etc.) vs. companies/groups/organizations highlighting offerings that ran as SaaS applications (Auth0, SauceLabs, Mashery, GitHub, etc.). This was different than AWS Summit a few weeks ago.
I picked the brains of community and developer evangelists to learn how they run successful meetups and hackathons. Lots of little tips that I hadn’t expected, and that I plan to incorporate into future events.
The OSCON community is generally very friendly. Probably because it doesn’t view itself as overly competitive, but rather trying to solve complex problems and focused on building things. They let the business’ business model worry about competition. While not always the case, the technology discussions are collaborative. The problems they focus on are modern web-centric problems (mobile, social, big data, etc.) and there isn’t talk of legacy integrations. There is demonstration vs. presentation. There’s a sense of shared ownership between builders and users. It’s an event that draws people focused on the next 5-10yrs worth of potential change. And Portland is a great place to host the event. It embraces weirdness, which is sort of how new ideas are often viewed. Everything is within walking distance. Nobody is in a rush. There are dozens of brew pubs nearby that let you sit and have engaging conversations.
It was an uncomfortable week, in a good way. I learned a lot. I had to turn off some old ways of thinking to hear how new thinking was taking place. Problem solving in ways that didn’t worry about certain limitations or constraints. I was one of the “normals”, which meant I was an outsider. That was OK, I knew that going in. I’m trying to reset my future compass. There was a sense that what’s discussed at OSCON is the emerging north.