Posted by: JohnMWillis
chef, controltier, puppet
A few weeks ago I attended Velocity “09″ in San Jose, Ca. One of the sessions used a phrase that I had never seen before and it stung me like a bee. In fact, in my opinion, this new phrase described one of the more dominant themes of the conference. These sometimes called “Internet 10″ companies had figured something out that the enterprise has not been able to figure out for over 30 years. They understand that managing your infrastructure is as important as managing your applications. Fortune 5000 enterprise have always given lip service to this concept. However, they purchase tools first, thinking that is all they need in order to say that their infrastructure is important. They use monitors and event managers that give them a warm and fuzzy feeling that they are doing all the right stuff. On the configuration and provisioning side, they use large monolithic distribution systems to provide software distribution and sometimes configuration. In the enterprise, they also raise their swords called ITIL and COBIT to protect their “as-important-as” piece of mind. This false sense of confidence always reminds me of the CEO who is confident that all employees are treated equal because they put silly motivational posters on all the walls. Meanwhile, his parking spot is the closest one to the door.
At Velocity, companies like Flickr, Twitter, Google, and Myspace were making this subtle point about how their gods were not found in the tools they used; moreover, they were the process they used. They understood something the enterprise has never quite grasped. That is, if the infrastructure is important in the business then why not treat it as such. Put your process where your mouth is and not the money you spend on your tools. Theses companies at Velocity understood that the code that manages your infrastructure is as important as the code that runs your applications. In fact, the Velocity presenters made the point time after time that the infrastructure “code” and the application “code” need to be treated as equal. To that end, some of the these companies used shared version control of application code and infrastructure code within the same tool. You don’t see that in the enterprise, folks! To quote my good friend @littleidea, “WTF?”.
What is infrastructure code and how would you put it in a version control system? Yeah, yeah, sure, sure, infrastructure is all those pesky objects that “Bob” the sysadmin understands, but know one else does. Yes Virginia, these objects are the glue that make our infrastructures work. It is this meta-nursha that is needed to deploy, manage, and and configure our infrastructures. In the enterprise they have infrastructure code too. NOT! . . . they have scripts, and tons of them . . . perl scripts, shell scripts, proprietary macro configuration languages that look like scripts. All of this “wannabe” metadata scattered across all sorts of buildings and geographies. Some items are embedded in products provided by Tivoli, Microsoft, BMC, and HP. Others are hidden in workload manager tools. Some are managed by the operating system scheduler (e.g., CRON). And last but not least, in Bob’s special directories on some not so well-documented file server. Oh yeah, and guess what, virtualized environments are usually managed by another completely different team.
Infrastructure as Code means managing all the things that appear just after the server comes up. It also means putting a process in place that let’s you better manage all these items. The funny thing is that Luke Kaines of Reductive Labs has been preparing me for the acceptance of this concept of “Infrastructure as Code” for over two years now. When I heard the phrase “Infrastructure as Code” at Velocity, I knew exactly what they were talking about. Luke has preached this concept in a couple of my Cloud Cafe podcasts as well as with Micheal Cote and I in our IT Management podcast series. The defining of the various system configuration files (user ids, mount points, services, etc.) as objects allows the organization to better manage their infrastructure resources. Reusable objects that can be referenced as code provide an enterprise with an object oriented model for managing their infrastructure. There is a beautiful analogy here. In the early 1980′s we switched our applications from a functional paradigm to an object oriented model. We still haven’t done this with our infrastructure yet. Are you starting to get my point?
At the Velocity conference, presentation after presentation pointed out the useful tools that companies are using to implement this “process-first” model. Puppet and Chef are clearly dominate figures in this new IT renaissance. However, they are clearly just the tools and not the process. You would hear things like – “Oh yeah, we use puppet along with things like capastrano and nanite.” In fact, one of the vendors at the Velocity conference, Controltier, had a nice poster describing this whole new stack when it comes to the concept of Infrastructure as Code. They look at the stack as a three layer model. The lowest layer being the virtual/cloud image or the bare metal. The second being systems configuration layer with tools like Puppet, Chef, and cfengine. Lastly, a third stack describing the application service deployment layer. Coincidentally, this is their specialty layer. Controltier manages the application lifecycle for large enterprise Java applications.
Describing a new concept is always difficult and lends itself to confusion (try to Google – “What is a Cloud?”). Infrastructure as Code might not be the best way to label this new concept. Quoting the brilliant Andres Shafer of Reductive Labs from an ongoing argument that I am having with him on this subject:
Care less about the labels, and more about what it enables. We are moving towards enabling what we don’t have words to describe, so I expect some communication to be clumsy…
Debate or no debate, this is a very exciting time for infrastructure and I look forward to working with some of the key players in this new area.