In this video, senior site editor Jan Stafford discusses the fact that KVM, Xen and other virtualization platforms are not interoperable, a subject that troubles users and was given short shrift by vendors at LinuxWorld/Next Generation Data Center Conference 2007 sessions.
Xen expert Bernard Golden sounds off on Microsoft’s presentation at LinuxWorld 2007 and the impact of Novell’s new Linux SUSE and virtualization products. Golden is a systems integrator, SearchServerVirtualization.com expert and author of the upcoming book, “Virtualization for Dummies”.
XenSource recently announced a partnership with Symantec that paves the way for Veritas Storage Foundation to be embedded in XenEnterprise 4.0, expected to ship Q307. Note that the OEM includes a fully licensed, unrestricted version of Storage Foundation. The majority of enterprises today rely on Veritas backup and storage management tools, so it makes perfect sense that XenSource would partner with Symantec to build out a more robust storage architecture for XenEnterprise virtualization platforms. By embedding Storage Foundation in XenEnterprise, storage resources will be able to be managed transparent to their dependent VMs. So XenEnterprise will support connecting VMs to disparate storage targets (FC, iSCSI, NAS, etc.), multipath, and relocation to storage resources as needed, without impacting VM availability.
If you’re already a Veritas shop, this announcement should come as significant news. As a result of the XenSource – Symantec partnership, organizations using Veritas Storage Foundation will be able to manage XenEnterprise storage resources using their existing management toolsets. Furthermore, the partnership is also going to result in certified NetBackup solutions for XenSource platforms. Many backup vendors are still sorting out their VMware backup solution set, while Symantec is steaming ahead by adding XenSource to its already supported VMware and Microsoft virtualization backup solutions. There’s a big difference between a “we support VMware and Xen backup” marketing check box, and a robust and well documented solution set for virtual machine data protection and recovery. Symantec clearly gets it. For example, NetBackup 6.5 is the first backup platform to support recovering VM images or individual files from a single VMware Consolidated Backup (VCB) job.
The OEM agreement may also impact organizations that are required to certify their storage management solutions with every new version release. By using a single storage management infrastructure for both server and virtualization platforms, re-certification of storage management following virtualization platform updates will be easier than on virtualization platforms using a proprietary storage management architecture.
Storage management, high availability, and backup support have been three key issues that have stalled XenSource’s assault on the enterprise. All three of these issues will be solved in XenEnterprise 4.0 as a result of the XenSource – Symantec partnership. With Storage Foundation embedded in XenEnterprise, organizations that do not run Symantec (Veritas) software will still be able to take advantage of the new storage features and manage them using their XenEnterprise management tools. High availability and dynamic VM failover will be included as well. Inclusion of high availability into their virtualization architecture will place XenSource in the high availability virtualization club that now includes VMware, Microsoft, Virtual Iron, Novell, and Red Hat.
When virtualizing mission critical systems, I have long viewed high availability and certified backup support as requirements, and have recommended that virtualization platforms devoid of these features remain relegated to training, test, and development environments. With the upcoming release of XenEnterprise 4.0, XenSource appears to be on the verge of crossing the chasm to join the enterprise virtualization elites such as VMware.
Senior Analyst, Burton Group
Note: This post also appears on the Burton Group Data Center Strategies blog.
Software engineers Jeremy Fitzhardinge of XenSource Inc. and Avi Kivity of Qumranet Technologies laid out the pros and cons of Xen and KVM in a LinuxWorld 2007 session, titled “Xen and KVM: Separating fact from fiction” just a few minutes ago. Panelists also included moderator Sunil Saxena and Jun Nakajima of Intel Corp., but I’m focusing on Fitzhardinge’s and Kivity’s takes on the facts about both virtualization technologies.
Let’s start with KVM.
“KVM is based on Linux which is the most scalable operating system on the planet. It shows very good scalability in terms of guests. We expect to be able to run very large guests with great scalability soon,” Kivity said.
KVM tries to leverage hardware virtualization and Linux kernel capabilities to the maximum, Kivity said. KVM leverages Linux memory management and I/O capabilities and its scheduler and security model. Real-time scheduling is made possible with KVM. It’s also easy to use existing management tools with KVM.
“There is work being done to get hardware assistance in KVM,” Avity said. In the meantime, KVM requires hardware assistance. Other cons include a lack of flexibility in device assignment and needs work in memory and power management.
Xen, on the other hand, can run only any platform without hardware support. However, without hardware support, performance can suffer, said Fitzhardinge. In most situations, hardware support is available, so this isn’t a big deal. Xen does need to draw from Linux support for power management and platform initialization.
“Xen has minimal support for power management at this point,” Fitzhardinge said.
On the plus side, Xen has great security isolation, is operating system independent and supports multiple hardware platforms. Xen is designed around paravirtualization, which gives it low overhead and good performance, Fitzhardinge said. Also, Xen supports both paravirtualization and hardware-assisted virtualization.
Unfortunately,KVM lacks manageability features that exist in Xen today, platform initialization with Xen is complex and performance on 64-bit processors isn’t great, Fitzhardinge said.
So, overall, scalability in number for guests and VCPUs is good on both Xen and KVM; but KVM supports power management and Xen does not. KVM has broad machine support, but Xen does not. Both have Linux vendor and community support.
Looking ahead, Xen and KVM developers plan to add enhanced backward compatibility and share I/O device support. Interoperability is also a concern that both development groups will work on. On the full virtualization side, machines images installed on one hypervisor will be obtainable on the other, Kivity said. To what extent the hypervisor will present the same hypervisor interface is the question. At the moment the Xen interface is quite different from the KVM interface, said Fitzhardinge. As Xen goes towards more hardware support, there are more possibilities for developing interoperable interfaces.
LinuxWorld vendor news
SWsoft is previewing Virtuozzo 4.0 at LinuxWorld. The new version, slated for release later this year, has a new, customizable interface and includes additional management tools (management tools trend, anyone?).
Virtuozzo 4.0 can perform cross-platform backups (i.e. store a Windows virtual server backup on a Linux server and vice versa). You’ll also see a full SDK for application integration, template improvements, and even tighter integration with Plesk.
LinuxWorld vendor news
The cry for better virtualization management tools has not gone unheard — at least not by Novell. Today at LinuxWorld, Novell announced a new release of its data center management solution: ZENworks Orchestrator 1.1. Not only does it improve management for a data center that incorporates virtual machines, it manages (or “orchestrates,” if you will.) both the physical and the virtual parts of the data center, doing so by overseeing a collection of management tools.
According to Novell’s press release, the 1.1 version should make implementation easier, and give users the ability to pick and choose which management tools are installed onto their systems.
Orchestrator handles resource management, job management, dynamic provisioning, policy management, accounting and auditing, and real-time availability.
The 1.1 version features a new interface and full lifecycle management. The orchestration engine allocates overall data center resources to be installed and run separately from specialized management components, such as virtual machine management.
Full management for SUSE Linux Enterprise from Novell running Xen virtualization is available with the new 1.1 version.
For more information, visit Novell’s ZENworks Orchestrator Web site.
This is my attempt at putting together something that every sysadmin and supervisor should have when they look at whether to take the next step in virtualization – a Common Sense (tm, patent-pending, sm, r) guide to putting virtualization in place. Truth be told, it works with any server virtualization product, including VMware’s VI3, ESX2.x, and Server, Virtual Iron 3.x, Xen 3.x, etc., but as I’m more familiar with VMware’s product line, that’s my default reference. I’m also taking some creative liberties and coining a new phrase – servirt, short for server virtualization. Lets see if it catches on (no, I don’t really expect it too – it just doesn’t sound as cool as Bennifer or Brangelina).
Rule #1: Don’t put your firewall in your servirt environment.
This rule, along with any future “don’t put your X in your servirt environment” rules, is geared mostly towards security. If your host system is compromised, it’s not a far step before your guest systems are compromised. Given a proper topology, a compromised set of virtual machines is no more dangerous than a compromised set of standard machines, but throw in an outward facing device and you have all the makings of the next NY Times poster child for data-protection reform. It is only a matter of time before serious servirt malware is designed to sneak into guests on an infected machine through the hypervisor (so called Guest-to-Guest, G2G, or GtG attacks). Imagine this situation – a host running Exchange, Oracle, Siebel CRM, ISA, and Sharepoint as guests. The host is infected, finds the app servers, infects them with a data collector, then finds the ISA server, infects it, and uses it as a giant gateway to stream all of your customer records to www.iownzyercreditzfilezd00d.info over http. All without hopping a physical box. For those who say I’m full of it, that such vulnerabilities are impossible or way off, all things are possible.
Rule #2 – Use your virtual switches wisely
It’s tempting to put everything on one virtual switch and then let the host handle the load. This doesn’t often get covered in a lot of howto documents, but it’s important. It’s particularly important in VMware VI3, because of the major improvements VI3 brings to virtual switching and the advantages present in those changes. Honestly, I think VMware should have made a bigger deal in marketing the switching improvements, but I’m not a marketing guru by ANY stretch of the imagination. With the recent Cisco/VMware rumors, the switching in VI3 may get the credit that it’s due. Anyway… Treat virtual switches as you would physical switches – create network segments with care and planning, vlan when necessary (if your product supports it), and make sure that if you have a lot of host servers, that your virtual networks align with the hosts. Packet storms should not take out your entire virtual environment!
Rule #3 – Don’t mess with system requirements. Ever.
Not on the guests. Not on the hosts. Not on the management boxes. Not ever. Very often it can be tempting to put less memory into a guest machine than optimally necessary in order to conserve limited physical memory on the host. Sometimes it’s even tempting to save a thousand dollars on a server (particularly if the spec you optimally need is a thousand dollars per server over your allocated budget) by cutting out memory, dropping the CPU down to a slightly slower model, using lower rpm hard drives, etc. DON’T DO THIS! It may be fine, but it can also come back to haunt you. In the guest-scenario, it may seem easy to say “If I hit a problem, I’ll just up the guest’s RAM”, but it’s a lot tougher saying that when the physical machine is maxed-out and is full of other in-production guests using up all that RAM. I’ve done this. It sucked. I had to take down a host server that impacted five departments, including finance, because I tried to squeak through without spending the dollars at a time when I was under a serious budget crunch. Oh, and the guest server in question that needed more RAM – a Jabber -based internal Instant Messaging server. Not exactly mission-critical, but it had a high profile because it was very visible to the entire company every time it mem-locked and dumped out. Lesson learned.
Rule #4 – There is NEVER a rule #4.
This is from an old USENET post, and I can’t find the reference to link it to. It was funny back in the day, and I’ve kept it up since.
Rule #5 -Use the freebies!
XenSource XenExpress, Xen itself, VMware Player, VMware Server, Virtual Iron Single Server Edition, and a host of other similar applications are free to try, and free to use. Some require host OS licenses that aren’t free (ahem, Microsoft, that’s you guys…) but most will run on free OSes like Linux and/or FreeBSD. I have a whole lab set up with Virtual Server on CentOS Linux, and it works great. We use VMware player to distribute some legacy applications that don’t play well on XP and/or Vista. Also, don’t forget about the free P2V tools out there. VMware’s free P2V converter is great – almost as powerful in the P2V side as enterprise products like Platespin’s PowerConvert. While we wait on new hardware to test Virtual Iron, we’re using a great freebie tool, that we found here to get a jumpstart and convert some of our testlab vmware machines to Microsoft’s VHD format, which we will then import into Virtual Iron. Before we even decided to do virtualization (ok, after we decided virtualization fit the business/finanacial/techincal needs of the company, but before we commited to it) we used demo versions of VMware as a proof of concept. The point is, there’s not a stage in your servirt environment’s development that can’t benefit from the judicious application of a little frugality. Except when it comes to system specs (see rule #3).
Rule #6 – Read the Famous Manuals
And the white papers. And the Wikipedia entries. And the promotional marketing material (if you’re into that kind of pain). In the case of Virtual Iron, read the forums… you might just find the install and admin docs there (yes, that’s a criticism of your website, Virtual Iron) Read whatever you can read on the subject of your servirt environment. For example, when the company I’m with went looking at VI3, I read through a ton of literature and came across an HP document that was immensely valuable. I also found this chart very useful, albeit becoming outdated. When we first embarked on our trip towards virtualization there must have been a gigabyte of material on my hard drive about VMware’s product offerings (at the time, that consisted of ESX and GSX). As we’ve progressed, I’ve accumulated a small library of PDF files, demo software, and links. Small like the New York Public Library is small.
Rule #7 – Don’t put all of your Active Directory domain controllers on the same hosts.
If you do, you’re in for trouble when a host falls over and goes boom. And they do, once in a while, fall over and go boom. Or they may face a G2G attack, in which case your entire AD environment is hosed. If you’re a Novell shop, good for you, but don’t put all your eDirectory servers on one host either. Red Hat Directory Server shop? See above. If you’re using VMware’s VI3, make sure that HA/DRS is configured to prevent all your directory servers from being on the same host, because even if you design it so it won’t happen by laying out the AD controller guests on different hosts, you’re just a slice of probability and a few resource utilization spikes from DRS putting them all on the same srver for you. Me, I leave one AD controller out in non-virtual land just because I can (ok, because I have a spare old server that does nothing else).
Rule #8 – Document Everything
The usual rule of document everything goes here, like it does everywhere. I won’t go into the obvious points, but there are a couple of not-so-obvious points that need to be mentioned. Naming conventions… there’s been some good talk about this, and I won’t repeat it, but remember to name your servers appropriately so when you do a quick network scan you can tell what’s what from the resolved names. Remember, not all management happens in a console, even if you are 100% virtual. What’s supposed to be where… this can change a lot in a well-designed servirt environment because of HA/DRS and similar tools, but document the starting points for all virtual servers and take regular performance metric updates to see what has moved where and why it’s moved.
Rule #9 – Switches and NICs, Switches and NICs, Gonna Get Me Some Switches and NICs.
This is about NICs, really. Lots of NICs. You can never have enough NICs. Fill every slot you can with NICs. Have the available external switch ports to support those lots of NICs. Why? Because some applications will eat your bandwidth like it was Kibble n’ Bits. That means some virtual machines will choke others, given the chance. To get off the 80’s dogfood commercial metaphor and onto a gardening metaphor, bandwidth is like sunlight, and some apps are like pretty weeds. The soak up everything they can, leaving little for others. You can’t kill them, either. If you find you’re in a situation like this, having lots of NICs in your server can make all the difference, because now you can add it to the virtual machine weed you’ve got and essentially transplant it away from the rest of the environment. Some care needs to be taken with HA/DRS, but in that case you need to look more at teaming and aggregating those many NICs and switch ports properly.
Rule #10 – Storage
In some cases, violate rule #5. In our lab, we started with freebie OpenFiler as an iSCSI solution, until it came time to test VMotion. Sometimes it went boom. Other times it was fine. We couldn’t figure out why until we followed rule #6 and found out it was a problem with IET’s (the iSCSI target under OpenFiler) use of SCSI commands vs. VMware’s interpretation. The point being, this is an extension of rule #3… only about storage. Having the right storage environment is crucial… you can have your servirt environment set up 100% perfectly, but if your storage isn’t 100% perfect you’re going to run into all kinds of problems with moving guests around. Since that’s the whole POINT of virtualization’s DR advantage, having a bad storage strategy is essentially having a bad DR strategy. This is to say, anything under 100% on both is an F… because in DR there are no Bs, C, or Ds… just an A+ and an F. For what it’s worth, the problems with OpenFiler and VMware seemed to be fixed at this point, and we’ve gone back to using it in test environments for possible production use once we have 100% confirmation.
Well, that’s it for now… another set of these common sense ideas will probably be forthcoming (maybe after I’ve finished playing with Virtual Iron in the lab and actually get around to posting that long-promised review).
Not to brag, but my recent story, ‘VMware and NetApp in cahoots?‘ has been corroborated this morning by a Network Appliance press release, which states that the two companies are collaborating on joint engineering, marketing, service, and support.
Specifically, NetApp cites development work on advanced application mobility, backup and recovery, and disaster recovery.
The release goes on to say that the two companies “are also targeted at further simplifying disaster recovery, streamlining test and development environments, optimizing storage utilization of virtual desktop consolidations, and ultimately creating a tightly integrated server-to-storage virtualization solution…”
The two companies are also working out an enterprise support agreement for streamlining case management when issues arise in joint VMware/NetApp environments.
NetApp’s fortunes took a turn for the worse in the past week, as the company’s stock plummeted after announcing that it had missed its numbers for the quarter. Not to be too cynical, but could this announcement be an attempt by the reeling storage vendor to associate itself with red-hot, pre-IPO VMware? Not a bad move, if you ask me.
One of the nice things about covering a hot topic like virtualization is that you get to talk to people who are really enthusiastic. Yesterday, Alex Bakman, founder and CEO of a new company called V-Kernel came to my office, and he was as fired up about the virtualization market as any vendor I’ve met in a while.
What V-Kernel has done so far is relatively straightforward. They’ve developed capacity and chargeback software that monitors the CPU, memory, network and disk resources consumed by VMware virtual machines, and maps those resources to a so-called business service – a grouping of VMs that are performing a business function, like “email” or “CRM.” From there, V-Kernel can generate chargeback reports.
Now, the idea of chargeback has always been nice in theory, but in my limited experience, things never really materialized. Bakman has an explanation. In the world of one-server-to-one-app distributed computing that virtualization is supplanting, chargeback was kind of silly, since resources weren’t shared, and it was easy to tell who owned what application. However, if you look at the mainframe – a massively shared resource – chargeback is alive and well. And when you look at today’s high-end x86 platforms, “I don’t care what you call them, they’re mainframes,” he said. As expensive shared resources, Bakman believes IT will move quickly to push back the cost of virtual infrastructure to those departments that are using them.
Virtualization also has the developer and independent software vendor (ISV) in Bakman all excited. Unlike previous systems management software he’s developed, the big thing that differentiates V-Kernel is that it’s delivered as a virtual appliance.
In V-Kernel’s case, that appliance includes a (stripped) Suse Linux operating system, Apache, Tomcat, MySQL, and a Java Ajax application server, all in 420MB that can be “dropped” right on any VMware host. For updates, the appliance includes one-button self-update feature that “calls home,” as it were, for any new security patches or code updates. And because it’s just a virtual machine, “it’s infinitely scalable,” Bakman said. To add more performance, just assign more memory or virtual CPUs to the VM.
The fact that V-Kernel runs Linux is important to data center managers because that’s the same reliable operating system that powers a lot of the other applications in the data center. Plus, writing the application to run on Linux instead of Microsoft sidesteps a lot of licensing fees. “If I were to ship it on a Microsoft OS, there would be licensing fees for every appliance I ship,” he said. Microsoft’s current licensing policies are already conspiring to turn smaller ISVs away developing for the Windows platform, he said.
Furthermore, the advent of Java/Ajax means that developers don’t need to give up any functionality to run over the Web. “Ajax is an absolute Microsoft killer,” Bakman said.
For now, V-Kernel is available only as a VMware virtual appliance, and as Bakman sees the market, there’s no reason to change that – for now. Over time, Bakman’s vision is to add more systems management appliances to V-Kernel’s bag of tricks. In the meantime, you can download a beta at http://www.vkernel.com.
I’m waxing philosophical for a bit here… What is the future of virtualization? What role will it play in the unfolding technological evolution of our society? Is the Technological Singularity really coming, and if it does, how will virtualization be important?
For those eternally-optimistic about our species’ future, there’s a great book that incorporates incredible optimism and the predicted result of the merger between biology and technology called The Singularity is Near: When Humans Transcend Biology, by Ray Kurzweil (the same Ray Kurzweil who first made the electronic piano sound like, well, a piano). I’ve been reading it, and two technologies keep popping into my head to solve some of the problems that crop up when I try and wrap my mind around the many charts and predictions in the book – namely grid computing and virtualization. I highly recommend reading this little gem, although I also recommend taking a lot of his timelines with a grain of salt. More on that later…
It’s my opinion that out of all the many technologies that will push the man-machine merger that Kurzweil so eloquently professes is coming, systems-level virtualization will be the most important, followed closely by grid computing. Many of the calculations RK uses to describe the increasing power of computers are based on Moore’s Law, his own Law of Accelerating Returns, and other mathematical formulae. One item I see in the book is that Kurzweil with doesn’t take into account in the book is the impact of utilization levels, which are far, far below the exponentially increasing numbers he presents in his prediction of future raw computing power. He may have considered this and come up with the same conclusion I have (or another, better one), but it’s not in the text, so I’ll go on with the assumption that its a blank slate. We’re going to have that raw computational power, sure, but we’re not going to make the best use of it (at least until our own intelligence is enhanced by non-biological intelligence, which he predicts in the 2030s time frame) without virtualization and grid technology. Virtualization will do what it was originally marketed for – take that pile of 5-20% utilized resources and merge them together to reach as close to 100% as possible. When it comes to controlling nanofactories (which take component elements and make goods from them – like literally making a car from atoms, or for that matter making a 100% real steak without a cow, or making replacement organs from within your own body), expanding virtual reality into realms an immersive as the real world (or moreso), even expanding our own consciousness directly into nonbiological substrates, there are going to have to be computers, powerful computers, running the show. And not all of these computers will be 100% active all the time. Add in the upcoming pervasive grid computing that will eventually connect all spare computing resources, and you can see the power of virtualization. Virtualized systems on hardware to ensure portability between platforms, to maintain the integrity of a system’s purpose, and to house what is needed where its needed, as well as raw computation resource sharing via a grid will maximize the potential of our raw computing power.
The disaster-recovery-friendliness of virtualization is going to make a huge difference in bringing about the singularity as well. One of RK’s predictions, and indeed his beliefs are clear that he feels that this is our destiny, is that we will overcome our biology and become what others call posthuman (he does not use this term, seeing humanity as consciousness, whatever the underlying origin, and independent of hardware, software, or material). He forsees the gradual transformation of humanity through the increasing use of medical implants (such as are used now to replace limbs and hearts, stop seizures, restore sight, restore hearing, etc.), cosmetic implants (anyone ever seen the Tiger Man?), and eventually nanomedicine – nanoscale sized machines working in conjunction to replace entire biological systems (such as the circulatory, respiratory, digestive, and even nervous). What are we then if not the Ghost in the Shell? And what happens when a shell crashes, as it will? That’s what DR is for, and that’s where virtualization plays such a key part – suppose you have a severe blood disease and in the future you are able to replace your blood with respirocytes (tiny machines that act like blood cells). What happens when there’s a massive systems failure of the OS controlling the little things? Probably not much because that’s an application for a distributed OS on a grid system. What happens down the road when your consciousness has moved, via the natural process of extending your life by replacing failing organs, to a completely computerized substrate, and THEN the underlying hardware fails? That gets trickier. Essentially, you die. Unless of course you happen to have VMotion / LiveMigrate / FutureVirtualizationDRTool’sNameHere, at which point you smoothly slide over into the next bank of hardware and keep on living.
This all assumes you consider computerized consciousness living (I do).
Sound a little far-fetched? It does, doesn’t it. But then again, I’m reminded that if I took many mundane items back a few hundred years in time, I would be worshipped as a god for my ability to cure, kill, and perform feats that defy the “laws” of nature as understood at the time. As Arthur C. Clarke once said – “Any sufficiently advanced technology is indistinguishable from magic.” This isn’t to say I agree with Ray Kurzweil’s timeline – I don’t. He’s a self-professed optimist, and while technology will undoubtedly advance unabated by boom or bust just as it has always done, I think he forgets about the pessimistic, particularly about greed – those who develop the technology to power the singularity first will hoard it, patent it, and sell it for exorbitant sums of money for a very, very long time before competition is able to thrive and prices come down to the point where the panacea to life’s many ills are cured. The price of immortality is immeasurable – and the rich and powerful will pay dearly for it. Dearly enough that it won’t be economical to sell to the masses of ordinary folks for a very, very long time, and for a sadly terrible long time people all over the world will have to endure disease, hunger, and death.
But when it does come about, rest assured that the Technological Singularity will be enabled by virtualization. It may not look the virtualization we know today, much like Mac OS X looks nothing like PWB/UNIX, but it will still have the underlying role as we know it today. So that’s my two cents on virtualization in the future – not much different than it is now, but oh-so-important to what will be.