Eric Siebert

Can you afford vSphere?

vSphere is out, and it contains lots of new features and functionality. But can companies afford to upgrade right now?

vSphere is a great release — if your hardware is supported and you have the money you may need to pay for additional licensing and training. If you do, then by all means upgrade and check out all the new features and functionality that is has to offer. If you don’t, consider these issues:

Continued »

Virtualization vendor comparisons

There has been a lot of mud slinging and FUD raising among virtualization vendors lately as the quest to rule the virtualization space continues.

One vendor will release information about its product, comparing performance, pricing or features to another vendor, with the other vendor firing back with its own response shortly thereafter. With all this going on, who are you to believe if you are in the market to adopt a virtualization solution in your own environment?

Comparisons by vendors themselves are always biased; after all, they want you to buy their product and not a competitor’s. Performance comparisons between vendors — even by third parties — don’t always tell the big picture and can be difficult to interpret.

Continued »

Adding virtualization to the PCI standard

Earlier this month, I wrote about how the PCI standard was recently updated but still failed to take virtualization into account. Shortly after, VMware announced its participation in the PCI council to help address virtualization within the PCI data security standards. While this is certainly good news and will help tighten up the security standards around electronic credit card payments, the outcome of this announcement remains to be seen. The following are a few improvements that shouldn’t be too difficult to implement right away:

1) First and foremost, the PCI council needs to recognize virtual hosts and include them in the scope of the standard if any of the virtual machines (VMs) that reside on the virtual hosts fall within the boundaries of the standard. Currently, any server, network or device that has anything to do with cardholder data would be included in the standard and any audits that occur.

Additionally, if any virtual machine is included in the scope, then all of the virtual machines on a host should be considered in the purview of the standard because they all reside on the same physical server. Finally, as virtualization allows for VMs to be easily moved between host servers for failure recovery and load balancing, all of the virtual hosts in a cluster should be included within the boundaries of the standard as well.

2) Clarify the confusing item (2.2.1) that dictates that you can only implement one primary function per server. All they have to do is exclude virtual hosts from this item.

3) Most of the security items that are listed in the standard can be applied to virtual hosts as well. This includes things like audit logging, password policies and applying vendor patches.

4) Address virtual networking. Ensure that the security settings on virtual switches do not allow things like promiscuous code, forged transmits and MAC address spoofing.

By simply addressing these four areas, the Payment Card Industry (PCI) standard would be moving in a better direction. From there the council could delve deeper and address other specific areas on virtual hosts using some of the existing security guidelines. Another distinction it should make is between bare-metal and hosted virtualization products. Hosted virtualization products are typically less secure because the underlying operating system is not optimized for virtualization. As a result, they should be subject to tighter scrutiny and control.

PCI Data Security Standard updated, but still does not address virtualization

Last week I noticed that the Payment Card Industry’s Data Security Standard (PCI-DSS) was recently updated on October 1, 2008, from version 1.1 to 1.2. PCI-DSS is a security standard set forth by a conglomerate of all the major credit card companies and is designed to protect cardholder data. As a result, any company that accepts credit cards is forced to comply with it.

About six months ago I wrote that the PCI-DSS standard did not specifically address virtual environments, and instead only focused on servers and networks that are directly involved with cardholder data. In other words, the specification dictates what must be done to secure a server that may store or process cardholder data, but if that server happened to be a virtual guest the host server would not be considered in the scope of the specification. Subsequently you could secure a virtual guest all you want, but if you do not properly secure the host server you could easily compromise the virtual guest regardless of how it was secured.

I downloaded the summary of changes document that specified all of the changes that were made from version 1.1 and 1.2, anxious to see if they had finally added parameters for virtual host servers. Out of the 14 pages of changes, there was still no mention of virtualization technologies in the specification. Surprised by this, I searched through the whole version 1.2, 72-page specification document for the word virtual and found only one instance of it for virtual private network.

I am puzzled as to why they would continue to ignore virtualization. After all, isn’t just about every company virtualizing in some fashion these days? Are the people that write the specification parameters just ignorant of what virtualization is, and that it has a direct impact on their regulations? Or are they just trusting that we are all securing our virtual hosts properly and there is no need to address them? If that’s the case then they have misplaced a critical amount of trust as I am sure there are a great many virtual environments that are not properly secured. Likewise, ignoring virtualization completely greatly reduces the effectiveness of their efforts to secure environments that deal with cardholder data. It’s essentially fortifying everything within a castle, but leaving the front gate open.

It wouldn’t require a great deal of effort for them to address virtual hosts. A number of security specifications for virtual hosts already exist, such as cisecurity.org’s for VMware’s ESX. Let’s hope that they wise up and address virtualization in their next update of the specification. Until then their efforts to protect cardholders are not complete. I just hope that my credit card data is not lying on a virtual machine somewhere that resides on an insecure host server that is ripe for the picking. After all, why try and hack a single virtual machine when you can instead hack into a whole host and gain access to all the VMs and their data?

Market share: So what?

A recent report from IDC claims that Microsoft’s market share in the virtualization arena grew drastically in the second fiscal quarter of 2008 because of the release of Hyper-V. While the accuracy of the report is questionable, as pointed out by one blogger, it does beg the question: Do customers really care about market share?

One common misconception is that market share makes one product better than another. Although typically the product with the greatest market share is the best product, this isn’t always the case. Just because a product is popular doesn’t mean it’s better than its competitors (take Internet Explorer versus Firefox or Opera as an example). In this specific case, however, VMware does have the better and more popular product. The recent market share increase by Microsoft is due in great part to the excitement generated by Hyper-V’s recent release rather than it being better than VMware ESX.

According to a recent Gartner report, VMware has an 89% market share and is the clear leader in the management/automation, maturity/stability, security and ISV support categories. The one area where it gets low marks is price, which in my opinion is not a big deal because if you look at value instead of price VMware would also get high marks.

Purchasing one product over another simply because of market share is not smart shopping. Someone looking to virtualize should carefully consider all of the available products before choosing one. This includes evaluating them, gathering RFPs, reading product reviews and talking to others who are using the product before finally making an informed decision on which product is best.

Would you buy a particular car brand simply because it was the most popular? Probably not. You would look at features, price, reviews, take a test drive and do whatever else you can to find more information before choosing the car that works best for you.

So is market share important to you and would it influence your decision to choose a virtualization product? Let us know in the comments below.

Free virtualization tools for tough economic times

Many IT departments feel the squeeze from the current economic crisis and have seen their budgets slashed. When times are tough you must get creative, and the best way to do that is to utilize products that won’t cost you a dime. Can’t afford new ESX licenses right now? Why not recycle some of that older hardware with one of the free hypervisors? Or better yet, take one of your big servers that only runs one application and install ESXi so you can run other applications concurrently. Let’s go over some free products that you can download and use in your VMware environment.

Free hypervisors:
VMware Server – Version 2.0 has lots of new features and can be installed on several versions of Windows, Linux and almost any hardware.

VMware ESXi – The entry-level edition of VMware’s enterprise-class hypervisor; the installable version installs bare metal on a variety of supported and unsupported hardware.

VMware Player – A great tool for starting up virtual machines without installing a full hypervisor on your system.

Free appliances:
The VMware appliance marketplace has hundreds of free appliances that span a variety of categories. Appliances range from simple firewalls to enterprise monitoring systems to full-blown Web and database packages (LAMP). You can run these appliances with VMware Player or import them into ESX/Server/Workstation and run them there.

Free management and reporting tools:
Embotics v-Scout – A free, agentless tool for tracking and reporting on virtual machines in VMware VirtualCenter-enabled environments.

Hyper-9 – This soon-to-be-released free search-based reporting tool is a great addition to every administrator’s toolbox. Watch for its release around the end of the year. If you are interested in participating in a beta version of this tool, drop me an email. Not all beta requests will be approved and the company is looking for feedback if you do participate.

RVTools – A handy little tool that displays a multitude of information about your virtual machines.

Solarwinds VM Monitor – A free management tool that monitors ESX hosts and virtual machines.

Snaphunter and Snapalert – Utilities that can report all running snapshots on ESX hosts, including name, size and date. They can also automatically email reports and optionally commit snapshots.

Visio Stencils – Some free Visio stencils from Veeam, VMGuru and the Visio Café to help you document your environment.

VMotion Info – A free utility that gathers system and CPU information from your hosts and puts it in a single overview to check for VMotion compatibility.

VM Explorer - A management tool that eases management, backup and disaster recovery tasks in your VMware ESX Server environment.

MCS StorageView - A utility that displays all of the logical partitions, operating systems, capacity, free space and percent free of all virtual machines on ESX 3.x or Virtual Center 2.x .

ESX HealthCheck - A script that collects configuration information and other data for ESX hosts and generates a report in HTML format.

Free administration tools:
Putty – A must-have utility for every administrator to remotely SSH into their ESX hosts.

Veeam FastSCP – A great SSH file transfer utility application.

WinSCP – Another speedy SSH file transfer utility application.

KS QuickConfig - Designed to reduce the time needed to deploy and configure VMware ESX servers as well as eliminate inconsistencies that can arise with manual operations.

VP Snapper – A free utility that lets you revert to multiple VM snapshots at once rather than one-by-one.

VMware Converter – VMware’s free application that lets you perform physical-to-virtual and virtual-to-virtual operations.

vmCDconnected – A handy utility that scans all virtual machines in your infrastructure and shows if they have a CD connected to any of them. After scanning you can disconnect all of the CDs with a click of a button.

CPU Identification Utility – VMware’s free utility that displays CPU features for VMotion compatibility, EVC and 64-bit VMware support.

VMTS Patch Manager – A great ESX host-patching application for those who don’t have Update Manager.

Free backup utilities:
VISBU - A free backup utility that runs from the Service Console and provides VMDK-level backups of any VM in storage that is accessible by the host.

VM Backup Script – A backup script to perform hot backups of your virtual machines.

Free storage utilities:
Openfiler – A free, open source, browser-based storage appliance that supports NFS and iSCSI. It can be downloaded as an ISO file to install on a server or as a VMware appliance to import to an ESX host. A great way to get more shared disk in your environment by turning physical servers into network-attached storage servers or turning the local disk on your ESX hosts into shared disk when using the appliance.

Xtravirt Virtual SAN – A free solution that turns local disk space on your ESX hosts into shared VMFS volumes to avoid purchasing costly storage area network disk space.

Free security tools:
Tripwire ConfigCheck – A free utility that rapidly assesses the security of VMware ESX 3.0 and 3.5 hypervisor configurations compared to the VMware Infrastructure 3 Security Hardening guidelines.

Configuresoft Compliance Checker - A free tool that provides a real-time compliance check that can analyze multiple VMware ESX host servers at a time. Also provides detailed compliance checks against both the VMware Hardening Guidelines and the CIS benchmarks for ESX.

If you know of any other free tools that you use in your VMware environment, feel free to list them in the comments section of this post.

VMware vs. Hyper-V: Comparing apples to carrots

OK, I had to laugh at this one. A Microsoft blog references an article in which a company that predominantly runs Microsoft applications said that it received a $50,000 quote from VMware to virtualize 16 physical servers to four virtual hosts. It claims that the cost comprised $25,000 in software costs and $25,000 in installation costs. The article also said the company chose Hyper-V instead because it cost only $49 per server. The article didn’t mention anything about hardware costs so presumably the company already had hardware or planned to purchase it separately.

The $50,000 price tag was obviously very high. Most likely the quote was for at least one Enterprise license per server as well as VirtualCenter, which may have come out to $25,000 or so. The company claimed to only have a 10-15% CPU utilization rate on its current servers, so it could have easily gone with only two ESX hosts. However, it is possible that they needed four hosts for Hyper-V.

I have to wonder if the company realized what it would get for $25,000. The VMware option provides very robust and feature-rich Enterprise licenses along with a VirtualCenter management server. Comparing this to Hyper-V is like comparing apples to carrots: They aren’t even in the same family. I also wonder if it thought that all it would need to fork out was $49 per server and thus the whole project was going to cost $196 compared with $50,000. Apparently nobody informed it of the underlying requirement of a Windows Server 2008 license for each Hyper-V server. If the company were instead looking at the recently announced Hyper-V Server 2008, which is free, it missed the fact that ESXi is also free and would not have included licensing costs.

As far as $25,000 in installation costs, that seems extremely high for setting up four ESX hosts and performing physical-to-virtual conversions of existing servers to virtual machines. Without seeing the details of the quote it’s hard to say what the company would have paid for. It apparently had no virtualization experience whatsoever, because if it had, it wouldn’t pay someone to install and configure its servers. Presumably it would still have to pay someone to virtualize its environment on to Hyper-V servers. Unfortunately the article made no mention of those costs.

I have to give the company the benefit of the doubt. Was it merely a victim of someone trying to sell it way more than it needed or did the person who provided the quote not understand the company’s needs? It could have easily gone with ESXi servers for free and paid a reasonable amount to have someone help with the installation. If it wanted more features it could have also gone with one of the ESX Foundation Acceleration kits bundled with VirtualCenter for only $3,600. It’s a shame that the company was quoted such a high price. I know if I saw a price tag like that to virtualize a small environment I would balk at it too. However, while looking at other alternatives I would also ask why the quote was so high and try to understand exactly what the cost entailed. It sounds as if someone were trying to sell the company a bunch of Ferraris when all it really needed was a couple of mini-vans.

So without all the facts all we can do is guess, but this seems to be just another case of comparing apples to carrots in an attempt to exploit the so-called price issue between ESX and Hyper-V that doesn’t exist if you do a fair comparison between the two.

Microsoft – Time to put up or shut up

Having seen a lot of anti-VMware propaganda coming out of the Microsoft marketing machine lately, it strikes me that Microsoft is desperate to do anything to try to catch up and compete with VMware. One example is the VMwareCostsWayTooMuch.com website, which it recently launched in conjunction with passing out $1 chips and flyers at VMworld. What’s next, Microsoft? Late-night TV infomercials on Hyper-V proclaiming its greatness? You might see if George Foreman is available — you could call it the lean, mean, cost-reducing virtualization machine.

Microsoft’s tactics strike me as childish. Instead of trying to mislead people, the company should spend its time and money making a product that can actually compete with VMware. Microsoft tries to push the cost issue without looking at the big picture numbers and the features you get with each product. VMware costs more because you get more with it; you get a proven, mature and feature-rich product with many integration, management and automation components.

Microsoft is way behind in the enterprise virtualization game and has a lot of catching up to do. VMware’s recent announcements at VMworld puts Microsoft even farther back in VMware’s rear-view mirror. Microsoft should be doing everything it can to polish its 1.0 product and add some of the many features and functionality that ESX already has. Good products tend to speak for themselves. Once Microsoft has a product that can stand up to ESX, it won’t be forced to sink to the guerilla marketing level to sell its product. I guess at this point Microsoft has to do everything it can to try and achieve global domination of the virtualization market. Maybe it’s time for VMware to start its own website, along the lines of HyperVLacksFeatures.com — but then again, why sink to Microsoft’s level?

Is a 100% virtualized environment possible?

Organizations that have virtualized their environments often virtualize only a portion of their servers, leaving some servers running on standalone physical hardware. Is a 100% virtualized environment possible? Certainly it is, because almost all workloads can be virtualized, but there are some arguments against completely virtualizing your environment.

I recently wrote about an experience I had with a complete data center power failure. The problems resulted from all the DNS servers being virtualized and until the host servers and storage-area network were online no DNS was available, which made it difficult for anything in the environment to function properly. Having a DNS server and Active Directory domain controller running on a physical server would have been a great benefit in that situation.

Additionally, many organizations are leery of having too many servers virtualized because they want to avoid the risk of a single host outage causing many virtual machines to go down at once. This risk can be partially offset by some of the high availability features that are available in many of the virtualization products. In addition, if a virtual environment relies on a single shared storage device and that device has a major failure, it can take down all the virtual machines that reside on that storage. This risk can also be partially offset by having a well architected SAN environment with multiple switches and host bus adapters so multiple paths to the SAN are available.

Another reason that you may not want to virtualize your whole environment is that many software vendors do not fully support running their applications on virtual machines and subsequently may require you to reproduce a problem on a physical system. Because of this it is a good idea to have a few physical servers running applications that may be effected by these policies. For example, if you have multiple Oracle, SQL or Active Directory servers, consider leaving one or two of them on physical hardware.

Finally, you may consider leaving a few physical servers for applications that have non-virtualization friendly licensing and hardware requirements that can be difficult to virtualize (licensing dongles, fax boards, etc.) or for servers that have extremely high I/O requirements.

So is a 100% virtualized environment possible? Yes it is, but is it advisable? In most cases it is not recommended. The cost savings that are typically seen by implementing virtualization will increase the more an environment is virtualized but you may want to stop at around 90% and leave a few physical server for the reasons that were previously mentioned.

Is VMware’s apology enough?

In the aftermath of the infamous bug in the latest release of VMware ESX, VMware CEO Paul Maritz has released a letter that apologizes for the incident and also explains what went wrong and how they are committed to ensure it never happens again.

For customers who were effected by the widespread problem with ESX 3.5 Update 2 released several weeks ago, is VMware’s apology and promise to improve their processes enough? Or is it going to leave some lingering doubt in the minds of some that may inspire them to look at other virtualization products?

The letter provided an explaination of what what happened:

The issue was caused by a piece of code that was mistakenly left enabled for the final release of Update 2.  This piece of code was left over from the pre-release versions of Update 2 and was designed to ensure that customers are running on the supported generally available version of Update 2.

And why it happened:

I am sure you’re wondering how this could happen.  We failed in two areas:

• Not disabling the code in the final release of Update 2; and
• Not catching it in our quality assurance process.

And finally what they will do to ensure it never happens again:

We are doing everything in our power to make sure this doesn’t happen again. VMware prides itself on the quality and reliability of our products, and this incident has prompted a thorough self-examination of how we create and deliver products to our customers.  We have kicked off a comprehensive, in-depth review of our QA and release processes, and will quickly make the needed changes.

Despite it all, VMware still has a great enterprise product that is robust and mature and is still the virtualization software of choice for most Fortune 500 companies. This incident still could have easily been prevented by following processes when preparing a beta build to become a final build. In addition, their QA processes which are usually designed to ensure a quality product also failed to detect that the time bomb code was still present and active.

Will VMware learn from this incident? Absolutely. Sometimes it takes a big event like this to inspire changes and improvements in a company that may have been set in its ways and wasn’t paying attention to details.

One area that many users were critical of was VMware’s communication on the matter. They were initially slow to issue public communications and proactively contact customers to let them know about the issue. The thread in the VMware Technology Network (VMTN) forums that was started on this issue became the rallying point for many of the users who were experiencing problems as a result of the bug. VMware employees did provide some updates to the thread which let users know they were aware of the bug but did not provide much other information until much later in the day. Another breakdown was that VMware’s knowledgebase that had information on the bug and is often the first place users go to when experiencing a problem becamse so overwhelmed by the number of requests that it was unavailable for over 6 hours.

VMware delivered the fix for the problem fairly quickly as it was available roughly 24 hours after the problem was first reported. Many users were hoping to get it quicker then that, but VMware needed time to package and test the fix before releasing it. VMware also did provide good communication later in the day with detailed updates and emails that were sent to customers.

So is VMware’s apology enough? In my mind it is. Yes, it was an unfortunate incident that caused many customers a good deal of grief but the end result is that VMware responded quickly and effectively and this incident will serve as a lesson that they won’t soon forget and will help make their products and processes stronger going forward.