Virtualization Pro

June 25, 2009  3:35 PM

More details on VMware’s Fault Tolerance feature

Eric Siebert Eric Siebert Profile: Eric Siebert

This week’s VMTN Community Roundtable podcast was about Fault Tolerance (FT). Henry Robinson and Karen Ritter of VMware joined to provide information about the development and future of FT.

Here’s a summary of some interesting details from the podcast, but if you haven’t listened to it yet, I recommend that you check out the recording as it provides a lot of valuable technical information.

  • VMware spent a lot of time working with Intel/AMD to refine their physical processors so VMware could implement its vLockstep technology, which replicates non-deterministic transactions between the processors by reproducing the CPU instructions on the other processor. All data is synchronized so there is no loss of data or transactions between the two systems. In the event of a hardware failure you may have an IP packet retransmitted, but there is no interruption in service or data loss. Continued »

June 23, 2009  3:34 PM

Killing a frozen VM on a vSphere ESX host

Eric Siebert Eric Siebert Profile: Eric Siebert

Occasionally virtual machines (VMs) get stuck in a zombie state and will not respond to a power-off command using the traditional vSphere client power controls. Rebooting a host will fix this condition — but rebooting is usually not an option. Fortunately, there are a few methods for forcing the VM to shut down without rebooting the host.

I previously documented these methods with VMware Infrastructure 3 (VI3) and wanted to make sure they all worked with vSphere. The methods below are listed in order of usage preference starting with using normal VM commands and ending with a brute force method.

Method 1 – Using the vmware-cmd service console command (the command-line interface equivalent of using the vSphere Client) Continued »

June 22, 2009  6:13 PM

The key to successfully virtualized Domino servers

Eric Siebert Eric Siebert Profile: Eric Siebert

Although there are several tech notes that suggest otherwise, most Lotus Domino workloads can be successfully virtualized. I thought I’d review some of the tech notes as they seem to be discouraging people from virtualizing Domino servers. Both of the notes I’ll review blame the virtualization layer as the cause of poor performance, but as Domino servers are resource intensive, the issue could very well have been improper architecture or the versions of Domino and vSphere. If you do virtualize Domino, make sure that the underlying architecture is sufficient and make sure to have the latest versions of Domino and vSphere, as the latest versions provide I/O and performance enhancements.

The first tech note, High CPU usage by router and poor performance of Domino under VMware, is about two years old. Here’s a summary of the problem, cause and solution. Continued »

June 17, 2009  4:55 PM

VMware’s hardware compatibility list

Eric Siebert Eric Siebert Profile: Eric Siebert

VMware publishes a guide called the Hardware Compatibility List (HCL). The HCL lists all of the hardware components that are supported by each version of ESX and ESXi. This very important guide is divided into different sub-guides, which include systems (server make/models), storage devices (SAN/iSCSI/NFS) and I/O devices (NICs/atorage controllers), and is updated frequently with new hardware added and older hardware removed. I was curious about the inner workings of how this guide is maintained, so I contacted VMware for some answers.

First, you might wonder why this guide is important? There are two reasons. The first is that ESX/ESXi has a limited set of hardware device drivers that are installed and loaded into the VMkernel, and while it is possible to install additional unsupported device drivers, it is not recommended. Consequently, if you use a network or storage adapter that is not on the HCL, there is a very good chance that it might not work because the driver for it is not included.

The second reason is that VMware only provides support for server hardware that is listed on the HCL. Just because server hardware is not listed on the HCL doesn’t mean it will not work with ESX/ESXi, however. There is a lot of older hardware and other hardware brands/models that are not listed on the HCL that work just fine but are not supported by VMware. So if you are using hardware that is not listed on the HCL and call VMware’s Global Support Services for assistance with an issue, you might wonder if they will help you at all. What VMware will do is assist customers in problem analysis to determine whether or not the issue is related to the unsupported hardware. If the issue is suspected to be hardware-related, VMware reserves the right to request that the unsupported hardware be removed from the server. If VMware determines that the problem is related to the unsupported hardware, they will request that you open a support request with the hardware vendor instead.

So you might be wondering how the hardware on the guide is selected? Hardware vendors have to test and certify that their hardware works properly with the latest versions of ESX and ESXi. Once this has been completed, VMware will add them to the guide. VMware works with hardware vendors as part of their Technology Alliance Partner (TAP) program, and any vendor can apply to have its hardware added to the HCL. Once an application is received, the vendor is responsible for completing the certification criteria and submitting its results to VMware for review and approval. The first step of this process requires vendors to submit a VMware Compatibility Analysis for the hardware that they intend to certify. After VMware reviews and approves the analysis, the next step is for the vendor to engage with a third-party testing lab (currently VMware works with AppLabs or Cognizant for this) to certify that its hardware works properly with ESX and ESXi. VMware will not disclose the specific testing criteria that is used for certifying hardware for the HCL, but it does use the same certification criteria for all vendors that apply.

You’ll probably notice that the HCL contains mostly newer hardware and that older hardware is periodically removed from it. VMware does not enforce an expiration period for hardware added to the HCL, but it is up to each vendor to certify its hardware for the most current VMware product releases. Vendors are free to initiate the certification process at any time it needs to have new hardware added to the HCL. Additionally, each vendor can choose to remove older hardware from the HCL as it releases newer hardware versions.

So while using hardware not listed on the HCL may be OK for labs, it is highly recommended that you only use hardware on the HCL for production use. Be sure to check the HCL periodically, especially if you plan to upgrade to a newer ESX/ESXi version, as you will want to make sure your hardware is listed before upgrading. You should check this guide before you purchase any server hardware to use with ESX/ESXi. Also, be sure all your server components are listed in the guide, including NICs and storage adapters. Often it may take a short period of time before newer hardware is added to the HCL. If you have newer hardware and it is not yet listed on the HCL, try contacting the vendor to see where it is at with getting its hardware certified by VMware.

The guide format was recently changed and is now searchable via an online form. Additionally, you can download the full guides in PDF format for each hardware component. Make sure you bookmark the guide and periodically check it — you don’t want to find out when you call VMware support that you’re using unsupported hardware.

Special thanks to John Troyer, who got me in touch with the right person at VMware to talk to, and to Nick Fuentes, who found the answers to my questions.

June 11, 2009  3:38 PM

VMware: the new bully on the block?

Bridget Botelho Bridget Botelho Profile: Bridget Botelho

VMware architect and virtualization expert Gabe van Zanten wrote an interesting post on his blog, “Gabe’s Virtual World” pointing out that VMware appears to be following in Microsoft’s footsteps by bullying partners and customers.

“Several stories have emerged that made it look like VMware has learned from Microsoft and is now practicing the same strategy. Maybe Paul Maritz’s tricks for Microsoft are now reused against VMware competitors,” van Zanten wrote. “First, there was a change in their VMworld policy reported by Brian Madden which according to VMware in an official response was just to prevent competitors from trashing VMware like Microsoft did at VMworld 2008. Although that is a viable explanation, the text now is in the legal documents and can be used as VMware pleases.”

Here’s another dirty move; VMware asked Veeam to remove its Backup & Replication support for VMware ESXi free edition in its latest Essentials Bundle, Acceleration Kits for VMware vSphere 4. Alex Barrett also wrote a great story on that, and how this kind of move might push ESXi users – which are mostly small and medium sized businesses – right into Microsoft’s arms.

These tactics raise the question of whether VMware plans to block more ESXi tools, forcing people to upgrade to a paid version of their software. As van Zanten wrote in his blog, “It is obvious they want ESXi Free to be “unmanageable”, since it is difficult to manage an ESXi free host with a read only remote administration kit. But why? Does VMware think that a small company will now switch to VMware’s vSphere Essentials edition just to be able to manage ESXi? Is VMware afraid that customers start building large clusters of ESXi Free hosts and use third party products to manage them?”

These concerns and others have yet to be answered. In the meantime, I’ll leave you with a couple text book characteristics of bullies:

Those who bully have personalities that are authoritarian, combined with a strong need to control or dominate… If aggressive behavior is not challenged in childhood, there is a danger that it may become habitual.

And the result of “habitual” bullying behavior, VMware, is alienation. In this case, of your customers.

June 9, 2009  1:55 PM

Upgrading production servers to vSphere: When and why

Eric Siebert Eric Siebert Profile: Eric Siebert

I wanted to know when current VMware users are planning to upgrade their existing VMware production environments to vSphere, so I ran a poll on my website. I then ran additional polls to find out the primary reasons that users are holding off on upgrading to vSphere, and to see if they are planning on upgrading their Enterprise licenses to the new Enterprise Plus licenses.

What I discovered is that while people are eager to upgrade to vSphere, there are reasons why others are waiting; and while some will upgrade from Enterprise to Enterprise Plus licensing before the end of the year, a number of respondents indicated that they have no plans to upgrade at all.

When will you upgrade?

The first poll clearly indicated that many existing customers will be quick to upgrade to vSphere to benefit from the new technologies. Out of 140 responses, 32% are planning to upgrade within 3-6 months and another 25% of users are planning on upgrading within 0-3 months.

Only 12% had already upgraded to vSphere, so my second poll was to find out the primary reason that users were holding off on upgrading to vSphere. Continued »

June 8, 2009  4:45 PM

Should you upgrade to VMware vSphere 4?

Texiwill Edward Haletky Profile: Texiwill

Since I am an independent consultant and VMware Communities Guru, I have recently been asked many questions about whether or not to upgrade to VMware vSphere 4. My answers depends on the following items:

  • The hardware involved. VMware vSphere has certain hardware requriements, if your target hosts do not support these minimal requirements, then they are not good candidates for running VMware vSphere. The basic requirements are:

- 64 Bit CPU support. This does mean that some EMT64 machines will work. However they may not be on the VMware Hardware Compatibility Guides.

- Intel-VT or AMD-V support. This pretty much goes without saying; it is impossible to use VMware vSphere if these features are not enabled within the BIOS.

- No eXecute (NX) or eXecute Disable (XD) support within the BIOS. In some cases you are required to enable this bit to allow VMware vSphere to run.

June 4, 2009  5:27 PM

Licensing complications frustrate vSphere users

Eric Siebert Eric Siebert Profile: Eric Siebert

VMware vSphere is a free upgrade for licensed customers with active Support and Subscription (SnS), but many users have reported problems obtaining their new vSphere licenses from VMware. Because VMware decided to go with a simpler license key instead of a more complex license file, new license keys are needed to use vSphere. When vSphere went GA on May 21st, VMware did license upgrades for its customers, creating vSphere license keys for them. Presumably this process looked at customers’ existing licenses, checked if their maintenance was current to see if they were entitled to the upgrade and, if so, generated keys for all the hosts and vCenter Servers that qualified.

My own experience with obtaining vSphere licenses was frustrating as well. On May 21st, when vSphere was first posted to VMware’s website, I logged into the website, went to download the vSphere bits and found I was unable to — the licensing portal claimed I had no active contracts and therefore was not entitled to the upgrade. There was also a big note at the top that stated that the license upgrades were in progress and could take some time to complete. OK, I figured, I’ll give them some time and try again later in the day. I was anxious to get the new GA code, however, and found I could download it right away by choosing to evaluate vSphere instead. The download process was fairly smooth, downloads were quick despite the large file sizes and they had a download manager utility that allowed multiple file downloads at once.

Continued »

June 1, 2009  9:30 PM

Configuration Maximum document updated for vSphere

Eric Siebert Eric Siebert Profile: Eric Siebert

VMware publishes a great document called Configuration Maximums that details all the configuration maximums for the various components of virtual machines, hosts and vCenter servers. With the vSphere release, VMware has created a new document that has both new and modified configuration maximums specifically for vSphere. I went through and compared the latest VI3 document with the vSphere document and I noted some differences between the two, which are listed below.

Virtual Machine

VI 3.5

vSphere 4

Number of virtual CPUs per virtual machine



RAM per virtual machine

64 GB

255 GB

NICs per VM



Concurrent remote console sessions



ESX host

VI 3.5

vSphere 4

Hosts per storage volume



Fibre Channel paths to LUN



NFS Datastores



Hardware iSCSI initiators per host



Virtual CPUs per host



Virtual Machines per host



Logical processors per host



RAM per host

256 GB

1 TB

Standard vSwitches per host



Virtual NICs per standard vSwitch



Resource pools per host



Children per resource pool



Resource pools per cluster



You should pay special attention to the many footnotes in the document that detail special circumstances for some of the maximums. Continued »

May 27, 2009  4:06 PM

New SiteSurvey utility from VMware checks for Fault Tolerance compatibility

Eric Siebert Eric Siebert Profile: Eric Siebert

While looking at the newly published VMware KnowledgeBase article on processor support for the new Fault Tolerance (FT) feature last week, I noticed a link to a new utility called SiteSurvey. While it wasn’t active at the time, this week SiteSurvey is available for download.

This utility specifically tests to see if the hosts in your clusters are compatible with the new Fault Tolerance feature. It is available as either a Windows or Linux download and once you install and run it, you will be prompted to connect to a vCenter Server.

Continued »

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: