VMware Converter

P2V conversion success: Disable non-plug-and-play drivers

While performing a physical-to-virtual (P2V) conversion isn’t a new trick, there are always additional enhancements that can be performed to ensure a clean transition to a VMware platform. Recently when performing a P2V conversion, one particular system was not behaving as expected after the conversion was complete. While I have mentioned before that removing drivers from the new guest virtual machine is a good idea, this particular system required some more attention.

I found that some of the device drivers that were loaded in the Windows guest operating system even after all driver software was removed. Further, after the P2V conversion the hardware would not be enumerated in the hardware inventory because it was not present. This particular system had SAN connectivity before, and the drivers related to the fibre channel interface were causing me concern related to disk access, and a lot of errors in the local log. On the VMware virtual machine, the driver installed by VMware Tools provides all of the required disk access and I needed to stop this physical system hardware driver from filling up the error log.

The driver was listed in the Non-Plug and Play Drivers section of the Windows device manager. To view this section, be sure to view the hidden devices in the MMC snap-in for Windows Server 2003 systems. The figure below shows this area of Windows:

Once I identified the driver that was causing the issue, it was quite easy to disable the device. On subsequent boots, the offending process did not fill the Windows logs up with the errors related to the device not being present.

Failed P2V conversion? Check DNS

In the course of performing many physical-to-virtual (P2V) conversions, I still come across situations where a small configuration can make or break the task. Recently, I performed a P2V conversion of a system that was successful in all measurable elements from the VMware Converter and ESX-based virtual machine standpoint, yet the application was not functioning as expected. One important part of this conversion is that the virtual machine changed networks during the conversion. Everything checked out locally on the virtual machine, including DNS configuration, subnet mask and the IP default gateway. Yet the application still had issues working correctly.

Later on, I had discovered that there was a DNS record for the same IP address of the converted system. In this scenario, the workload had a separate DNS name that was in use by the application and not exclusively using the host name. Further, I believe that the  record was a previous computer name of the converted system, and the application had some calls that still used the old name. Before I noticed this entry in DNS, I had checked the local host file on the Windows virtual machine to ensure that there were no entries pointing to an IP address other than 127.0.0.1 for any local references.

So what can be done to make sure that this bizarre information does not happen again? In the pre-conversion steps, take a look through the DNS zones and see if the IP address or the hostname are pointing to anything that references a specific IP address. While applications may have configuration beyond the scope of a P2V conversion, identifying any DNS or host file configuration before the conversion can save some frustrating and possibly unexplained application failures.

VMware product name changes: vCenter, View

VMware has just announced that they are changing the names of their current products to the new product names that were announced at VMworld. The biggest changes are that VirtualCenter is now being referred to as vCenter Server and Virtual Desktop Infrastructure (VDI) is now being referred to as View. vCenter is also the name that will be used for all of their infrastructure and management applications (i.e. Lab Manager is now vCenter Lab Manager) and View will also be the name for all of their desktop applications (i.e. Virtual Desktop Manager is now View Manager).

While this was a good time to change the name for VDI to View with the release of the new version (View 3) it was expected that the vCenter name change would not take place until the release of VI4. Subsequently this may lead to confusion for a while as the vCenter name is effective immediately for the current version of the product (VirtualCenter 2.5 Update 3). VMware has updated their website accordingly for all the product download and documentation pages but the name has not been changed in the current documentation or product. Whether they apply the name change inside the application and documentation in the next release of vCenter (presumably 2.5 Update 4) or wait until VI4 is released is not currently known. Additionally this only applies to the current versions which presumably means any version of vCenter 2.5, version 2.0.x is still referred to as VirtualCenter.

All of the product name changes affected by this are listed below:

VMware VirtualCenter → VMware vCenter Server
VMware Lifecycle Manager → VMware vCenter Lifecycle Manager
VMware Converter → VMware vCenter Converter
VMware Lab Manager → VMware vCenter Lab Manager
VMware Stage Manager → VMware vCenter Stage Manager
VMware Update Manager → VMware vCenter Update Manager
VMware Site Recovery Manager → VMware vCenter Site Recovery Manager
VirtualCenter Foundation → vCenter Server Foundation
VMFS → VMware vStorage VMFS
VMware Virtual Desktop Infrastructure → VMware View
Virtual Desktop Manager (VDM) → VMware View Manager
VMware Administrator Interface → VMware View Administrator
VDM Agent → VMware View Manager Agent
VDM Web Access → VMware View Portal
VDM Client for Windows → VMware View Client for Windows
VDM Client for Linux → VMware View Client for Linux

Quick performance enhancement for VMware Converter P2V conversions

During a recent VMware Converter-based physical-to-virtual (P2V) conversion, I made an observation and subsequent change that enhanced the performance of the conversion. During the conversion, the locally installed antivirus application became extremely CPU-hungry while VMware Converter was running. I have written quite a few blogs on topics ranging from converting large storage, cleansing drivers, breaking the conversion into stages and some gotchas related to block size on the VMFS volume. Now, there is another way to streamline a VMware Converter P2V conversion: stop the antivirus service on Windows guests.

Some antivirus platforms, howeverm don’t seem to affect the P2V conversion. This particular system was a new antivirus program for me, and it was consuming approximately 70% of the CPU during the conversion. Once the Windows-based service was stopped, the VMware Converter executable, vmware-ufad.exe, was able to consume more processing power and proceed with the conversion more quickly.

Stopping the antivirus software is a natural complement of other preparatory stages of the P2V process. Other steps that can quiet down the candidate virtual machine during the conversion process include stopping database services if installed, stopping file sharing on large file shares, changing the guest machine to a private network that is only routed to the VMware host for direct access for the conversion, and possibly using a VMware Converter boot CD for the cold clone of the computer in a zero-transaction state.

VMware Converter failures due to block size

Recently during a P2V conversion, I had an issue where the conversion failed for reasons I could not explain. In looking at the logs of VMware Converter, there was no clear indication of the specific issue. Some of the relevant log entries are shown below:

[#2] [2008-11-08 08:16:32.953 'App' 1592 error] [managedVMCreator,1033] CreateVm task failed: Insufficient disk space on datastore ”.

[#2] [2008-11-08 08:16:33.171 'App' 1536 error] [imageProcessingTaskImpl,552] VmiImportTask::task{4}: Image processing task has failed with MethodFault::Exception: vim.fault.NoDiskSpace

After some deeper research, it turns out that this particular datastore was created with the default 1 MB block size, making VMDK files limited to 256 GB. Fellow SearchVMware.com blogger Eric Siebert has an excellent post on this site about the formatting options of the VMFS volume as it is brought into VI3. When you add a LUN, you select the block size during the format shown in the figure below:

While running VMware Converter on very large volumes is rare, this was one that made me stop and think for while. There are plenty of strategies for very large storage for a P2V conversion, but this was one where the other LUNs were configured with the 1 GB block size and that was fine thus far.

Luckily, there was space that I could evacuate all other virtual machines on that LUN so that I could remove the LUN and format it with a larger block size to get the VMDK file size I needed. The tricky part of this situation is that with VMware Converter experiencing the error on the block size, it was displayed in the log in a way that made it a little tricky to discover.

Secure method to P2V across security zones

A common VMware Communities question is how to P2V or convert a system from within a demilitarized zone (DMZ) to a virtual machine (VM) running within an ESX host that will be part of the DMZ virtual network.

P2V works by imaging the physical host within the DMZ and transferring that image to the administrative/management network attached to the service console (management appliance) of the VMware ESX(i) host. This in essence crosses security zones and could connect the hostile DMZ to the ‘in need of protection’ virtualization management network. Access to this network from the DMZ could be disastrous.

One solution is to perform the P2V migration in stages.

1. Create the DMZ virtual network within your virtual infrastructure.
2. Get your security team to bless a laptop/workstation for work within the DMZ. Ensure this laptop/workstation has enough removable storage to contain the resultant VM or VMs of the physical servers you wish to convert.Use your  P2V tool to convert the VM and store it on the removable media.
3. Disconnect the removable media and bring it to your secure administrative network.
4. Connect the removable media to a workstation within the administrative network. Ensure this connection is read-only for the moment if possible.
5. Virus Scan the removable media, but note a VMDK can give false positives; you are really looking for anything that may be hidden from view.
6. Use VMware Converter to import the VM or VMs into the virtual infrastructure ensuring they are connected to the proper virtual network.
7. Power on the VM with the network disconnected and fix any issues that are caused by the P2V migration, such as the need to remove hardware agents, and fix anything that needs to be fixed.
8. Reboot the VM with the network connected

The P2V migration is now complete and isolated from the network. The key to this is to only power on the VM once you are within a safe environment and to check for viruses and worms that may live within your DMZ.

Why you should upgrade to VI3 Update 1

VMware Infrastructure 3 Update 1, made available on April 10 2008, introduces some core updates to ESX Server 3.5, VirtualCenter 2.5, and the VMware Infrastructure Management Installer. The biggest reason to upgrade, however, is the inclusion of Storage VMotion.

Among the core features now available with ESX 3.5 Update 1 are the addition of the Intel 82598 10 GB Ethernet controller, support of Jumbo Frames and NetQueue, additional Microsoft Clustering Services (MSCS) support, additional backup product and management agent support, additional guest operating systems, and additional server models.

I’ve been working with ESX 3.5 Update 1 for a few weeks, and the installation and behavior are indistinguishable from both ESX 3.5’s base release and ESX 3.02, with the exception of context sensitive tasks or options.

When I test upgrades, I make a point to test the upgrade in an environment with dissimilar ESX host server releases. For example, most of my hosts are ESX 3.02. When I upgrade the first one to ESX 3.5, I want to make sure that nothing goes wrong. I want to know that I’ll be able to sustain a mixed environment with all functionality. When I migrate running virtual machines through host-based VMotion to the ESX 3.5 host, and the reverse, I want to make sure to the best of my ability that nothing will fail. I also want to ensure that all of the VMware DRS and VMware High Availability rules are still enforceable with the mixed-host inventory.

Outlining a functionality matrix and the verification of the behavior is key to having no surprises during a live upgrade. Testing the update to VirtualCenter is a little more difficult but I am setting up a test environment soon to ensure that everything functions as expected in my environment. Overall, the fixes and new features make ESX 3.5 Update 1 an attractive upgrade for systems that are not there already.