Virtualization Pro


December 4, 2008  7:43 PM

VMware View Composer and vCenter architecture

Rick Vanover Rick Vanover Profile: Rick Vanover

With the recent release of VMware View, one of the hottest components of the desktop virtualization component is the linked clone technology. In planning how VMware View works, it is important to understand the critical component – VMware View Composer.

VMware View Composer is simply a Windows service that resides on a vCenter server. VMware View Composer interacts with both vCenter and the View Connection Manager. For environments that already have a server based VMware environment with vCenter and ESX hosts, it is clear that a separate environment is a good idea. This would be best served through dedicated hosts, storage and a separate vCenter server. The figure below shows how the VMware View Composer and vCenter installations would work together:
VMware View Composer Service
The VMware View Composer service, or svid, interacts with vCenter from the configuration set forth from VMware View Connection manager, which functions as the broker for connections. Once the linked clone virtual desktops are created, they then deliver the storage optimization that we have been anticipating with the release of VMware View.

One other feature of View Composer is storage over-commit. This functionality is a configurable level of how the linked clones’ delta disk, or differencing disk, is allocated. Looking at a guest virtual machine, the delta disk would be a very small percentage of the parent or base VM. This setting will determine the behavior of determining how many VMs will fit on a datastore. A setting of conservative will enable less VMs to fit on a datastore, yet run less of a likelihood of running out of space. While a more aggressive level will attempt to put more VMs on the datastore and reserve less storage reserved for the delta disks. With that information, it is critically important to get an expectation of the delta disk behavior to best utilize the storage.

A final key component of View Composer is the Quickprep feature. Quickprep does the guest VM specific tasks such as domain membership, organizational unit placement in Active Directory and run any scripts on the guest VM.

With this information primer on VMware View Composer, it is important to isolate the vCenter and more importantly be aware of how the virtual desktop managment agents will interact with the vCenter server. More information on VMware View can be found on the VMware website.

December 2, 2008  6:23 PM

VMware product name changes: vCenter, View

Eric Siebert Eric Siebert Profile: Eric Siebert

VMware has just announced that they are changing the names of their current products to the new product names that were announced at VMworld. The biggest changes are that VirtualCenter is now being referred to as vCenter Server and Virtual Desktop Infrastructure (VDI) is now being referred to as View. vCenter is also the name that will be used for all of their infrastructure and management applications (i.e. Lab Manager is now vCenter Lab Manager) and View will also be the name for all of their desktop applications (i.e. Virtual Desktop Manager is now View Manager).

While this was a good time to change the name for VDI to View with the release of the new version (View 3) it was expected that the vCenter name change would not take place until the release of VI4. Subsequently this may lead to confusion for a while as the vCenter name is effective immediately for the current version of the product (VirtualCenter 2.5 Update 3). VMware has updated their website accordingly for all the product download and documentation pages but the name has not been changed in the current documentation or product. Whether they apply the name change inside the application and documentation in the next release of vCenter (presumably 2.5 Update 4) or wait until VI4 is released is not currently known. Additionally this only applies to the current versions which presumably means any version of vCenter 2.5, version 2.0.x is still referred to as VirtualCenter.

All of the product name changes affected by this are listed below:

VMware VirtualCenter → VMware vCenter Server
VMware Lifecycle Manager → VMware vCenter Lifecycle Manager
VMware Converter → VMware vCenter Converter
VMware Lab Manager → VMware vCenter Lab Manager
VMware Stage Manager → VMware vCenter Stage Manager
VMware Update Manager → VMware vCenter Update Manager
VMware Site Recovery Manager → VMware vCenter Site Recovery Manager
VirtualCenter Foundation → vCenter Server Foundation
VMFS → VMware vStorage VMFS
VMware Virtual Desktop Infrastructure → VMware View
Virtual Desktop Manager (VDM) → VMware View Manager
VMware Administrator Interface → VMware View Administrator
VDM Agent → VMware View Manager Agent
VDM Web Access → VMware View Portal
VDM Client for Windows → VMware View Client for Windows
VDM Client for Linux → VMware View Client for Linux


December 1, 2008  7:22 PM

Understanding VMware ESX security zones

Texiwill Edward Haletky Profile: Texiwill

There is confusion around VMware ESX with regards to security zones. On one hand VMware ESX is a single multi-homed physical machine. On the other hand, it contains multiple security zones. We need to look within the physical to properly understand security zones within VMware ESX and ESXi.

Security zones within VMware ESX are either based on networks employed (Management, VMotion, Storage, DMZ, Production, etc.) or it can be based on functionality such as hypervisor, management tool, backup systems, or virtual machine.

Each of these zones reflects a configuration for the VMware ESX host. It also reflects what is being run within the virtual machines as well as what network upon which the VMs and hosts resides.

Network security zones are relatively easy to understand. Each network would be isolated from one another except through well defined gateways and firewalls. The confusion among some security administrators is that they believe since the VMware ESX or ESXi host has multiple adapters that it would therefore act as an undefined gateway between the disparate networks.

This is simply not the case. Unlike other VMware products where networking is done using bridging technology, the virtual switch within the ESX host is similar to any other layer 2 physical switch. The physical NICs within the VMware ESX or ESXi host act as uplink ports between the physical switch and the virtual switch. Whether a virtual NIC used by virtual machines (VMs), or a vmkernel NIC used by the hypervisor, each virtual device connects to a portgroup within a virtual switch then out the physical NICs to the physical switches unless the communication is VM to VM on the same portgroup.

Since there is no way to layer virtual switches or portgroups, the only way to speak across them is by the use of well-defined gateways or firewalls. VMs can act as firewalls and gateways but only if they have more than one virtual NIC. So the rule of no multi-homed VMs is the best place to start unless it is a well defined virtual firewall or gateway.

The other definition of security zones is based on the roles used within the virtual infrastructure as well. There are three well-defined roles within the virtual infrastructure: hypervisor, virtual machine, and management tool.

The hypervisor controls everything so access to this must be limited to only the service console or management appliance, data stores used as VM storage, or VMware VMotion interfaces. Two of these are through vmkernel NIC used by the hypervisor as described previously: data stores for VM Storage and VMware VMotion Interfaces. Anything that accesses the hypervisor must be protected from those items that should not be able to access the hypervisor.

A virtualization management tool is another role within the system, these tools interact with the service console or management appliance for the virtualization host which will indirectly interact with the hypervisor. This indirect interaction is the key to handling this role. Virtualization management tools such as VMware VirtualCenter, HP SIM VMM plugin, VMware Lab Manager, etc. should be firewalled from other networks as they have indirect access to the hypervisor.

Backup systems act as another role within a virtual environment. These systems can interact directly with the storage devices attached to virtualization hosts through VMware VCB, or through the service console or management appliance just like virtualization management tools. This can also lead to indirect interaction with the hypervisor. Due to the direct and indirect access these systems should be firewalled as well from other networks.

The last role is that of a standard virtual machine. These virtual machines could live on any network but they should not be able to directly access the service console or management appliance of thse virtualization host in addition they should not be able to directly attach to the storage device ued by the virtualization hosts. Virtual machines are considered to be hostile to the virtualization host and should be treated as such.

Security zones depend on the security roles each component of your virtual environment directly or indirectly interacts. These few tips will aid in designing a secure environment.


November 26, 2008  4:24 PM

Quick performance enhancement for VMware Converter P2V conversions

Rick Vanover Rick Vanover Profile: Rick Vanover

During a recent VMware Converter-based physical-to-virtual (P2V) conversion, I made an observation and subsequent change that enhanced the performance of the conversion. During the conversion, the locally installed antivirus application became extremely CPU-hungry while VMware Converter was running. I have written quite a few blogs on topics ranging from converting large storage, cleansing drivers, breaking the conversion into stages and some gotchas related to block size on the VMFS volume. Now, there is another way to streamline a VMware Converter P2V conversion: stop the antivirus service on Windows guests.

Some antivirus platforms, howeverm don’t seem to affect the P2V conversion. This particular system was a new antivirus program for me, and it was consuming approximately 70% of the CPU during the conversion. Once the Windows-based service was stopped, the VMware Converter executable, vmware-ufad.exe, was able to consume more processing power and proceed with the conversion more quickly.

Stopping the antivirus software is a natural complement of other preparatory stages of the P2V process. Other steps that can quiet down the candidate virtual machine during the conversion process include stopping database services if installed, stopping file sharing on large file shares, changing the guest machine to a private network that is only routed to the VMware host for direct access for the conversion, and possibly using a VMware Converter boot CD for the cold clone of the computer in a zero-transaction state.


November 25, 2008  4:29 PM

CompareMyVM website to offer VMware ESX virtual machine configuration comparisons

Rick Vanover Rick Vanover Profile: Rick Vanover

Virtual machine provisioning sometimes requires administrator guesswork which doesn’t always yield the best results. To help address this need, Portsmouth, N.H.-based VKernel has launched a beta-stage community website, CompareMyVM.com, which will post configuration of virtual machines to help virtualization administrators.

At this point on the site, there are hardly enough virtual machines to make a quality sample of how the world is using ESX-based VMs. So, I encourage you to upload to the site, and as it matures the quality of the site will improve with more data. Uploading a VM or appliance’s configuration is easy, simply go to the CompareMyVM website and select to Snapshot My VM via a Java applet that will connect to VirtualCenter or the ESX host, or you can click the Contribute My VM button to enter the configuration manually. Once a virtual machine is loaded into the site, it is shown in the list among the other contributed VMs. An example of a VM I uploaded is shown below:
CompareMyVM website

Uploaded VMs can be given information about what applications they run, such as a Windows domain controller, Apache 2.2 web engine or an Exchange mail server. At first glance, you may think that this is simply a way that virtualization admins can put their data on the CompareMyVM site and have it sold as data that says how administrators configure their VMs.

Not so according to Christian Simko, director of marketing and communications from VKernel who states that “The idea is to keep it open and never charge for it… We may eventually publish a best practices guide with data derived from CompareMyVM, but that will also be a freebie.” This is good news, as the site develops, there will surely be value to determine how other administrators are provisioning VMs with similar application inventories.

As the site matures, look for more comparative performance data on the contributed configurations as well. To date, there is only a limited number of virtual machines posted from virtualization users. I’ve put some VMs up there and hopefully you will as well. As more users contribute data, this free service will hopefully provide some valuable comparative information on how people use virtualization across different market segments. Most importantly, this information will be vendor-neutral, so different server hardware as well as various software configurations will be represented on the community site.


November 20, 2008  9:00 PM

Handy VMware guest compatibility documentation

Rick Vanover Rick Vanover Profile: Rick Vanover

The planning steps are incredibly important for a successful configuration of any VMware implementation, regardless of shape and size. One specific area you should consider while planning is supported guest operating system configuration limitations.

VMware frequently updates the Guest Operating System Installation Guide (GOSIG), an online book that gives specific information for VMware ESX Server, VMware GSX Server, VMware Server, VMware ACE, VMware Workstation and VMware Fusion guest operating systems. This guide gives very specific configuration matrices for the VMware product and the guest OSes that can be run within the product. Further, there are very handy known issues sections for each guest OS.

Supported VMware Server 2.0 configurations

A specific example that I have found this guide helpful in regards to VMware Server 2.0. According to the GOSIG resource, Windows Server 2003 is only a supported configuration on VMware Server 2.0 with Service Pack 1, Service Pack 2 or the R2 features added to Windows Server 2003. VMware ESX, however, has a fully supported configuration for Windows Server 2003, including the base release without any Service Packs or the R2 features, in all versions from 2.0 through 3.5 Update 3.

Supported Windows Server 2008 configurations

Some configurations are more obvious, such as running Windows Server 2008 as a guest operating system on a hypervisor that predates the release of the guest OS. In the GOSIG guide, Windows Server 2008 64-bit guest OSes are supported only on more current products. Some platforms, such as VMware GSX server and VMware ESX Server 2.x and 3.0x are not a supported configuration for this guest OS. Even with all of this information, and the officially supported configurations – you may find that certain situations are successful even though they are not listed in the GOSIG documentation. A better practice would be to match the hypervisors with the supported configurations in regards to the guest OSes, and this may mean standing up different versions of VMware products to cover the full range of OSes that are required in your environment.


November 20, 2008  8:41 PM

Antivirus software on the VMware ESX Service Console?

Eric Siebert Eric Siebert Profile: Eric Siebert

This question is often asked, should I install antivirus (AV) software on the VMware ESX Service Console? If you ask VMware and many seasoned ESX Administrators the answer is usually no. According to VMware’s Security Hardening Guide ESX is less susceptible (not immune) to viruses and if you follow proper security best practices installing AV software on the service console is not recommended:

Because it is based on a light-weight, kernel optimized for virtualization, VMware ESX Server is less susceptible to viruses and other problems that affect general-purpose operating systems. However, ESX Server is not impervious to attack, and you should take proper measures to harden it, as well as the VMware VirtualCenter management server, against malicious activity or unintended damage.

Because ESX Server runs a customized, locked-down version of Linux, there is much less likelihood of security exploits than in a standard Linux distribution. If you follow the best practice of isolating the network for the service console, there is no reason to run any antivirus or other such security agents, and their use is not recommended. However, if your environment requires that such agents be used, then use a version designed to run on Red Hat Enterprise Linux 3, Update 6.

The key here is following proper security best practices. This means taking steps like keeping your host patched in a timely manner, isolating Service Console network traffic from the other traffic on the network including VM traffic and following the additional hardening best practices listed in VMware’s Security Hardening Guide and CISecurity’s ESX Security Benchmark.

There are some good reasons for not installing AV software on the Service Console. The first reason is that it can impact on the performance of the ESX host and subsequently all of the virtual machines that reside on it because of the extra CPU, memory and disk resources that antivirus software typically uses. Antivirus software can be particularly resource intensive and can draw resources away from the virtual machines and potentially have a very negative impact on them. Another reason is that there is the possibility that the software may cause other issues on the Service Console as is true with any additional third party software that is installed on the Service Console.

Despite the reasons for not installing it some enterprises mandate that AV software be installed on all systems regardless of how secure they are. You might try and exempt your ESX hosts from this by explaining the following points:

1. The design of ESX does not allow for a VM that is infected by a virus to spread it to the ESX host through the VMKernel. It can only spread through traditional means which is typically over the network by leveraging open ports and OS vulnerabilities. If you isolate your Service Console network then it is protected from any VMs that may be infected.

2. Most viruses are written to exploit Windows systems, there are very few viruses that are written specifically for Linux systems.

3. ESX has a built-in firewall that protects the Service Console and blocks all ports except those few required by ESX and VirtualCenter.

4. ESX has a very good historical track record, I’ve never heard of a virus infecting the ESX Service Console. This doesn’t mean it can’t happen just that the chances are extremely low.

5. Explain the negative performance impact the AV software will have on an ESX host which can affect all of the VMs on the host.

If you are forced to install antivirus software on the Service Console because of security requirements in your environment then you should make sure that you run a version that is designed for the version of Linux that the Service Console uses. Additionally try and configure it as minimal as possible to minimize the impact on the server’s resources. It’s best to exclude all your VMFS volumes from scanning or at a minimum exclude specific virtual machine files like vmdk and vswp files that are frequently written to. Make sure and keep a close eye on the performance of the host to see how much impact the antivirus software is having on it. Also if you have to run on-demand scans make sure they are performed in off-peak hours when activity on the ESX host is low.


November 17, 2008  6:31 PM

Security outside the box

Texiwill Edward Haletky Profile: Texiwill

Virtualization security depends on more than securing the virtualization host and hypervisor. It depends on everything that touches your virtualization host, directly or indirectly, also being secure.

Let us take a quick look at the daily operations you may perform within your virtual infrastructure.

  • Create/Move/Modify virtual machines (VMs)
  • Backup/replicate VMs
  • View performance and other data about hosts and VMs
  • Install/Reinstall/Update hosts
  • Add or remove administrative users from VMs and hosts
  • Monitor your VMs and hosts
  • etc.

The list is quite endless. Each one of these actions will affect security or be affected by security.

Ease of use, ease of administration is sometimes paramount. This is the age-old debate of security over usability. Do not knock holes in your firewalls, instead, use existing secure protocols to improve the experience. One I use is OpenVPN to access my administrative network workstation using pre-shared keys for encryption. The extra step of starting up the VPN is trivial with the service provided, this gives me secure access to the management network over which the tools can be run.

Affected by Security

There will be at least one more step involved in use of daily tools then you would normally use. Mainly some way to ensure your access to the management tools is secured from sniffing by your non-administrative network. Or the use of different procedures to ensure you have proper auditing in place.

One tool I use is a simple expect script called expectsudo which will allow me to run remote commands on my VMware ESX hosts while retaining logging of all access. To use this tool you must first install the expect RPM onto your host. Then setup sudo to allow access to the appropriate commands.

#!/usr/bin/expect --
set pass [lindex $argv 0]
set timeout 5
 
spawn /usr/bin/sudo [lrange $argv 1 end]
expect "Password: {send "$passr"; exp_continue
sleep 1

In the above script the first argument will be the password to use. This script can not be run remotely using any form of SSH. For example:
ssh user@hostname expectsudo password esxcfg-vswitch -l
The only drawback to this script is the need to have the password available, instead you can setup pre-shared keys for SSH and setup Sudo to allow the  user to issue certain commands without requiring a password. This form of single sign on relies on the security of the management workstation in use.

Security of your VMware Infrastructure relies on the security of all the management workstations and servers in use. Not just on the hardening of the VMware ESX host.


November 17, 2008  5:22 PM

VMFS volume names: UUID and symbolic

Eric Siebert Eric Siebert Profile: Eric Siebert

If you have ever done a directory listing of your VMFS volumes on a VMware ESX host from the Service Console or using a file browser application like WinSCP you will notice the names of your VMFS volumes but also a number of directories that consist of a long string of numbers and letters as seen below.

VMFS Volume Listing in WinSCP

If you look in these directories the contents of them are exactly the same as the directories that are named the same as your VMFS volumes. So what are these directories? Let’s explain what happens when you create a VMFS volume to find out.

When creating VMFS volumes you are prompted to name them. This name is not what the ESX host uses to reference the volume; it is purely a friendly name to make it easier for the user to identify the volume. The ESX host actually uses a unique identifier called a Universal Unique ID (UUID) to reference the volume. The name you specify when you create a VMFS volume is a user-defined device name which is a symbolic link to the UUID of the VMFS volume. This is done to solve the problem of changing the device name, when you change the volume name you are only changing the user-defined device name and not the UUID of the volume. So when you look in your /vmfs/volumes directory you will see both a UUID, ie. 4404e8b4-bcfd52fc-1e4b-0017a4a91076 and the symbolic link, i.e. ServerA-Local. Changing to the symbolic link name by using the cd command or clicking on it in WinSCP simply takes you to the UUID directory. You can see the relationship between symbolic links and UUID’s by using the ls –l command inside the service console as shown below.

VMFS Volume Listing in the ESX Service Console

Additionally you can see the UUID of a volume in the VMware Infrastructure Client by selecting a volume in the Configuration, Storage section and then looking in the Details pane at the Location field. It’s definitely a lot easier to remember the volumes friendly name then it’s long UUID which is why symbolic links are used with ESX.


November 17, 2008  5:19 PM

VMware Converter failures due to block size

Rick Vanover Rick Vanover Profile: Rick Vanover

Recently during a P2V conversion, I had an issue where the conversion failed for reasons I could not explain. In looking at the logs of VMware Converter, there was no clear indication of the specific issue. Some of the relevant log entries are shown below:

[#2] [2008-11-08 08:16:32.953 'App' 1592 error] [managedVMCreator,1033] CreateVm task failed: Insufficient disk space on datastore ''.

[#2] [2008-11-08 08:16:33.171 'App' 1536 error] [imageProcessingTaskImpl,552] VmiImportTask::task{4}: Image processing task has failed with MethodFault::Exception: vim.fault.NoDiskSpace

After some deeper research, it turns out that this particular datastore was created with the default 1 MB block size, making VMDK files limited to 256 GB. Fellow SearchVMware.com blogger Eric Siebert has an excellent post on this site about the formatting options of the VMFS volume as it is brought into VI3. When you add a LUN, you select the block size during the format shown in the figure below:

Selecting a block size

While running VMware Converter on very large volumes is rare, this was one that made me stop and think for while. There are plenty of strategies for very large storage for a P2V conversion, but this was one where the other LUNs were configured with the 1 GB block size and that was fine thus far.

Luckily, there was space that I could evacuate all other virtual machines on that LUN so that I could remove the LUN and format it with a larger block size to get the VMDK file size I needed. The tricky part of this situation is that with VMware Converter experiencing the error on the block size, it was displayed in the log in a way that made it a little tricky to discover.


Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: