The Virtualization Room

April 6, 2007  12:22 PM

VMware price list

Alex Barrett Alex Barrett Profile: Alex Barrett

Robin Harris of StorageMojo has procured a VMware price list, adding to his extensive collection of storage products. You can find it at:

According to the Price List introduction, “These prices are discounted “street prices”, roughly what a corporate customer would pay.” It’s unclear to me who submitted these price lists, and how current they are, but they might be useful nevertheless.

April 6, 2007  10:09 AM

Managing VirtualCenter2 Management Service (vpxd) failures

Kutz Profile: Akutz

The VirtualCenter2 management service (vpxd) is not very robust. If it cannot connect to the back-end database, the service will halt. It *should* continue to run, periodically trying to connect to the database, but this is not the case. There are also problems with the service coming up before the network when the server boots, causing the service to halt upon start. IPSec SA token mismatch/renewal issues also cause the service to halt. The vpxd service is very important — it manages DRS, collects performance statistics, and allows users to manage their VMs. This is a service that should be a lot more capable of handling foreseeable circumstances. To that end I have written a script that can be used to restart the vpxd service in case it halts or fails to start. This script can be linked to an often underutilized feature of Windows — the ability for the Service Control Manager (SCM) to restart a service upon failure.

The script is fairly basic. I will not post it in its entirety in this blog because its formatting will get munged by WordPress’ draconian style settings. You can download the script from

A description of the script can be found in the script’s source, “This script will attempt to start the vpxd service if it is not started. If the serivce is already started or it starts successfully the first time no further action is taken. This script can also be run upon a VCMS failure. It will notify a specified e-mail address of the failure. It will check the connection to the VCMS database. If the database connection is valid then the script will start the VCMS service. If the connection is not valid then the script will go into a loop, attempting to restart the VCMS service every specified number of minutes. This script assumes your VC database is on a SQL server. If it is not then please see for a good reference on how to build an ADO connection string to fit your needs.”

In addition to using this script to correct a vpxd service failure, it can also be run as a scheduled task, set to run 5 minutes after the server boots. This can correct the problem of the vpxd service not starting successfully on boot because it comes up before the network is available.

I hope this script helps to make your VMware VirtualCenter2 Management Server a little more robust, and a lot more script-diddly-licious!

April 6, 2007  7:40 AM

Not so needless Windows Server licenses

Alex Barrett Alex Barrett Profile: Alex Barrett

Last week, I wrote a story entitled Virtuozzo user avoids needless Windows Server licenses. Many of you wrote in to inform me that in fact, the Windows Server licenses that the subject had not purchased were in fact required under Microsoft licensing terms. In the words of one reader:

There is a lot of misinformation occurring specific to license reduction fees and Virtuozzo. I work for a Fortune 100 with an extremely strong VMware (ESX) presence and we too were initially enamored with the ability to reduce OS licensing fees.

Quite Simply – you need to pay for each virtualized (‘VPS’ in Virtuozzo speak) instance just like you would for separate Virtual Machines in the VMware World.

SWsoft used to tout that you only have to pay for the ‘parent’ Windows OS — that was until Microsoft gave them a ‘stern reprimand’. Today you will not find a single SWsoft employee or any verbiage on their site that indicates you only have to pay for one OS license.

The only exception to this is Microsoft Datacenter Edition. John Yanekian is in breach of license obligation with his current configuration if he has not licensed all of his VPS’s.

The story has since been updated and renamed. Thanks to everyone who pointed out this error.

April 5, 2007  3:23 PM

Physical-to-virtual (P2V) migration blooper

Jan Stafford Jan Stafford Profile: Jan Stafford

Since I wrote the post, P2V wins and losses, I’ve been hearing from IT managers who’ve done the P2V deed. One migration was halted by a too-proactive IT guy. Here’s the story, but I’ll let the parties in question remain anonymous.

So, this company was moving from physical to virtual servers to reinstall its Oracle ODBC drivers and number of third party support applications. During the migration of one physical server, the process failed. Why? Says the IT manager:

“The first attempt failed because our support personnel noticed the server being replaced was down and rebooted it in the middle of the P2V process.”


After that, migrations went smoothly, and no more mistakes were make. However, performance issues arose when this particular server’s virtual machines server went into production. The IT manager explained:

“The original server had been a two-processor hyper-threaded system, and it was moved to a single processor VM. After adding a second virtual processor, it ran at an acceptable level.”

This company has done successful P2V migrations since then and plans to do more.

Here’s hoping your P2V moves are blooper-free. If they aren’t, share your goofs with us by commenting or emailing me at We can all learn from your bloopers and chuckle at the same time.

April 5, 2007  9:11 AM

Generated an update script for ESX 3.0.0 and 3.0.1

Kutz Profile: Akutz

In my last post I showed you how to get all the available updates for an ESX 3.0.0 and 3.0.1 server. This post showcases a script that will generate the commands you need to run to update your server with the downloaded patches in the order that they were released. The script, newupdates, examines installed updates and ignored updates (via a file you specify) and generates esxupdate commands that can be used to update an ESX server in one fell swoop. This script can be downloaded from

Note: The ignored updates file is simply a file with a list of ESX patch numbers, IF or line delimited.


# require arguments or print usage
if [ “$1” == “” ]
echo “Usage: newupdates [-r REPOS_PARENT_PATH]

exit 1


# get the options

while getopts r:i: o

do case “$o” in



[?]) echo “Usage: newupdates [-r REPOS_PARENT_PATH]

exit 1;;



# get a list of installed updates

INSTALLED_UPDATES=`esxupdate query |
grep -io “ESX-[[:digit:]]*”`

# get a list of the updates in the parent repository path


# if the user defined an ignore file then read its contents

if [ -f “$IGNORE_FILE” ]




for R in $REPOS


# get the update/patch number from the repo

# directory name (strip the date prefix off)

UPDATE=`echo $R | sed “s/(.*-)(ESX-.*)/2/gi”`

# check to see if the update is already installed

echo “$INSTALLED_UPDATES” | grep -ioq “$UPDATE”


# if the user specified an ignore file, check

# to see if the update should be ignored

if [ -f “$IGNORE_FILE” ]


echo “$IGNORED_UPDATES” | grep -ioq “R$”





# generate an update command for the repo if it is neither

# installed or ignored

if [ “$R_INSTALLED” != “0” ] && [ “$R_IGNORED” != “0” ]


U_COMMAND=”esxupdate -nr file:$REPOS_PARENT_PATH/$R update”

# strip any double slashes out of the update command.

# the esxupdate utility barfs on them

U_COMMAND=`echo “$U_COMMAND” | sed “s//////gi”`




April 5, 2007  9:05 AM

Virtual Iron Offers up Performance Benchmark

cwolf Profile: cwolf

Following Simon Crosby’s release of a XenSource performance benchmark, I began to needle the folks at Virtual Iron about publishing a benchmark of their own. In short time, Chris Barclay, Virtual Iron’s Director of Product Management, sent me some numbers with his blessing to make public.

Their benchmark was based on the Windows Server 2003 OS running on an Intel Xeon 2.66GHZ dual socket/dual core server, with a 1333MHz FSB and 4GB of DDR2 667MHz RAM. For their tests, 1GB of RAM was allocated to the OS and the VM connected to raw SAN storage. So the test environment, in my opinion, is very fair.
Now onto the results…



Virtual Iron


SPECInt 2000




netperf tcp stream send




netperf tcp stream receive









     Network (MB/sec)

     Disk (MB/sec)

     Disk (Transfers/sec)










So overall the Windows Server 2003 VM was able to perform at or below a 3% performance degradation. The Virtual Iron tests followed the same benchmark pattern used by VMware. If you would like to see the VMware results and also get more detail on what each individual benchmark is testing, take a look at VMware’s document “A Performance Comparison of Hypervisors.” Keep in mind that the Xen performance numbers in the VMware paper are under significant debate, with most of us (myself included) seeing Simon Crosby’s Xen benchmark numbers as being more accurate.

Throughput degradation has been very important in many of the virtualization projects that I have been involved with, so having some hard numbers for performance comparison between VMware, XenSource, and Virtual Iron is extremely helpful. I’m hopeful that we’ll see a similar benchmark from Microsoft once the Windows Server Virtualization (WSV) service is available in Longhorn Server, or even for Microsoft Virtual Server 2005 R2 SP1 for the time being. If not, I’ll churn WSV or Virtual Server through the VMware benchmarks and post some numbers myself.

~Chris Wolf

April 4, 2007  8:32 AM

Getting ESX 3.0.0 and 3.0.1 updates

Kutz Profile: Akutz

It can be a real hassle to download updates for ESX 3.0.0 and 3.0.1. Wouldn’t it be nice if there was some program that could take care of this for you? Now there is! Introducing my patented (not really) chk4updates script. Using arcane majiks and other-wordly sorceries, I have divined through intense enchanting a process by which you can grab all available updates for ESX versions 3.0.0 and 3.0.1. I have included the script inline below, and you can also grab it from


# debug level. set to ‘1’ to have extra debugging
# information sent to stdout.

# the version of esx server to download patches for
# valid values are ‘3.0.0’ and ‘3.0.1’

# set to ‘1’ to download tarballs
# set to ‘0’ to just have a message printed to stdout

# the directory to place the downloaded tarballs in

# get a list of the already downloaded patch files

if [ “$DEBUG” == “1” ]

# get a list of the updates
wget –quiet

# parse the updates list for the actual links
PATCH_LINKS=`cat vi3_patches.html

| egrep -io “[[:digit:]]*-patch.html”`

# remove vi3_patches.html
rm -f vi3_patches.html

# iterate through the patch links
# get the patch information
wget -q $LINK

# parse the actual file name from the patch link
LINK_FILE=`echo $LINK | sed “s/(.*)(/)([^/]*.html)/3/g”`

# determine if this is an esx 3.0.0 or 3.0.1 patch
egrep -ioq “Download Patch ESX-[[:digit:]]*
for VMware ESX Server 3.0.0″ $LINK_FILE
egrep -ioq “Download Patch ESX-[[:digit:]]*
for VMware ESX Server 3.0.1″ $LINK_FILE


# 3.0.0
if [ “$IS_300” == “0” ] &&
[ “$ESX_VERSION” == “3.0.0” ]

# 3.0.1
if [ “$IS_301” == “0” ]
&& [ “$ESX_VERSION” == “3.0.1” ]

if [ “$DEBUG” == “1” ]

if [ “$GET_TARBALL” == “1” ]
# get the link to the tarball we want to download
TARBALL_LINK=`cat $LINK_FILE | egrep -io “http.*gz”`

# strip the extraneous stuff of the link
# so that only the name of the tarball remains
TARBALL_LINK_FILE=`echo $TARBALL_LINK | sed “s/(.*)(/)([^/]*gz)/3/g”`

# parse the patch release date from the patch html
PATCH_RELEASE_DATE=`cat $LINK_FILE | egrep -io “<b>released .*</b>” |
sed “s/(<b>released )(.*)(</b>)/2/gi”`

# get the name of the directory that is
# created once the patch is decompressed
| sed “s/(.*)(.[^.]*$)/1/gi”`

# prepend the patch release date to the decompressed directory

# determine if the patch has already been downloaded by
# comparing the name of the patch to the files that
# exist in the output directory

if [ “$DEBUG” == “1” ]

if [ “$ALREADY_DOWNLOADED” != “0” ]
&& [ “$TARBALL_LINK” != “” ]

if [ “$DOWNLOAD_TARBALLS” == “1” ]
echo “fetching $TARBALL_LINK_FILE …”

# fetch the patch

# move the patch to the outpdir directory

# change the working directory to the output directory

# decompress the patch

# rename the decompressed patch dir to include the
# patch release date

# remove the patch tarball

# change into the previous working directory
cd –

# remove the patch information
rm -f $LINK_FILE

April 3, 2007  12:38 PM

Virtual machine back up licensing challenges

Ryan Shopp Ryan Shopp Profile: Ryan Shopp

Recently, I chatted with a sys admin about his experience with VM migrations and management challenges. His primary goal right now is creating backup copies of his VMs as part of his disaster recovery plan. The biggest hurdle? Licensing. Or rather, the cost of licensing, because he wants to avoid the cost of treating each VM like a physical box.

Right now, he’s backing up the VMs that don’t need to be online 24/7 by shutting down his guest OSes, backing up the virtual hard disks using Backup Exec, and restarting. Costs are less for backing up in this fashion, because it only takes one Backup Exec license to backup the files on the host.

But for his mission critical VMs, he’s stuck between a rock and a hard place. Take the VM offline, install the Backup Exec Client and pay the license fee for *each* VM (which could get pretty expensive pretty quickly), or… don’t back up.

He’s starting to research snapshots as a lower-cost solution for his mission-critical VMs, but when I last checked in hadn’t gotten too far in the process. Any suggestions? Leave a comment and I’ll pass your suggestions along.

April 2, 2007  9:33 AM

A little off topic – Parallels Desktop for Mac “Coherence”

Joseph Foran Profile: Joe Foran

Ok, this blog is normally about server virtualization, but I thought I might digress into a slighly-off-topic realm today, to bring some opinions on a product I’ve been using for some time, called Parallels Desktop. This is the program that gets the little text box at the bottom of the “Run Windows and Mac” commercials. You know the ones, John Hodgeman, with the many-titled commentator/reporter from The Daily Show and Justin Long, the guy from about a million small roles. I’ve been using it for some time, having the need to run such things as Visio and Access, as well as my custom MMC for all our Windows server management, and VirtualCenter to manage our VMware environment from my Mac. Parallels is one of the two ways to get windows working on your mac, and until the recent release of VMware’s Fusion beta, the only virtualized way, and I use it all the time. I don’t play the upgrade game with applications that I need on a daily basis, so I’ve been running a slightly older build for a while, at least until my curiousity about Coherence got the better of me.

So, I fire up my XP machine after the upgrade, and click the button for Coherence. What do I see – A Control-Alt-Delete box in the middle of my Mac. Hilarity. True hilarity. Function-Control-Alt-Delete and I’m in. Yuck. My first non-enjoyable part of Coherence… lets hope that it’s my last. The Windows taskbar has just invaded my Mac desktop. Start button, quick launch, and system tray… my favorites (not!). Easy to dispel… just a simple click in the options box and they’re gone. Getting back to the Start button is easy – just a double-click on the parallels icon and I have my menu. Some re-arrangement of what I have pinned there vs. what I used to keep in my quick launch tray, and I’m ready to roll.

Windows command prompt. Adobe Designer. IE7. All working. Everything is working. My shared folders between the virtual and the physcial mac means I can move docs back and forth. What about drag and drop? Works like a charm. It even pops up a box that lets me choose what access I give Parallels. The whole kit and kaboodle looks like this. You can see that the app creates icons in my dock, just as if they were Mac programs. You can see a couple of Mac-native apps running, like iChat, Grab, Thunderbird, and Firefox. I fired up my VMware Server management console to connect to the test lab, and sure enough, it worked great. It feels native. It’s seamless once I’m past that warning box. It’s simplicity is brilliant.

What it means for the deskop is obvious. It means no more worrying about running legacy apps. Got a must-use app for Win9x? Need to run a Windows app on a Linux desktop, and WINE can’t run it? All possible… not with Parallels specifically, but the technology in general. It won’t be long before other products are out there that do these things, that take advantage of this huge conceptual leap in client-side application virtualization.

Imagine a Linux client, talking to a Citrix server that is hosting Mac and Windows apps, and sharing them via http. Wow.

Off-topic a bit? Yes. But keep this kind of converged (Coherent?) virtualization approach in mind as the line between operating systems continues to blur in the server market. Will we one day see one server serving applications to end-users, customers, etc. from a virtualized environment similar to Coherence? I wonder what this means for streamed applications, like those pushed out via Citrix? Will Citrix take advantage of this kind of technology in its own app virtualization products? One can only hope. What will this do for sandboxed applications? It’s a bit off in the future, but expect this sort of innovation to make it’s way upstream in any number of ways.

March 30, 2007  5:10 PM

Steps along the path of enlightenment

Shayna Garlick Shayna Garlick Profile: Shayna Garlick

It’s great to see VMware finally embrace Paravirtualization. As a result of a tremendous community effort to develop a common interface between Xen and VMware, that benefited from the collaboration of VMware, IBM, Red Hat, Novell, XenSource, Intel and many contributors, the first common API between the two hypervisors will appear in Linux kernel 2.6.20.  It’s a pity VMware’s PR didn’t acknowledge the community contribution though…

Here’s what’s going on:  The issue at hand is Paravirtualization – a technique of modifying an operating system so it will run optimally on a hypervisor.    The best known example of a paravirtualizing hypervisor is the Xen hypervisor, but the concept originates from IBM mainframe OSes from the late 70’s and has been widely used on vertically integrated (hardware plus software) systems for some time.  Paravirtualization was first introduced to the x86 architecture by the Xen project instead of the binary patching technique used by VMware, because (as VMware recently acknowledged) it offers significantly improved performance for virtualized guests.

But Paravirtualization has a bit of a downside – it requires modifications to the guest operating system to enable it to co-operate with the hypervisor.  VMware gets around this today using binary patching, which modifies the guest “on the fly” by rewriting the code. Intel VT and AMD-V help a lot- but not with I/O.  The performance benefits of paravirtualization have led all x86 OS vendors to adopt paravirtualization for their next major OS release (though Microsoft calls it “enlightenment”).  Xen-style paravirtualization also allows OS vendors to ship the hypervisor with the OS – something VMware understandably isn’t that keen on.  In the case of Linux and Solaris this is achieved through the inclusion of the Xen hypervisor, and in the case of Microsoft the forthcoming enlightened Longhorn Server OS will be augmented at some point with the Windows Hypervisor, which is architecturally very similar to Xen.

Today many distros are delivering Xen as an embedded hypervisor.  The Xen hypercall API paravirtualization hooks are added by the vendor to their kernel once they select a particular version from  This is tedious/painful, and the obvious right way to do this is to have the hooks included and maintained by    The Xen project was happily working away (albeit rather slowly) to get the Xen hypercall API upstreamed to, when VMware introduced VMI at OLS in 2005.    VMI is a lower level interface than the Xen hypercall API.  It’s much more suitable to a binary re-writing hypervisor like VMware’s.   But it deserved serious consideration because it offered a useful new feature – the same kernel could run native and virtualized.   But VMI is closed source – an ABI not an API, which is a serious problem for many in the open source community.  Everyone agreed that having a single interface for multiple hypervisors would be preferable to having many.   So, at the Ottawa Linux Symposium in 2006 the Xen project began to work with VMware to develop a common set of kernel hooks that could accommodate the VMI ABI and the open source Xen hypercall API.  Since then, there has been a very positive effort on all  sides, with  IBM, HP, Red Hat, Novell, and many other core developers playing a key role in getting the work done.

So, what’s in 2.6.20 is a common API called paravirt_ops, developed collaboratively by a group of contributors, and the first implementation of the VMware VMI interface into paravirt_ops comes in 2.6.21.  The Xen interface into paravirt_ops should follow shortly, likely in the 2.6.22 time frame.  The Xen API is more extensive than VMI, and the work is taking a bit longer to get done.   Once this set of changes is complete, future Linux kernels will have the paravirtualization hooks built in, which will dramatically simplify the kernel development processes of the distros.

The bottom line: Future Linux kernels will have a common hypervisor interface called paravirt_ops that will allow Linux to run on either Xen or VMware with high performance.   Through XenSource’s relationship with Microsoft, it’s reasonable to expect that these Linux kernels will have the ability to run as first-class “enlightened” guests on the future Windows Hypervisor.  Of course, all of this is only relevant to the market when the next major enterprise Linux distributions take new kernels to market that include paravirt_ops, but overall it is good to see harmony emerging in this particular piece of the virtualization landscape.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: