I met with some of VMware Inc.’s desktop virtualization reps yesterday to discuss the next version of VMware View (4.0), which is due out next week, and learned there is a correlation between desktop virtualization adoption rates and the 2009 H1N1 flu (formerly known as Swine Flu).   

Though he couldn’t share specific numbers, Raj Mallempati, the desktop virtualization marketing manager, said VMware’s VDI adoption rates increased in direct correlation with H1N1 flu. In fact, he said VMware sold a VDI license in Australia for the first time when H1N1 started alarming people there this summer.

“It makes you think, did VMware invent swine flu?” Mallempati joked.

They saw the same spike in desktop virtualization adoption when the SARS virus hit a few years ago, he said.

VMware claims to have around 1.5 million VMware View license holders so far, which is between 6-7% of the company’s revenue, according to a VMware rep.

While these correlations could be coincidental, deploying desktop virtualization certainly makes sense for corporations that don’t want to lose productivity every time a deadly virus pops up (gotta love capitalism). With VDI, employees who don’t want to come into work because they are afraid of the sneezer in the next cubuicle can access their desktops from home. Or if they are sick, they can still work from their quarantine and not infect everyone else in the office.

So, although I haven’t seen any marketing campaigns fear-mongering customers into buying desktop virtualization as a way to avoid H1N1, it wouldn’t surprise me.

There are two methods for backing up a VM, traditional backup methods that install an agent inside the OS and back it up file-by-file and image-level backups that back up the single virtual disk VMDK file. Image-level backups are usually done by backup applications that are designed to specifically back up virtual machines, like Veeam Backup and Replication. These applications use the snapshot feature that is built in to VMware to stop disk writes to the virtual disk so it can be safely backed up. Backup applications read the original virtual disk file which is now read-only as new disk writes get written to a newly created delta virtual disk file. Once the backup application has read all the blocks from the original virtual disk file the snapshot is committed, which takes the data from the delta virtual disk and writes it to the original virtual disk. Once this is complete the delta virtual disk is deleted.

In virtual environments it’s more efficient to do image-level backups, but they also offer more options when you need to do a restore. If you want to restore the whole VM to a previous state you can simply restore the whole virtual disk file. Or, if you want to restore individual files, you can easily do this also. When using a backup application for virtual environments the process for individual file restores in virtual machines is easy and straight-forward. For example, Veeam Backup and Replication can quickly mount the backed-up VM disk file from the backup location so it is available for browsing; the files that need to be restored can then be selected and copied back to the original VM, a process that takes only minutes.

VMware’s own VM backup product, VMware Data Recovery can also do individual file restores via a command-line application that mounts the virtual disk from a selected VM/restore point to a Windows drive letter so the files can be accessed and copied. Even if you use simple scripts to back up a VM by copying its disk to another storage location you can restore files easily enough. Virtualization makes this fairly simple as you can just add the backed-up virtual disk file to another helper VM as an additional disk, browse the files on that disk and copy and paste them back to the original VM.

As you can see restoring individual files to virtual machines is a simple and easy process when using a backup application designed to work with virtual machines. It may not be the same process that is used with physical servers and traditional methods, but it is definitely possible and not all that difficult. If anything virtualization makes backup and recovery easier by providing many unique alternative backup and recovery methods then are not possible with traditional methods.

After doing some research I found out they were part of the new vApps feature in vSphere. I’ve heard a little about vApps but never looked at them in depth, so I thought I would take the time to research them and write about them.

We’ll come back to IP Pools in a bit. First we’ll cover what a vApp is and how they work in vSphere. VMware’s definition of a vApp is below:

A logical entity comprising one or more virtual machines, which uses the industry standard Open Virtualization Format to specify and encapsulate all components of a multi-tier application as well as the operational policies and service levels associated with it.

A vApp is basically a resource container for multiple virtual machines that work together as part of a multi-tier application.

An example of a multi-tier application is a typical Web-based application where you might have three tiers: Web, application and database; which are often run on three separate servers. For example, you may have Microsoft IIS running on one server (tier 1), IBM WebSphere running on another server (tier 2) and a Microsoft SQL Server running on a third server (tier 3).

The three applications on each server all work together and are mostly dependent on each other for the application to function properly. If one part of the tier became unavailable, the application will typically quit working as it relies on all the tiers for the application to work.

Virtualization can introduce some challenges with multi-tier applications. For example, if one tier is performing poorly due to resource constraints on a host, then the whole application will suffer as a result. Another challenge comes when powering on a host server, as often times one tier relies on another tier to be started first or the application will fail.

VMware introduced vApps as a method to deal with these problems by providing methods for setting power-on options, IP address allocation and resource allocation, and provide application-level customization for all the virtual machines in the vApp. When you configure a vApp in vSphere you specify properties for it, including CPU and memory resources, IP allocation, application information, and start order, as shown below.

Once you are done configuring a vApp, you can add virtual machines (VMs) to it by dragging them using the vSphere client. You can also create resource pools inside of them and nest vApps inside of vApps. If you edit the settings of a VM, select the Options tab, and then select vApp Options, you can enable the vApp functionality for the VM and set individual vApp options for the VM. Once you have created a vApp you can easily export it in Open Virtualization Format (OVF) format, as well as deploy new vApps from one that are already created. To use vApps you must have a Distributed Resource Scheduler-enabled cluster; all of the meta-data information for a vApp is stored in the vCenter Server database.

So now that you know what a vApp is, back to what IP Pools are. IP Pools, as you might have guessed, are pools of IP addresses that you can associate with vSwitch port groups. They essentially act as Dynamic Host Configuration Protocol (DHCP) servers to assign IP addresses from the pool to a VM, so essentially the vCenter Server is acting as a DHCP server. When you configure an IP Pool you specify a range of either IPv4 or IPv6 addresses, DNS and proxy settings, and finally select which vSwitch port groups that the pool will be available to.

To configure an IP address range you need to do it in the following format with the starting IP address, a pound sign and the number of IP addresses, like this:

172.20.20.155#10, 172.20.20.175#5

So the above range would make the following IP addresses available in the IP Pool:

172.20.20.155 - 172.20.20.164 and 172.20.20.175 - 172.20.20.179

Once you have an IP Pool configured, you can assign it to a vApp by editing its properties and changing the IP Allocation Policy from Fixed to Transient. For more information on configuring and using vApps and IP Pools check out the resources below.

• VMware vApp Developer Blog
• vApps and advanced VM templates in vSphere 4 (VMworld 2009 attendees or subscribers only)
• vSphere Basic System Administration (Chapter 10)

Screenshot of PowerWF

In case you haven’t heard, a little company called Devfarm Software has been working on a product called PowerWF (pictured, right). They had a great demo at VMworld 2009, and I was so impressed that I gave them five minutes at the end of my own breakout session to do a demo for my audience.

I’m a big PowerShell and PowerCLI nut, so why do I care about some graphical user interface (GUI) application? Well, there are two answers to that. One, PowerShell is an automation engine, not just the command-line shell. There are plenty of tools out there which use PowerShell behind the scenes to enable the user to automate tasks. Exchange 2007’s admin console and VESI (which I need to spend some time talking about soon) are just two examples. Just because I like to write scripts doesn’t mean I don’t respect the GUI.

The second answer is that PowerWF is sort of like scripting with Visio diagrams. At its heart is the concept of a workflow. A workflow is a graphical representation of a series of steps which belong to a task which you want to automate. There’s hundreds, maybe thousands of these steps available. They arranged on the left of the window in a palette with several collapsible headings such as FlowControl, Visualization, and of course, VMware PowerCLI.

Yeah, now you can see where I’m going. The Devfarm guys have been doing VMware solutions for a while, so it is natural that their workflow tool integrates very heavily with VMware’s products. In fact, you can do stuff with VIX, not just PowerCLI, so that opens some possibilities for you guys using that API to automate VMs and guests.

Anyway, I’ve been really vague so far, so I feel the need to give some concrete examples of what the heck PowerWF can do. I have to admit that I’m still a newbie with it, so I’m going to cheat here and point you to some videos that the Devfarm guys made:

• Transcribing PowerShell scripts as Worfklows
• Convert PowerShell/PowerCLI to Workflow

PowerWF is really, really big. I mean, it can do a lot of stuff. It can cover most any task that you would do with PowerCLI. It can run PowerShell scripts. It can output information in a variety of ways, such as console, text file, RSS feed, HTML, or datagrid. It can also do some really interesting things I haven’t even fully grokked yet such as create WMI classes and Web services.

In order to help you get started working with PowerWF you may want to check out these blog posts that the team is working on:

• What is a Workflow Anyway?
• PowerWF Layout and Terminology

As I said, I’m just getting started with the application myself, so I’ll try to report back to the blog with my findings as I get deeper into the product.

So how do you find I/O bottlenecks? You start with the built-in performance monitoring tools in the vSphere Client on the Performance tab. The esxtop command line utility can also be very helpful for looking at real-time metrics.

Using the Windows built-in Performance Monitor (perfmon) utility is usually not recommended, because some statistics may not be accurate because they are not aware of the underlying virtualization layer. VMware, however, has recently added some VM specific performance counters to perfmon that are aware of the virtualization layer and provide accurate metrics. There is also a utility introduced in ESX 3.5 called vscsiStats that can be used for storage performance analysis.

Understanding how to properly use these built-in tools to collect data and interpret it can take some practice. There are a number of free and paid third-party tools that can help you with this  such as Reflex Systems VMC, Veeam Monitor, Vizioncore vFoglight, eG Monitor, Hyper9 and Netuitive Si.

If you’d like to try and learn more about performance troubleshooting that will help you identify  I/O bottlenecks, check out some of the below resources:

• Understanding Performance
• Using esxtop to Troubleshoot Performance Problems
• Interpreting esxtop Statistics
• Using the esxtop tool to identify VMware ESX memory use
• Using Perfmon For Accurate, ESX Performance Counters
• Performance Troubleshooting for VMware vSphere 4 and ESX 4.0
• Understanding VirtualCenter Performance Statistics
• Using vscsiStats for Storage Performance Analysis
• Performance Analysis Methods
• CPU Performance Analysis and Monitoring
• Memory Performance Analysis and Monitoring
• Network Performance Analysis and Monitoring
• Storage System Performance Analysis with Iometer
• VMware Infrastructure - Performance Optimization & Troubleshooting (VMware ESX / VMware ESXi) (VMworld 2008 - free registration required)
• Intepreting Performance Statistics in VI3 (VMworld 2008 - free registration required)
• Troubleshooting virtual machine performance issues (KB Article)
• Using esxtop to identify storage performance issues (KB Article)

So how do you explain it to someone for the first time? I find that using analogies that anyone can relate to is a good way to explain things to people. Before I attempt a virtualization analogy I’ll try explaining it in basic computer terms.

Virtualization software, also called a hypervisor, emulates computer hardware allowing multiple operating systems to run on a single physical computer host. Each guest operating system appears to have the host’s processor, memory, and other resources all to itself. The hypervisor, however, is actually controlling the host processor and resources and allocates what is needed to each operating system, making sure that the guest operating systems (called virtual machines) cannot disrupt each other.

There are two types of x86 virtualization: bare-metal and hosted. Sometimes these types are referred to as Type-1 and Type-2 hypervisors respectively. Bare-metal means the virtualization layer (hypervisor) installs directly onto a server without the need for a traditional operating system like Windows or Linux to be installed first. “Hosted” means that an operating system must first be installed on a server, and the virtualization layer is installed afterwards, just like an application.

Bare-metal hypervisors include VMware ESX, Citrix XenServer and Microsoft Hyper-V Server. Hosted hypervisors include VMware Workstation, Fusion, VMware Player and VMware Server, Microsoft Virtual PC and Microsoft Server, and Sun’s VirtualBox. Some of the differences between hosted and bare-metal hypervisors are listed below.

Hosted hypervisors

• Requires a host operating system (Windows/Linux/Mac), installs like an application.
• Virtual machines can use all the hardware resources that the host can see.
• Maximum hardware compatibility as the operating system supplies all the hardware device drivers.
• Overhead of a full general-purpose operating system between the virtual machines and the physical hardware results in performance 70-90% of native.

Bare-metal hypervisors

• Installs right on the bare metal and therefore offers higher performance and scalability but runs on a narrower range of hardware.
• Many advanced features for resource management, high availability and security.
• Supports more VMs per physical CPU then hosted products.
• Because there is no overhead from a full host operating system performance is 83-98% of native. There is a small bit of overhead from the virtualization layer of the hypervisor

Why is virtualization such a great thing? Because most computers do not fully utilize the resources (memory, CPU, disk, network) that they have which is very wasteful. Would you rather have 10 computers that are all using less then 20% of their total resources, or three computers that are using 70% of their resources?

You might think you could avoid this by simply installing more applications on one computer but this is often a bad idea as the applications may conflict with each other and cause problems, and a single OS crash will take down all your applications. Virtualization solves this by allowing the applications to run on the same physical computer, but separates them by allowing each one to have its own isolated guest operating system.

So those are the basics on what virtualization is. Now on to an analogy.

Imagine computers as cars on the road in motion. Each car has it own resources, such as fuel, heat/cooling, radio, etc. Most cars are never filled to capacity, and many have only one person in them which is wasteful.

Imagine virtualization as a bus, instead of many people driving in many cars you now have many people being moved around by a few buses. A person may only ride one bus at a time, but if a bus becomes inoperable due to a flat tire or an engine problem, the people may simply get off and transfer to another bus that has unused seats. In virtualization, this “transfer” happens because of features like High Availability (HA).

A person may also hop from one bus to another if it becomes too crowded while it is moving. In virtualization, this is called VMotion, if you’re using VMware, or Live Migration if you’re using Hyper-V. By utilizing buses that hold more people instead of cars, fewer resources are wasted, while all the people still get where they are going. Buying and operating one bus instead of 10 cars is a lot cheaper and more efficient.

Contest!

I’m always interested in hearing how other people describe virtualization in their own words so I’m going to do a contest. Submit to me by email either your own definition of what virtualization is or an analogy describing it, and I’ll give away a copy of Train Signal’s great new vSphere video training course to the one I think is the best.

I’ll also pick a random entry and give away a copy of my book to them.

Entries should be at least a few sentences long (preferably a paragraph or two), and must be received by Friday, Oct. 23rd. I’ll publish all the entries in a future blog post so everyone can benefit from other peoples definitions of virtualization.

• Virtualization platforms
• Virtualization management
• Virtualization security
• Hardware for virtualization
• Backup and storage for virtualization
• Desktop virtualization

The call for entries is open now through Nov. 13, 2009. Products qualify if they were released between Nov. 1, 2008, and Nov. 1, 2009 (including beta). Click here for deadlines, details and criteria on Products of the Year. And click here for the direct link to our form.

In my experience with desktop support many people become so used to GUIs that they don’t want to consider any command line work, even for basic things like running ipconfig at the MS DOS prompt. Others feel that only purists use command line tools, bringing to mind a Hollywood image of some genius hacker hunched over a keyboard, surrounded by empty pizza boxes, writing code in a bunch of terminal windows in his own compiled operating system. Alas, that is not always the case (sometimes it’s take out Chinese boxes).

Most of us “Average Joes” use a combination of both.  It is important to know how to use command line tools (even if you have to double-check the syntax most of the time).

Here is an experience I had a couple weeks ago that illustrates the value of knowing and using the VMware command line tools in the Service Console.

Environment: ESX 3.0 hosting multiple VMs.

One individual was patching the operating system on a virtual machine (VM), and then rebooted it. The VM didn’t come back online. I then tried to help the person by using the GUI to force the VM to power off.

The status of the power-off command hung at 95%.

One green tea latte and an hour later the operation is still at 95%.

Uh oh.

At this point, I had to consider my options. I could either use the GUI to power-down the other VMs and reboot ESX.  No good, as they are production VMs, and cannot have downtime. Or, I could use a command-line tool to try to reset the misbehaving VM.

For a command-line tool interface, I use PuTTY (free), customized with a glaring obnoxious orange background and black text. I find it easy to read and the colors helps me keep my concentration since it is so difficult to ignore.

To resolve this situation, I used

ps -ef|grep <servername>

to get the PID (process identifier) number. Then,

kill -9 <PID>

Next, back to the GUI to start up the VM, which worked without a hitch.

Resolution time, including looking up the command line tools to make sure I spelled them properly: less than 10 minutes.

There is no reason to be intimidated by command-line tools in the Service Console. They are not much different than DOS, or even older versions of NetWare. That said, it can take some time to become comfortable with command-line tools. One way that worked for me was finding an old computer, installing Linux, and using the terminal window to try different things such as browsing directories, finding files, and so on. There are also some terrific Service Console command line resources available on the Web, such as Mike Laverick’s RTFM Education.

Using command line does not make you a purist in the same way that using GUIs doesn’t make you a newbie. Both have their place, and the ability to use all the tools in our toolbox makes our lives that much easier!

Cheers!

Mak King

Even if you select the Show Hidden Devices option in Device Manager you will not see the old hardware devices as it only displays non-Plug and Play devices that are actually present in the system. Devices that were previously installed but are not currently present in the server are not displayed. What you need to do to display them is to set an environment variable before you launch the Device Manager application.

You might be wondering why you should even remove them in the first place. There are two reasons. The first is that it can cause conflicts when you attempt to assign IP addresses to any new network adapters that are discovered. This is because the old network adapters are still configured on the server and still have their original IP addresses even though they are not physically present. You will not see them, however, when you go in to your network connections, and if you attempt to assign the new network adapter the same IP address as the old ones you will get an IP conflict error message. To resolve this you need to remove the old non-present adapters which will release the IP addresses assigned to them. Microsoft has a KB article that provides more detail on this.

The second reason you want to remove them is that the old non-present device drivers still consume server resources even though the devices are no longer physically present. The resource usage is minimal but when you have multiple virtual machines (VMs) running on a single physical server the resources that are wasted can add up. Fortunately removing those pesky old devices is fairly easy, just follow these steps:

1. Open a Windows command prompt and type the following:

set devmgr_show_nonpresent_devices=1 

This sets an environment variable that tells the device manager to show all non-present devices.

2. At the same command prompt, type

 devmgmt.msc

…which launches the device manager application. Do not try and launch it through the graphical user interface (GUI) instead as the environment variable set in step one only applies to commands run inside the command prompt.

3. Once the device manager window opens, select View from the top menu, and then Show Hidden Devices.

4. If you expand the device categories you will see both present and non-present devices. The non-present devices are shown by the grayed-out icons next to the device names. You can remove them by right-clicking on them and selecting Uninstall. Unfortunately you have to do this one by one as you can not select multiple devices. Click OK at the warning message that appears. If you accidentally delete a device that is present, don’t panic — once you reboot, Windows will automatically install the device driver for it again.

As an example of how many non-present devices there are after a P2V, I removed over 60 of them from a VM that came from an HP DL360 server, as you can see in the images below.

The process of removing these devices can be tedious and time-consuming, especially if you have many servers to do, but is worth doing so your new virtual servers are clean and don’t carry over remnants from their previous hardware. Fortunately there is a way to automate this process; Mike Laverick has created some great scripts that will automate the cleanup of old hardware after a conversion, which are available here.

“Today 10ppl on vSphere course all very knowledgeable & working @ VMware partners though none of them knew about any of our blogs Reality check :-)”

It really surprised me that people who are really into VMware virtualization wouldn’t look to the Internet for information on technology that they use everyday. After all, what you know and learn is generally limited to what you read about and what you experience, both of which can be constrained based on the amount of time that you have to do them. Imagine multiplying that with very little effort simply by reading and benefiting from other peoples knowledge and experiences. Instead of spending hours or days trying to learn the intricacies behind VMware’s High Availability feature why not instead spend minutes learning from someone else who took the time to do so. It’s almost impossible to be an expert at every little thing when it comes to virtualization, some guys focus on storage, others on networking and so on. Learning from others is a great and easy way to expand your knowledge. It’s all about working smarter instead of harder.

Which brings me to the point of this post — VMware has an incredibly rich ecosystem of bloggers, authors and evangelists that write about their experiences with VMware technology everyday. To ignore this vast stream of valuable information is like walking through a gold mine and not picking up gold nuggets that lay on the ground. You’ll find information and tips on blogs that you will never find in any class or documentation. There are so many blogs I have a hard time keeping up with them myself, so I made it easy for anyone to find them all by creating my vLaunchpad where I maintain them all, including the top 20 blogs.

So here’s my challenge to you. If you’re reading this post you most likely know about VMware blogs and websites, why not help get the word out to others who may not know by taking the time and spreading the word? I see the same lack of awareness at local VMUG events; people are ignorant to all the great information available on the internet. I made a presentation at two of them that was all about resources for learning VMware where I highlighted all the great blogs and websites that are available.

Why not take your game to the next level and learn from masters like Chad Sakac, Duncan Epping and Scott Lowe. There is a lot of great information available out there and it can be yours with very little effort and no cost. If you’re serious about virtualization there’s simply no excuse for ignoring some of the best information available. If you want to make it even easier, check out Planet V12n.