February 24, 2009 6:29 PM
Posted by: Eric Siebert
, vCenter Server
Many people were hoping that VMware would use VMworld Europe to announce the release date of the next generation VI3 software called vSphere 4.0. While this didn’t happen, VMware did release the next maintenance version of vCenter Server 2.5, which is Update 4. This new release brings the usual bug fixes but also adds a new feature, performance overview charts.
Performance Overview Charts aggregate the normal VM and host performance charts into a single view of key performance metrics for CPU, memory, disk and network usage. This eliminates the need to have to look at each factor individually and helps to show trends or resource issues.
This new feature is a plug-in component to vCenter Server and uses Java, so to use it you’ll need to download the Java SE Development Kit 6u11 and install Java Development Kit 1.6. If you are using Oracle or SQL Express there are some additional steps that you need to complete. Below are the KB articles you should read if you want to use this new plug-in:
• KB Article on how to install the Performance Overview plug-in
• KB Article on using the plug-in with Oracle database
• KB Article on using the plug-in with SQL Express
One other interesting new feature is an update to the Virtual Machine Monitoring feature that is part of HA (High Availability) and will restart VMs on the same host if there is an guest operating system failure and the OS stops responding. This feature previously relied on a “heartbeat” provided by the VMware Tools application, which is installed on the guest OS. If that heartbeat were to stop, the guest OS would be considered defunct and automatically restarted.
The enhancement allows vCenter Server to not only monitor the heartbeat but also the disk and network activity on the VM. Together, these features add more failure triggers to better indicate a failed OS. One situation where heartbeats may stop for a short period of time is when the VMware Tools application is being upgraded. This would previously cause the VM to restart, as the short stop time would indicate a failure. The added requirement vCenter to report dead network and disk activity for a predetermined I/O stats interval ensures that the VM guest OS is truly in a failed state (i.e. Blue Screen of Death) before it’s restarted.
As usual, it’s a best practice to read the release notes before you install the upgrade to vCenter Server. This upgrade includes updated plug-ins for both Converter and Update Manager, so you should read the release notes for them also. Finally, once you install the upgrade you will have to manually update your plug-ins in the VMware Infrastructure Client on your workstations that use them.
February 23, 2009 9:19 PM
Posted by: Eric Siebert
Now that Citrix is giving away the XenServer hypervisor for free (albeit sans advanced features), is it time for VMware to follow suit by giving away the base edition of VMware ESX as it did with ESXi?
In my open letter to VMware last month, one of my points was that VMware should give away ESX. It is the exact same hypervisor as ESXi with a few minor differences. Architecturally the main difference between the two is that ESX comes with the full service console and ESXi comes with the limited Posix-based BusyBox management console. The full service console that comes with ESX is useful for running scripts and other management functions but is not essential for VM operations and administration.
You can see from the below graphic the different editions of VMware ESX and what features come with each of them.
All of the editions allow you to use ESXi instead of ESX if you choose to do so. Currently the Foundation edition of both ESX and ESXi (which is the base version) includes a vCenter Server agent and support for Update Manager and Consolidated Backup. So why not give a base edition of ESX away for free and not include those components? If someone wants the more advanced features then they can upgrade to the other editions. Or why not take it a step further and include the vCenter agent, Update Manager and Consolidated Backup with the free version?
If VMware gave away ESX and included High Availability, VMware’s free product would be one step ahead of Citrix’s.
So how will VMware make any money if it gives away more of its products for free? First, having the aforementioned add-on features available for free would require anyone that wants to use them to purchase vCenter Server, which is necessary for them to work.
VMware can charge for users that want the vMotion/Storage vMotion and Distrubuted Resource Scheduler and Distributed Power Management features which most enterprises are going to want. It can also charge for vCenter Server which is a must have in all large environments. Additionally, VMware has many more automation and management products that it can sell, not to mention support subscriptions for all of its products.
In the increasing competitive virtualization arena, VMware needs to do more then just have a better product to attract and retain customers. Most companies are concerned about cost, and with other vendors giving away their products VMware needs to do the same thing. Giving away ESX and some basic features will help them to compete and eliminate the cost arguments that other vendors are constantly making when comparing their product to VMware’s.
February 23, 2009 5:51 PM
Posted by: Gabrie
, Gabrie van Zanten
, vCenter Server
, virtual desktop
, VMworld Europe
After much anticipation, VMworld 2009 in Cannes has finally begun. After talking about it for days, seeing lots of blogs from people planning to visit VMworld, we’re finally on the go.
Andy Hunt, Vice President for the EMEA Partner Organization from VMware kicked off partner day today, which traditionally is the opening day for VMworld. He welcomed us all and thanked us for coming to VMworld despite that poor economic climate. He was very happy to see about 1,500 visitors at partner day, and was happy to announce that there should be about 5,000 visitors at VMworld in total.
Next to take the stage was CEO of VMware Paul Maritz. Maritz explained that VMware has a budget of $515 billion dollars for research and development (R&D) and that the team of engineers in VMware’s R&D department is larger than any team he has ever worked with while working for Microsoft.
This may sound like a typical blanket statement, but keep in mind that Maritz has worked with Microsoft for 14 years where, amongst other functions, he has been Vice President of the Platform Strategy and Developer Group where he oversaw the development and marketing of System Software Products (including Windows 95, Windows NT and Windows 2000).
Maritz also showed us that in today’s world, IT departments use 70% of their budget to just keep the lights on – a mere 30% is used for research and competitive development. VMware sees potential to move that 70% around, and is focusing on ways to reduce typical running costs so that more money is available for R&D.
The three biggest VMware initiatives at the moment are:
- Foundation for the cloud. This is an area where the Virtual Data Center OS (VDC-OS) will play a big role. VDC-OS will be about service and policy (namely availability, security and scalability) on one side and about aggregation (with products like vCompute, vStorage, vNetwork) on the other side. VDC-OS should become the new software mainframe.
- Choice and cloud federation. This is all about vCloud working on and with standards to provide high interoperability between in-house and external clouds.
- Desktop as a service. The key is to provision to users, not to provision devices. Depending on the type of worker, users could get a virtual desktop or a client-side hypervisor.
Maritz also announced the new name for VDC-OS. It’s officially vSphere. The name didn’t really come as a surprise because rumors have been humming for a few weeks now, but finally all those people under a non-disclosure agreement can shout it out loud: VSphere is here! (But not entirely, the name may be official but the product isn’t here just yet.)
Three more announcements Maritz made:
- Collaboration with Teradici. On the desktop side, VMware has announced the collaboration with Teradici to improve the VDI protocol. Tomorrow, VMware will announce the VMware Client Hypervisor.
- VCenter Server Heartbeat. Maritz announced a new vCenter Server add on called vCenter Heartbeat. Through vCenter Heartbeat it will be possible to make your vCenter fully redundant. VCenter Heartbeat is to be expected in mid March.
- Additional security. VMware vShield zones will make it possible to deliver security and compliance to the internal cloud.
As a consultant, these are all products that I expect to see at customer’s sites very soon, and I welcome the product announcements.
These releases should give VMware a new boost in the battle for the data center. Bring it on!
February 19, 2009 9:42 PM
Posted by: Eric Siebert
, Eric Siebert
x86 computer operating systems utilize protection rings that provide a range of protection levels called rings in which code can execute. These rings are arranged in a hierarchical manner from the most privileged (most trusted, Ring 0) to least privileged (least trusted, Ring 3) as shown below.
The enforcement of these rings is done by the processor (CPU) which uses different operating modes that place restrictions on the operations that can be performed by the process currently running in the CPU. Ring 0 has the highest level privilege and is where the operating system kernel normally runs. Code executing in Ring 0 is referred to as running in kernel mode, which is also known as privileged or supervisor mode. All other code such as applications running on the operating system operate in less privileged rings, typically Ring 3. With non-virtualized systems, the operating system runs in privileged mode in Ring 0 and owns the server hardware, applications run in Ring 3 with less privileges as depicted below.
On virtualized systems the hypervisor or Virtual Machine Monitor (VMM) runs in privileged mode in Ring 0 and the VM’s guest operating system must instead operate in Ring 1 as depicted below.
This can cause problems however because most VM guest operating systems are designed to run in Ring 0. To overcome this, the VMM fools the VM’s guest operating systems into thinking they are running in Ring 0 by trapping privileged instructions and emulating them by the VMM. This emulation causes a slight bit of overhead and is the reason that VM performance can typically only achieve up to 98% of native performance compared to physical servers. To overcome this, newer CPUs like the AMD-V and Intel-VT have features that were specifically designed for virtualization and use a new privilege level called Ring -1 (minus one) for the VMM to reside in as depicted below.
This allows for better performance as the VMM no longer needs to fool the VM guest operating system into thinking that it is running in Ring 0 as it can run in there without conflicting with the VMM which has moved to a different level.
The bottom line: When looking for new hardware for your virtual hosts, be sure and choose servers that have one of these types of CPUs that are optimized for use with virtualization.
February 18, 2009 4:45 PM
Posted by: Texiwill
Edward L. Haletky
I have had several major issues with PowerShell (and still do), but that was mainly due to lack of cmdlets rather than lack of PowerShell functionality. So am I now one of the converted, given VMware’s integration with PowerShell? Yes and no.
Yes, because to me, PowerShell is just another scripting language.
No, because I want a scripting language that works well on all operating systems. (People will suggest Java, but that is another blog post.)
I wrote a Perl application that would log in to an ESX host, run an assessment of its configuration and from there aid me in judging the security of the host and some smaller parts of the virtual environment. Since it was written in Perl, it would be hard to get people to use excepting myself and other Perl users. This breed is found mostly within the Linux world; many VMware infrastructure administrators may not understand Perl well as it comes with a learning curve.
So I went looking at various other tools and discovered PowerGUI. PowerGUI is a great tool that has PowerPacks of prewritten PowerShell cmdlets that can run within the GUI.
There is a PowerPack for the VMware Infrastructure Toolkit, Hyper-V and Xen. The idea is to present a graphical interface for PowerShell cmdlets. This is just the interface I wanted for my tool, something I could plug into with ease to present the information in a way my users would easily understand with very little fuss. For them, it just has to work easily.
Unfortunately, PowerShell did not have the major facility I needed to make this work. It does not natively support secure shell and Expect functionality. Expect functionality allows a script to wait for predetermined output from a log-in session (or any other session) that often requires a password . Expect is a very valuable addition to any Linux programmers toolbox.
After posting my questions to the PowerShell channel on irc.freenode.net, I received some answers to my simple PowerShell programming questions. I also used Google searches to find how specific objects worked within PowerShell versus pre-existing cmdlets. Jaykul, a helpful participant on the irc channel, went even further to assist me by developing a set of SSH routines using the SharpSSH client written in .NET that actually has embedded Expect functionality. This type of assistance within a scripting community is outstanding. It shows that the backers really want PowerShell to be a success, and that Powershell can make use of any .NET library on a system.
This, combined with help from PowerGUI guru Scott Herold, will allow me to plug my tool into the PowerPack for VMware Virtualization as an action that can run on each ESX host or cluster as needed. This also implies I can worry about the guts of the code and leave the UI to someone else entirely.
After combining PowerGUI, the VMware Infrastructure Toolkit PowerPack and Jaykul’s Scritable SSH cmdlets I was off and running. My tool works for me and should be easy to understand for others, meaning it will work without tweaking and very few downloads, namely just As in it JUST WORKS with no twiddling and very few downloads, namely PowerShell and perhaps PowerGUI.
So as a long time Linux-centric developer, I have finally found a scripting language for Windows that works well. It is missing functionality that I am used to, but the community is extremely active and very helpful. For additional help, check out the Get Scripting Powershell podcast .
There are great scripts for the PowerShell version of the VMware Infrastructure Toolkit, and the list is growing daily. Be sure to bookmark Alan Renouf’s vDiagram script and his blog Virtu-AI. You’ll find many blog posts on PowerShell tools and links to useful scripts.
Now if they could make PowerShell run on Linux and Mac systems!
February 16, 2009 8:23 PM
Posted by: Eric Siebert
I was doing some research for a virtualization 101 presentation that I am giving for a local VMware User’s Group meeting. I am going to include a few slides on the history of VMware. I focused mainly on the release dates for the virtualization products which I pieced together from a number of sources including the roadmap from virtualization.info, milestones and news releases from VMware’s website and a few tips from other users. You can see from the number of product releases and VMworld attendance that virtualization has really taken off in recent years, which really shouldn’t be a surprise to anyone. I’ve only been using VMware ESX since 2005, having started with version 2.5.
I know there are a lot of veterans out there though that have been using it since the early days. If you’re one of those VMware old-timers who’ve used any of the 1.0 and 2.0 products let us know about it in the comments and tell your experiences from the early days.
- VMware founded by Diane Greene, Dr. Mendel Rosenblum and Ed Bugnion
- VMware Workstation 1.0 released for Windows and Linux
- VMware Workstation 2.0 released
- VMware ESX 1.0 (Elastic Sky X) and VMware GSX 1.0 (Ground Storm X) released
- VMware Workstation 3.0 released
- VMware ESX 1.5 released
- VMware GSX Server 2.0 released
- VMware VirtualCenter 1.0 released with VMotion
- VMware ESX 2.0 released with vSMP support
- P2V Assistant 1.0 released
- VMware GSX Server 2.5 released
- VMware Workstation 4.0 released
- 1st VMworld conference in San Diego, CA (1,400 attendees)
- VMware ESX 2.5 released
- VMware GSX Server 3.0 released
- VMTN technical communities launched
- EMC acquires VMware
- VMware Player 1.0 released
- P2V Assistant 2.0 released
- Second VMworld conference (3,500 attendees)
- VMware Workstation 5.0 and 5.5 released
- Third VMworld conference (6,700 attendees)
- VMware Infrastructure 3 released with VMware ESX 3.0 and VirtualCenter 2.0
- VMware Server 1.0 released, VMware GSX dropped from product line
- Fourth VMworld conference (10,800 attendees)
- VMware ESX 3i released
- VMware Converter 3.0 released (formerly P2V Assistant)
- VMware Fusion 1.0 for Mac released
- VMware ESX 3.5 and VirtualCenter 2.5 released
- VMware Workstation 6.0 released
- Fifth VMworld conference (14,000 attendees)
- VMware Fusion 2.0 released
- VMware Server 2.0 released
- VMware Workstation 6.5 released
- VMware ESXi becomes available for free
- Paul Maritz takes over as CEO
- VMware Converter 4.0 released
- VMware Infrastructure 4 to be released
February 13, 2009 7:47 PM
Posted by: Eric Siebert
A recent KnowledgeBase article from VMware describes a method to monitor the snapshot deletion process which can take a long time if the snapshot has been active for a while.
When you delete snapshots using the VMware Infrastructure Client, the taskbar that displays the deletion progress immediately jumps to 95% and stays at 95% the snapshot is entirely deleted. The taskbar does not show any progress while the delta files are being merged back into the original .vmdk file. Additionally, vCenter Server has a default 15 minute task timeout, so if the snapshot deletion task takes longer then 15 minutes it will timeout in vCenter Server even though it is still running on the host server.
The method in the KnowledgeBase article uses an ESX service console command to perform a directory listing on .vmdk files, so you can see their modified times changing and that the delta.vmdk files disappear after the snapshots are deleted. Here is the command from the knowledgebase article:
watch "ls -lhut --time-style=full-iso *.vmdk"
The way the syntax is currently published, however, is incorrect. Is it missing the double-dash before the time-style option; hopefully VMware will correct this soon.
Let’s go over what the command does. The watch command runs a command repeatedly and displays its output every 2 seconds. The command that watch is running in the previous sequence is the ls command, which lists file information in the directory. The options used with the ls command are –lhut, which are individual options that are all grouped together and are listed below:
• -l – use long listing format which displays more information
• -h – display file sizes in human-readable form (i.e. GB instead of bytes)
• -u – when used with –lt sorts and shows files by access time
• -t – sorts by modification time
Additionally the -–time-style option set to full-iso shows the modified time in longer form with seconds, and the final *.vmdk option tells the ls command to only display .vmdk virtual disk files.
While this command works I modified it a bit to drop the group/owner information (-og instead of –l) from being displayed, so the results fits better on the screen. I also tweaked it to use the shorter –-fulltime parameter instead.
watch “ls –oghut –-full-time *.vmdk”
To use the command, log in to the VMware service console, go to your VM’s directory where the snapshot files are located, and run the command. A listing of the files will be displayed along with size and modification information as shown below:
Once you see the –delta.vmdk files disappear you will know that the snapshot has been deleted, as shown below:
To stop the command from running just press ‘.’ While you will probably not use this method when deleting smaller snapshots, it is useful for larger ones that can take quite a while to delete.
February 9, 2009 9:23 PM
Posted by: Texiwill
Edward L. Haletky
, Local Storage
, VMware ESX
VMTN Communities forum users have recently been asking how to make use of a LUN or partition on your local host within a virtual machine (VM) the same way you would if you had a SAN available. This is a more difficult task than some, and not every RAID controller allows this when using the VMware Infrastructure Client. We have to resort to command-line methods to make this happen.
A VM with a raw disk or raw disk map (RDM) allows the guest OS to write directly to the LUN or disk partition within a LUN assigned to it, utilizing a pass-through mechanism. Using a raw disk or RDP won’t result in a noticeable performance gain, but there is often an improvement in management. In general, when a VMDK grows past a certain size set by the administrator, the administrator will opt to use a raw disk or RDM.
With VMware ESX v2.5 and ESX v3.0 it took a few simple VM configuration file edits to enable a raw disk when using local storage. Unfortunately, those methods no longer work on VMware ESX v3.5. There is, however, a solution. It is not very elegant, but it will work.
The solution is to use vmkfstools to import or copy a virtual machine disk file to the LUN or partition to use for the raw device. Here is an example that I just tested and seems to work. First, I needed to find out which device held the LUN I was going to assign to my VM.
1. Run fdisk -l to find the LUN.
# fdisk -l
Disk /dev/cciss/c0d0: 146.8 GB, 146807930880 bytes
255 heads, 63 sectors/track, 17848 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/cciss/c0d0p1 * 1 13 104391 83 Linux
/dev/cciss/c0d0p2 14 650 5116702+ 83 Linux
/dev/cciss/c0d0p3 651 1287 5116702+ 83 Linux
/dev/cciss/c0d0p4 1288 17848 133026232+ f Win95 Ext'd (LBA)
/dev/cciss/c0d0p5 1288 1924 5116671 83 Linux
/dev/cciss/c0d0p6 1925 2561 5116671 83 Linux
/dev/cciss/c0d0p7 2562 3198 5116671 83 Linux
/dev/cciss/c0d0p8 3199 3453 2048256 82 Linux swap
/dev/cciss/c0d0p9 3454 17835 115523383+ fb Unknown
/dev/cciss/c0d0p10 17836 17848 104391 fc Unknown
Disk /dev/cciss/c0d1: 440.4 GB, 440430842880 bytes
255 heads, 63 sectors/track, 53546 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/cciss/c0d1 doesn't contain a valid partition table
2. Run esxcfg-vmhbadevs to find the vmhba device associated with the LUN.
3. Create the VM with a standard virtual disk (VMDK).
4. Use vmkfstools from the command-line interface to import the VMDK into the vmhba device associated with the LUN using a disk target of RAW. Kill the import after you hit 1%.
# vmkfstools -i OpenFiler.vmdk -d raw:/vmfs/devices/disks/vmhba0:1:0:0 OpenFiler_1.vmdk
Destination disk format: raw disk out of '/vmfs/devices/disks/vmhba0:1:0:0'
Cloning disk 'OpenFiler.vmdk'...
Clone: 1% done.<ctrl-C>
5. Review the resultant VMDK metadata file. Note that the size of the LUN is references. For size take 860216490 and multiply by 512 for 410 GBs.
# cat OpenFiler_1.vmdk
# Disk DescriptorFile
# Extent description
RW 860216490 VMFSRAW "/vmfs/devices/disks/vmhba0:1:0:0"
# The Disk Data Base
ddb.virtualHWVersion = "4"
ddb.uuid = "60 00 C2 97 5c a6 e9 59-f3 de ba f6 83 ed 15 73"
ddb.geometry.cylinders = "1044"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"
5. Using the VMware Infrastructure Client (VIC), add an existing disk into the VM or add the following lines to the VMX file. (Note that you want to use a virtual SCSI controller not already in use. Also, the VM you make these changes to should be powered off and perhaps even unregistered if you are doing this from the CLI. If you use the VIC add the device following the red arrow path in the diagram below.)
scsi1.present = "true"
scsi1.sharedBus = "none"
scsi1.virtualDev = "lsilogic"
scsi1:0.present = "true"
scsi1:0.fileName = "OpenFiler_1.vmdk"
scsi1:0.deviceType = "scsi-hardDisk"
Now you have a VM that can directly access the local LUN on your VMware ESX host. I imagine this will work the same way for ESXi.
February 9, 2009 7:47 PM
Posted by: Eric Siebert
, Eric Siebert
, virtual disk
There’s an ongoing debate about whether or not VMware users should defragment virtual machine disks. We’ll answer this question in two parts: the first part will cover virtual machine (VM) disks inside a guest operating system. The second part will cover VM disks outside of a guest operating system that reside as a large file on a host machine’s VMFS data store.
Should you defrag a VM’s disk at the guest operating system level? The answer is yes. If a virtual machine disk is excessively fragmented, you should treat a VM’s disk just as you would a physical server and defrag as necessary.
If you do defrag your VM’s disk, then you should be aware of a few things. First, a defrag operation causes high disk I/O on your host server so it is best to perform a defrag during off hours or when disk activity is low on your host. You should also try not to defrag too many VMs on a host simultaneously as this can really drive the I/O up on your host server, which could slow down other VMs on the same host, or even on other hosts that have VMs on the same storage as the VMs being defragged. Finally, you should never defrag a VM’s disk if the VM has an active snapshot. While this will not cause any harm to the VM it will make your snapshot file grow rapidly as many disk blocks are being modified. It will also severely decrease LUN performance because every time a snapshot grows in 16 MB increments it causes a SCSI reservation which briefly locks the LUN.
So should you defrag your host’s VMFS volumes? The answer is no, for two reasons. First, it is not possible as there are no built-in or third party utilities available to execute the operation. The second reason is that the host VMFS data stores do not really fragment because of how virtual disks write to the VMFS volume. That’s not to say a VMFS volume cannot become fragmented at all. They do, but not to the extent that traditional OS disks become fragmented. This is because a standard OS can consist of many thousand of files that are fairly small in size, and a VMFS volume contains mostly very large virtual disk files.
When a standard thick disk is created for a VM all space on the disk is allocated at once, which results in the disk having mostly contiguous space. There are some exceptions to this. If you manually create a thin disk on a VMFS volume it will grow on demand which can result in fragmentation as the blocks are written to different free spots on the physical disk. Also, if you copy a disk file to a VMFS volume using a utility like SCP the space is allocated as the file is copied and not all at once. The proper way to import or copy a virtual disk file to a VMFS volume is to use the vmkfstools utility which allocates all space at once before the copy is made.
Over time, a VFMS volume can become fragmented as VMs and snapshots are created and deleted, and other smaller files like the vswp files and log files are also created and deleted. These changes result in new VM disks that do not have contiguous space. But this typically isn’t a problem because even though the disk blocks are no longer contiguous, they are still split up among several very large contiguous spaces which results in a disk that has minimal fragmentation.
In short, go ahead and defrag your VMs’ disks if they becomes fragmented, and don’t worry about your host’s disk, which will not fragment like traditional disks do.