VMware High Availability (VMware HA)

Replacing a VMware ESX SAN

Replacing or upgrading a SAN is no trivial task. There are a few tried-and-true steps to take when replacing a SAN which I’ll outline in this blog post, including a key step to the process that will ensure a successful switch.

I recently upgraded from an HP MSA 1000 to an IBM DS3400 because I wanted to improve performance and lower my overall energy costs.  One of the reasons I decided to replace my old SAN is because it is much cheaper to have a single 2U device running than the three devices for the old SAN. In addition, I dropped from 42 drives to 12 drives with more storage. Minimally my SAN power costs should drop to just 1/3 the original. I also have gone from 2.5 TB to 3 TBs of storage. Not a huge increase in storage capability.

I will know next month if my energy cost reductions have been realized and will report back then.

The steps for replacing a SAN are not all that tricky, but there was a single gotcha that could be avoided with careful planning. To replace a SAN, follow these steps:

1. Plug in the new SAN to your existing fabric. Luckily I had a pair of unused fibre connections and Gbics available else this would have been another expense and a delay until the cables and Gbics arrived.

2. Find a system on which to install the management console. For the IBM DS3400 I chose my VirtualCenter and VMware Consolidated Backup (VCB) server to be the management console for the SAN. There are two methods to manage the IBM DS3400: in-band or over the fibre channel fabric, or out of band using Ethernet — even a VM would suffice given networking is connected to the SAN. Software exists for both 64-bit Linux and Microsoft Windows.

3. Create the LUNs on the new SAN. This is a good chance to correct any problems you may have with the LUN configuration on the old SAN. I did a one-to-one mapping, except I slightly increased the size of the LUNs.

4. Present the LUNs to your VMware ESX host(s) and VCB server(s).

5. Rescan the storage adapters for new LUNs using the VMware Infrastructure Client (VI Client) for the first VMware ESX host. Once this is completed, you can then add as many Virtual Machine File Systems (VMFSs) as required.

6. Rescan the storage adapters for new LUNs and VMFSs using the VI Client on all the other ESX hosts.

7. Employ Storage VMotion via the VI Client to migrate VMs from one LUN to another LUN. This works if you have the patience to move all the VMs one by one. If not you can employ other measures. If you do this, however, you will end up having to edit the VMX files for each VM migrated to change the location of the virtual disk files. There are scripts to do this for you as well. This second option, however, also requires you to power off all VMs. Use of Storage VMotion does not require any VM downtime. Be sure to move all files from the LUNs in use.

8. For a LUN with an RDM (mine was a Linux file server), use Storage VMotion to move any VMDKs related to the VM. Then map the new RDM to the VM. You will have to reboot the VM to complete. Then create a new filesystem on the new RDM mount the file system. Then you must copy all the files from the old RDM to the new RDM. I used the following command to complete this task to copy all files from /files to /files2.

• cd /files; rsync -ravlpog * /files2

9. Then I modified the mount point for /files within /etc/fstab to be the correct new location. Finally I powered off the VM, deleted the old RDM from the VM and powered on the VM picking up the new data.

Here is the gotcha. I missed it, but it will be extremely useful for you (and me) going forward: Remove the old SAN’s LUNs from each VMware ESX host. If you miss this step when you finally disconnect the old SAN, the ESX hosts will go into a state of constantly attempting to failover the old LUNs. This will spew massive failures into the log files. If this happens there is no recourse but to reboot the VMware ESX hosts.

Now the SAN has been replaced. With the exception of dealing with any RDMs, it is possible to migrate to a new SAN without any downtime.

VMware adds another Windows user to its customer list

VMware, Inc. announced last week that another Windows customer is using VMware Infrastructure 3 (VI3) instead of Microsoft Hyper-V to consolidate servers and reduce costs.

Independence Blue Cross (IBC), the largest health insurer in Philadelphia, has grown quickly in recent years, and their  computing demands and costs have grown along with its business. Physical server sprawl and increasing power consumption has plagued the hospital and the cost of acquiring and managing new hardware was growing out of control, VMware reported.

To reverse these issues, IBC turned to virtualization. The company looked at Microsoft Hyper-V, but ultimately chose VMware because “it offered a more complete solution and robust tool set, rather than simply a hypervisor,” VMware’s spokesperson said on behalf of the customer. Another plus for VMware was that is offers VMotion to live migrate virtual machines (VM), which Micrsoft’s Hyper-V product won’t offer until the next version, as well as high availability,  resource pooling, manageability and automation, VMware said.

So far, IBC’s Windows application environment is approximately 70% virtualized, including applications like Active Directory, Exchange, SharePoint and SQL Server, PeopleSoft and Oracle 9i. There are 386 VMs are running on 48 physical hosts, and CPU utilization has increased from 5% to 75%, VMware reported.

Michael Garber, director of distributed infrastructure, at IBC, stated in the release that VI3 paid for itself in less than 16 months and helped IBC avoid more than $1 million in hardware costs.

VMware currently has over 3,000 hospitals on its list of customers, according to VMware.

VMware release lots of customer case studies to show the world how great they are, but when they announce Windows users as customers, Microsoft Hyper-V takes a bullet.  Hyper-V is built right in to Windows Server 2008, so why wouldn’t a Windows user just virtualize with Hyper-V? That’s Microsoft’s argument, and it looks like people aren’t buying it.

There is a ton of speculation on whether Hyper-V will be able to surpass VMware in the virtualization market, but I haven’t seen anything from Microsoft (like Hyper-V customers!) signaling that possibility.

VMware fixes VirtualCenter 2 bugs with VirtualCenter Update 3

On Friday, VMware released the latest version of VirtualCenter, Update 3 (no update 3 for ESX yet). Unlike Update 2 (which contained some great new features), this version is mainly focused on fixing bugs. VMware administrators may be a little leery of installing this update after the time-bomb debacle that occurred several months ago with Update 2, but there are a few fixes (including many for HA) outlined below that make it worth installing.

• Permissions Can Be Configured for Individual Virtual Machines and Resource Pools in VI Client - Starting with this release, when the VMware Infrastructure Client is connected directly to a host running ESX Server 3.5 or higher, the Permissions tab is available for individual virtual machines and resource pools.

The first notable fix is for displaying a login password in clear-text. You may not think this is too big of a deal, but if someone were to be standing near you when this happened they could see your password (or at least the one you entered) displayed in clear-text.

• VMware VirtualCenter password might display in error window - This release resolves an issue where a user’s password can be displayed in clear text on the login screen. When logging into VirtualCenter Server 2.0 with Virtual Infrastructure Client 2.5, the user password might be displayed in a dialog box on the VI Client in clear text if the login fails. The dialog box alerting the user to the failed login might be hidden under other windows.

The next fix updates the JRE used on the VirtualCenter server to the latest version which is 5.0 Update 16 (1.5.0_16) which fixes some security issues.

• WebAccess component JRE updated to version 1.5.0_16 - The currently installed version of JRE depends on your patch deployment history. For more information about security issues fixed in version 1.5.0_16 and in earlier versions, see the JRE release notes at http://java.sun.com/j2se/1.5.0/ReleaseNotes.html.

An update to the Flex License Manager server is included. This update is not installed automatically when upgrading existing installations and must be installed separately.

• FLEX license server upgrade - This release of VirtualCenter upgrades the FLEX license server to version 10.8.6 in order to resolve known issues with previous releases of the license server and to provide enhance debugging capabilities. The new FLEX license-server is packaged with the VMware Infrastructure Management Installer. Fresh installations of the license server will use the new version but the license server will not be automatically upgraded when using the VMware Infrastructure Management Installer to upgrade an existing installation. To upgrade an existing license server installation, use the standalone installer (VMware-licenseserver.exe) that can be found in the /vpx folder of the installer directory structure.

This one caused some issues with HA because of a network compliance check that was introduced in Update 2. A new HA advanced setting has been added to bypass this check.

• HA network compliance check - During the configuration of HA in VirtualCenter 2.5 Update 2, the Task & Events tabs might display the following error message and recommendation: HA agent on in cluster in has an error Incompatible HA Network: Consider using the Advanced Cluster Settings das.allowNetwork to control network usage. Starting with VirtualCenter 2.5 Update 2, HA has an enhanced network compliance check to increase cluster reliability. This enhanced network compliance check helps to ensure correct cluster-wide heartbeat network paths. VirtualCenter 2.5 Update 3 allows you to bypass this check to prevent HA configuration problems. To bypass the check, add das.bypassNetworkVerification=yes to the HA advanced settings.

In addition to the fixes and updates to the Update Manager and Converter, plug-ins have been released. If you upgrade to Update 3 you must also update these plug-ins or they will no longer work.

• Plug-ins Updates - Previous versions of VMware Update Manager are not compatible with VirtualCenter 2.5 Update 3 and must be upgraded to VMware Update Manager 1.0 Update 3 during the installation process. This release of the VMware Infrastructure 3 software suite also includes the following: • An update to VMware Converter Enterprise. For more information, see the VMware Converter Enterprise Update 3 for VirtualCenter 2.5 Release Notes. • An update to VMware Update Manager. For more information, see the VMware Update Manager 1.0 Update 3 for VirtualCenter 2.5 Release Notes.

This one has caused a few people who upgraded to Update 2 some grief — thankfully VMware has quickly addressed it.

• In HA-DRS cluster, the enter maintenance mode task stalls and VMs do not migrate - In previous releases of VirtualCenter, virtual machines might not be automatically migrated off a host entering maintenance mode if there is not enough failover capacity in an HA-DRS cluster. The enter maintenance mode task stalls at 2% indefinitely and does not complete even if HA admission control was disabled. In this release, the issue has been resolved by allowing the evacuations if HA admission control is disabled. Note that admission control is enabled by default, see Implications of enabling or disabling VMware HA strict admission control when using DRS and VMware DPM (KB 1007006) for more information. Note that the task might also stall if the virtual machines cannot be evacuated for other reasons. And finally a minor one with not being able to delete HA advanced settings.  

• Once HA advanced settings are created, they cannot be deleted - In previous releases advanced settings created on the Advanced Options page for an HA cluster cannot be deleted. Attempts to delete the advanced setting would result in an object reference error dialog box being displayed. This release resolves the issue and advanced settings can be deleted normally.

As always read the release notes before upgrading. This release may not have all the cool new features of the previous release but bug fixes, while not glamorous, are a necessary evil to ensure a stable product.

Virtualization virtual tradeshow offers VMware networking opportunities

Trade shows are great – if you have time to attend, have staff to cover while you’re away learning about a new technology, can avoid summons back to the office during the show, can find a show in your local area or can get budget approval to attend a show that requires flight or hotel reservations.

Enter the virtual trade show (VTS); an online conference conceived to mitigate the above challenges. Last week, sister sites SearchDataCenter.com and SearchServerVirtualization.com hosted an advanced enterprise virtualization VTS. I helped staff the networking lounge and editorial booth where I had the opportunity to chat with VMware users about two of the virtualization provider’s newest tools, Site Recovery Manager (SRM) and Storage VMotion.

IM chatting with attendees
Conversations ranged from general IT talk (“Anyone use virtual desktops?”) to small talk (“What’s the weather like in Maine?”). Trying to be the friendly host, I said “Good morning” to the room. I immediately got the reply “Good evening” and was subsequently told this particular user was signed on from –literally–the other side of the world.

I ended up chiming-in on another user’s question about if anyone was familiar with VMware Site Recovery Manager (SRM). The respondent had said that he was, and I ended up asking him about his experience via private IM. SRM orchestrates your virtual machine disaster recovery (DR) plan in the event that your main data center goes down. It prioritizes which virtual machines (VMs) are brought up at the failover site based on available resources, syncs your VM configurations between the main site and the failover site, and allows for DR plan testing without having to take the system offline. It’s a relatively new addition to the VI3 lineup, having been on the market for four months (at the time of publication).

Our conversation turned to plug-ins, and he raved about Andrew Kutz’s Storage VMotion plug-in. The plug-in adds a user interface to the out-of-the-box product, which operates through a command line interface. The attendee explained that he’s primarily a “Windows guy,” so a graphical user interface makes using Storage VMotion much easier.

Kutz recently released an update to the Storage VMotion plug-in.

“The new release now ignores raw device mapping,” Kutz said. “Previously, if you had a raw device that pointed to a 300 Gig disk, the plug-in would look at it as an actual disk and screw up the disk size map.”

He also removed the majority VMware’s internal code from the plug-in (excepting the code that loads the plug-in), replacing it with code based on the VI Toolkit for .NET.

Impressive user interface
The VTS emulates the look of a physical tradeshow floor, which makes navigation a bit friendly, though not as intuitive as I would have liked. You could either move around with the help of a clickable navigation bar, or point-and-click your way from the main entryway to the desired location, be it the conference hall, vendor hall, networking lounge or “library” where you can download PDFs of presentations and various information from vendors, which then moves into your “suitcase,” displayed on your personal page.

VTSs are essentially fancy webcast packages displayed in unconventional ways. In this particular show, the topics were “Protecting your Virtual Environment: Backup and Storage,” “Virtual Infrastructure Automation and High Availability Best Practices” and “Virtual Infrastructure Tuning and Advanced Management.” The speaker was displayed on the left side of the screen presenting his slides via streaming video. The slides were displayed on the right hand side. Users could ask questions via a box at the bottom of the screen.

The VTS, if done correctly, has many more plusses than minuses. As long as there is a reliable Internet connection, there’s no need to leave the data center (if you don’t have a reliable connection in your data center, you might think about leaving for good). The content is almost exactly the same as at a physical trade show (that’s how they got the video of the speaker to begin with). And editorial staff can send IT pros direct links to helpful guides that they know of if an IT pro wants to know about, for example, virtual desktop drawbacks.

If any SearchVMware.com readers passed up the opportunity to “attend” a virtual trade show, I suggest you test it out next time a topic of interest comes around. It’s actually fun to use (think AOL in the 90’s minus the “you’ve got mail”) and offers great learning potential and networking opportunities.

An archived version of the advanced enterprise virtualization virtual trade show is available online, short registration required.

Automatic recovery for failed virtual machines

A little known feature, called Virtual Machine Failure Monitoring (VMFM), was introduced in ESX 3.5. VMFM offers the ability to leverage HA to monitor VMs for operating system failures, such as blue screens, and have them automatically restarted. Previously, HA would only deal with ESX host failures by automatically restarting VMs on alternate ESX hosts in the event of a problem with the host server.

VMFM also extends HA to monitor VMs through a heartbeat sent every second when using VMware Tools. This new feature is disabled by default and is considered ‘experimental’ by VMware. This typically means it works but it is not officially supported for production use yet. In order for this feature to function properly you must first ensure the following conditions exist:

• ESX hosts are version 3.5
• VirtualCenter is version 2.5
• VMware Tools is installed on VMs and is the latest version
• You have a Cluster configured and HA enabled

To enable it follow the below steps:

• Edit the Settings for your Cluster
• Choose VMware HA and click the Advanced Options button
• Add the following Options and Values

das.vmFailoverEnabled – true (true or false)
das.FailureInterval – 30 (declare virtual machine failure if no heartbeat is received for the specified number of seconds)
das.minUptime – 120 (After a virtual machine has been powered on, its heartbeats are allowed to stabilize for the specified number of seconds. This time should include the guest operating system Boot up time)
das.maxFailures – 2 (Maximum number of failures and automated resets allowed for the time that das.maxFailureWindow specifies. If das.maxFailureWindow is ‐1 (no window), das.maxFailures represents the absolute number of failures after which automated response is stopped and further investigation is necessary)
das.maxFailureWindow – 86400 (Either -1 or a value in seconds. If das.maxFailures is set to a number, and that many automated resets have occurred within that specified failure window , automated restarts stop and further investigation is necessary)

I enabled this on a Cluster and tested it by simulating a blue screen on a VM running Windows 2003 Server and it worked perfectly. After 30 seconds the loss of heartbeat was detected and the VM was automatically restarted. Currently there are no notification alerts that can be configured when this occurs. That is, if you check the events for the VM you will see no evidence of this happening. The only mention of it that I found in the logs was in the hostd log on the ESX server ([2008-06-26 11:47:22.552 'ha-eventmgr' 3076440992 info] Event 101 : VM1 on Esx1.xyz.com in ha-datacenter is reset). Hopefully this will change in a later version when the feature is no longer considered ‘experimmental’. You can read more about this new feature in a white paper that VMware has provided on it.

Isolate ESX hosts in separate clusters for maintenance, upgrades

Performing key maintenance on a VMware Infrastructure 3 (VI3) host OS or can be an involved process, as can the process of simply adding a host. Because of the length of time that a system may be offline or because of what exactly is being performed, having the hosts in an isolated cluster can allow a safe environment for virtually any task. An isolated cluster will not apply to the same DRS and high availability rules that may apply to the live cluster. Further, you can reboot the host as needed without needing to wonder if there will be any effect to the live workload.

With the isolated cluster, the following tasks can be safely performed:

-Version upgrades (ESX 3.0x to 3.5 Update 1)
-Hardware maintenance
-Adding a new host to an existing cluster
-Testing network connectivity to various port groups
-Confirming VMware HA and DRS performance with in an isolated set of rules
-Importing or configuring new storage types

A good practice for the isolated cluster would be to keep it less visible to the live workload and named accordingly within the VMware Infrastructure Client, so a name like “TestingCluster” or “ZZUpgradeCluster” can distinguish the collection to be different from that of the live workload.

The figure below shows a cluster with a name and position that contains one host in maintenance mode for any task better suited in an isolated environment:

It is important to note that this cluster would still require licensing as it would be in live workload cluster. More information on creating a VI3 cluster can be found in the VI3 online library.

Storage and network planning takeaways from Iowa virtualization user group meeting

Sean Clark is a VMware Certified Professional (VCP) and a member of the Des Moines, Iowa-based Central Iowa Virtualization Users’ Group (CIVUG). The CIVUG emerged from the Central Iowa Linux Users’ Group as a way for virtualization users to learn more without being tied to one vendor. That is, the CIVUG discusses Hyper-V and Citrix Systems’ Xen in addition to VMware.

Clark has cut his teeth on VMware and other virtualization platforms as a solutions architect at Alliance Technologies, where he works with small and medium—sized businesses and some enterprise businesses to develop strategies to best implement virtualization, be it storage, server or application virtualization. Clark posted notes on a presentation he did at a recent meeting on Red Hat Network File System (NFS) configuration being used with VMotion, High Availability (HA) and Distributed Resource Scheduler (DRS).

NFS vs. Fibre Channel and iSCSI
The NFS configuration how-to Clark followed was posted by Mike Laverick on RTFM Education and was chosen for the demo not only because of it’s easy configuration, but also because implementing Red Hat NFS is a good low-cost alternative to shelling out big cash for more expensive devices for virtual machines (VM) storage. “Most people think that you need a Fibre Channel SAN [storage area network] or an iSCSI SAN; you’ll spend at least 20 or 30 grand on that device. But all you really need is a reasonably new server with some pretty fast discs, and you can basically have a SAN for the cost of your hardware.”

Clark also says that despite what some benchmarking stats say, performance won’t suffer on the cheaper NFS. “If you do your homework, you’ll find that in many benchmarks when you compare the performance of NFS versus iSCSI, they’re almost nearly identical. You can spin your benchmarking tasks so that one will come out on top every time, but for the majority of uses out there, and especially for test and dev, NFS is just as good as iSCSI.”

The storage system lines blur when you scale out to enterprise-level requirements because the needs go beyond the hardware and software backing your VM storage. Clark says that it’s really about having enough discs. “It doesn’t matter if you have NFS, Fibre Channel or iSCSI if you don’t have enough discs to meet the I/O demand that the number of virtual machines can put on a SAN. Most of the time, it becomes almost a religious decision — whatever you’re most comfortable with.”

HA snafu highlights proper network configuration
After walking through the NFS configuration, Clark went on to demo VMotion, HA and DRS, but ran into some problems with the HA portion. At his home in Pella, Iowa, Clark has a small test lab of two 1U servers running two ESX VMs, which he brought to Des Moines for his presentation.

With little time to reconfigure the servers (Clark was asked only a few days prior to the meeting to give the presentation), Clark decided against bringing his PC that he had used as his NFS server in favor of a Red Hat VM running on his laptop, which served as the basis for his presentation. Although both the ESX servers were able to see the Red Hat VM through a little Linksys switch, the demonstration came to a standstill during the HA portion.

The problem? Because the conference room that the VUG meets in is an island network, there is no default gateway address available as there was at Clark’s lab in Pella. An available, ping-able gateway address is required for HA to work. This is because when HA is set up, each ESX server establishes and communicates through a heartbeat and that’s how it determines whether each server is awake or not. If that heartbeat ceases, then HA makes a decision to restart the VMs that are running on a different ESX server. Sometimes the network becomes unavailable, but the other ESX server continues to run.

The isolation address was the pain point in Clark’s HA demo. The default gateway that used to exist in Pella didn’t exist and HA failed. As Clark explains, “It’s just an additional IP address that an ESX server can gain to say, ‘OK, I lost my buddy, but I need to make sure that the network is still up.’ So as long as it can reach that other IP address, then it can assume that the other host is actually truly dead and that it needs to restart its failed VMs.”

Proper planning and redundancy is key to successful deployments
Clark says that he almost never has problems because he is careful to use VMware-supported hardware and plans out each deployment carefully. But not all IT departments are so careful. “I ran into a customer the other day that didn’t want to invest in redundant networking; they had a four-host cluster and all their networking was going through one physical switch. They had HA set up, and when that switch went down, ever single VM that was on one of their ESX servers powered down.”

Clark says that this can happen as a result of not properly planning your configuration. “One of the configuration options for HA [concerns] what to do if an ESX host becomes isolated. Because it lost its network, there wasn’t a second physical switch for redundancy. Each host, even though it was still running, became isolated according to the HA configuration. It was probably a default setting at the time when they set up the HA cluster, and it powered down all the VMs.” Clark says that with proper preparation, the ESX hosts may not have appeared isolated to HA and this organization may have been able to save the headache of restarting all its VMs and troubleshooting what caused the power-down.

Friends don’t let friends, like VMware, act like Google

I like VMware. I like Google. Heck, both of them keep me more than busy with development ideas. But I have a problem with them. Google started it with Gmail. Although it is hard to remember now, Gmail was in beta forever. Oh wait, it still is? Huh. I guess I just figured it *must* have hit production by now. Then there is Google News, Google Apps, Google Page Creator, Google everything else — all beta . I am honestly surprised search hits don’t come back with the “beta” tag next to them. I guess they thought ICQ was the cat’s meow, and that the whole beta thing had a nice ring to it.

Enter VMware, which is perilously close to become the next Google in terms of heavily pushing new features, but then labeling them as beta or experimental. Take for example Storage VMotion (SVMotion). VMware played up this new feature to VI 3.5 last fall at their North American VMworld conference, but when it was release there was no graphical user interface (GUI) option for it. How is that ready for prime-time? And then there is virtual machine (VM) high availability (HA),  another marketed feature that is so experimental you have to edit an advanced setting (as a free-form string) just to enable the functionality.

I wouldn’t actually have a problem with VMware doing this if they didn’t market the heck out these new “features.” Excuse me for being old fashioned, but it isn’t enterprise-ready if it is beta or labeled experimental. And VMware makes no bones about this; they plainly state that these features should not be used in production. However, on the other hand they make a big show about the same set of features, whipping the crowd to a fever pitch of excitement. You can’t have it both ways, guys.

Take VMware Fusion 2 or VMware Server 2. These products are in beta stages right now and VMware is not making a big deal about them. Sure, they are out there for people to get, but VMware isn’t throwing them at customers, not the way they revolved last year’s North American and this year’s European VMworld conferences on features that were not even ready for production.

Then there is the other end of the spectrum as well. I recently discovered that VMware is strategically hiding a long sought feature of ESX in the bowels of its software development kit (SDK). Since version 2.5 of the SDK (VI 3.5), VMware has included the ability (although it does not yet appear to be working correctly) to create network address translation (NAT) and dynamic host control protocol (DHCP) devices directly on ESX servers for VMs to use. This is awesome! Prior to this, the only way to create NATd networks on an ESX host was to dual home a VM to a public and private port group, have it act as the NAT and DHCP server, and then attach other VMs to the same network as its private interface. This solution was cumbersome and did not work well when VMotioning VMs. If I was VMware, I would make a little bit more noise about the fact that they are working on this feature.

I want to reiterate that I like, if not love, VMware. I just hate getting jazzed about a new feature that they have thrown at me, only to find out that it is a curve ball. VMware needs to make sure that features that are experimental should be announced with an asterisk next to their headline, while at the same time working a little harder to ensure that some other upcoming features get the love they deserve.

Why you should upgrade to VI3 Update 1

VMware Infrastructure 3 Update 1, made available on April 10 2008, introduces some core updates to ESX Server 3.5, VirtualCenter 2.5, and the VMware Infrastructure Management Installer. The biggest reason to upgrade, however, is the inclusion of Storage VMotion.

Among the core features now available with ESX 3.5 Update 1 are the addition of the Intel 82598 10 GB Ethernet controller, support of Jumbo Frames and NetQueue, additional Microsoft Clustering Services (MSCS) support, additional backup product and management agent support, additional guest operating systems, and additional server models.

I’ve been working with ESX 3.5 Update 1 for a few weeks, and the installation and behavior are indistinguishable from both ESX 3.5’s base release and ESX 3.02, with the exception of context sensitive tasks or options.

When I test upgrades, I make a point to test the upgrade in an environment with dissimilar ESX host server releases. For example, most of my hosts are ESX 3.02. When I upgrade the first one to ESX 3.5, I want to make sure that nothing goes wrong. I want to know that I’ll be able to sustain a mixed environment with all functionality. When I migrate running virtual machines through host-based VMotion to the ESX 3.5 host, and the reverse, I want to make sure to the best of my ability that nothing will fail. I also want to ensure that all of the VMware DRS and VMware High Availability rules are still enforceable with the mixed-host inventory.

Outlining a functionality matrix and the verification of the behavior is key to having no surprises during a live upgrade. Testing the update to VirtualCenter is a little more difficult but I am setting up a test environment soon to ensure that everything functions as expected in my environment. Overall, the fixes and new features make ESX 3.5 Update 1 an attractive upgrade for systems that are not there already.

Coming soon: Application High Availability (AHA)

Scott Lowe recently blogged about things that he loves but are not yet available. VMware, it seems, is getting as bad as Apple in announcing products far ahead of their release dates. This generates great market buzz and sends stock prices soaring (or in the case of VMworld Europe leaving them as low as they have been), but it does little to satiate the desires of systems administrators, it only serves to increase want and provide little respite. Well, then I am about to send you into the Sahara with the promise of an oasis, only to find a sign-post that says “Coming Soon!” at the end of your journey.

Please join me in anticipation of VMware Application High Availability (AHA).

AHA has not been announced by VMware, but it is coming. How do I know this? It is next logical step for their HA portfolio. ESX 3 brought us server HA. 3.5 introduced us to VM HA. And the recent announcement of the VMsafe all but secures the eventual release of AHA.

What is AHA? Simply put, you will be able to right-click on a VM from the VI client and indicate that if Microsoft Exchange fails then the Exchange service should be restarted, or the VM should be failed over to another ESX server. Or perhaps you want to monitor the Apache web server — just check a box. How will VMware achieve this level of fine-grained control? Allow me to refer to the VMsafe product page:

Process execution
VMsafe provided in-guest, in-process APIs that enable complete monitoring and control of process execution.

This API will allow VMware to monitor and control processes within the guest OS. That, my friends, is how AHA will work.

AHA will allow VMware to take even more market share away from companies like Sun and Microsoft, both who have their own clustering technologies. Why cluster at the OS-specific level when you can create clusters the same way no matter the OS or application underneath!

VMsafe is set to debut later this year, and I am quite certain that alongside VMsafe, or shortly thereafter, we will see VMware announcing and releasing its application level high availability software. I hope you aren’t too parched : )