Virtualization Pro

Feb 9 2009   9:23PM GMT

VMware ESX storage: How to get local storage to act as a raw disk for VMs

Texiwill Edward Haletky Profile: Texiwill

VMTN Communities forum users have recently been asking how to make use of a LUN or partition on your local host within a virtual machine (VM) the same way you would if you had a SAN available. This is a more difficult task than some, and not every RAID controller allows this when using the VMware Infrastructure Client. We have to resort to command-line methods to make this happen.

A VM with a raw disk or raw disk map (RDM) allows the guest OS to write directly to the LUN or disk partition within a LUN assigned to it, utilizing a pass-through mechanism. Using a raw disk or RDP won’t result in a noticeable performance gain, but there is often an improvement in management. In general, when a VMDK grows past a certain size set by the administrator, the administrator will opt to use a raw disk or RDM.

With VMware ESX v2.5 and ESX v3.0 it took a few simple VM configuration file edits to enable a raw disk when using local storage. Unfortunately, those methods no longer work on VMware ESX v3.5. There is, however, a solution. It is not very elegant, but it will work.

The solution is to use vmkfstools to import or copy a virtual machine disk file to the LUN or partition to use for the raw device. Here is an example that I just tested and seems to work. First, I needed to find out which device held the LUN I was going to assign to my VM.

1. Run fdisk -l to find the LUN.

# fdisk -l
Disk /dev/cciss/c0d0: 146.8 GB, 146807930880 bytes
255 heads, 63 sectors/track, 17848 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot    Start       End    Blocks   Id  System
/dev/cciss/c0d0p1   *         1        13    104391   83  Linux
/dev/cciss/c0d0p2            14       650   5116702+  83  Linux
/dev/cciss/c0d0p3           651      1287   5116702+  83  Linux
/dev/cciss/c0d0p4          1288     17848 133026232+   f  Win95 Ext'd (LBA)
/dev/cciss/c0d0p5          1288      1924   5116671   83  Linux
/dev/cciss/c0d0p6          1925      2561   5116671   83  Linux
/dev/cciss/c0d0p7          2562      3198   5116671   83  Linux
/dev/cciss/c0d0p8          3199      3453   2048256   82  Linux swap
/dev/cciss/c0d0p9          3454     17835 115523383+  fb  Unknown
/dev/cciss/c0d0p10        17836     17848    104391   fc  Unknown
Disk /dev/cciss/c0d1: 440.4 GB, 440430842880 bytes
255 heads, 63 sectors/track, 53546 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/cciss/c0d1 doesn't contain a valid partition table

2. Run esxcfg-vmhbadevs to find the vmhba device associated with the LUN.

# esxcfg-vmhbadevs
vmhba0:0:0     /dev/cciss/c0d0
vmhba0:1:0     /dev/cciss/c0d1

3. Create the VM with a standard virtual disk (VMDK).

4. Use vmkfstools from the command-line interface to import the VMDK into the vmhba device associated with the LUN using a disk target of RAW. Kill the import after you hit 1%.

# vmkfstools -i OpenFiler.vmdk -d raw:/vmfs/devices/disks/vmhba0:1:0:0 OpenFiler_1.vmdk
Destination disk format: raw disk out of '/vmfs/devices/disks/vmhba0:1:0:0'
Cloning disk 'OpenFiler.vmdk'...
Clone: 1% done.<ctrl-C>

5. Review the resultant VMDK metadata file. Note that the size of the LUN is references. For size take 860216490 and multiply by 512 for 410 GBs.

# cat OpenFiler_1.vmdk
# Disk DescriptorFile
# Extent description
RW 860216490 VMFSRAW "/vmfs/devices/disks/vmhba0:1:0:0"
# The Disk Data Base
ddb.virtualHWVersion = "4"
ddb.uuid = "60 00 C2 97 5c a6 e9 59-f3 de ba f6 83 ed 15 73"
ddb.geometry.cylinders = "1044"
ddb.geometry.heads = "255"
ddb.geometry.sectors = "63"
ddb.adapterType = "lsilogic"

5. Using the VMware Infrastructure Client (VIC), add an existing disk into the VM or add the following lines to the VMX file. (Note that you want to use a virtual SCSI controller not already in use. Also, the VM you make these changes to should be powered off and perhaps even unregistered if you are doing this from the CLI. If you use the VIC add the device following the red arrow path in the diagram below.)

scsi1.present = "true"
scsi1.sharedBus = "none"
scsi1.virtualDev = "lsilogic"
scsi1:0.present = "true"
scsi1:0.fileName = "OpenFiler_1.vmdk"
scsi1:0.deviceType = "scsi-hardDisk"

Now you have a VM that can directly access the local LUN on your VMware ESX host. I imagine this will work the same way for ESXi.

 Comment on this Post

There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to:

Share this item with your network: