Virtualization

Open Virtualization Format (OVF) released

The Distributed Management Task Force (DMTF) released the finished version of the Open Virtualization Format (OVF). The OVF is a set of metadata tags that can be used to deploy virtual environment across multiple virtualization platforms.

Using OVF users can download a virtualized application and run it on the operating system on which it was developed. Since the OVF is not hypervisor dependent users can install it on the virtualizaton platform of their choice. Admins and other users can download the OVF packaged software and install it within their own virtual infrastructure instead of creating the virtualization platform on which the software was originally installed. By using this admins can get new applications into production faster.

OVF is actually a set of metadata that describes the virtualization container. This allows the virtualization platform to translate the machine into it’s own environment. While the practice of competing virtual platforms translating virtual machines to run on their platform is not new until now there has not been an open source application that would wholesale translate any virtual machine into any virtual environment. This is all accomplished by using the metadata standard developed for the OVF,

OVF can also be used to manage virtual machine installations. For example if machines are required to start in a certain order to allow for dependencies the OVF can handle this. You can find out more about the OVF here.

-j

Want to know how the Federal Government uses virtualization?

I have often wondered how the really big technology users, like the Federal Government, utilize various technologies such as virtualization.

Now we can all get a first hand look by watching an eSeminar presented by Government Computer News. They are presenting Anil Karmel, a solutions architect in the network and infrastructure engineering division at Los Alamos National Laboratory, in an eSeminar at 2 p.m. Tuesday, March 24.

In the seminar Mr. Karmel will present on the initiatives taken by Los Alamos to address such things as green computing, disaster recovery and security. During the presentation he will discuss

How Los Alamos National Laboratory implemented virtualization to reduce their carbon footprint and consolidate data centers across their campus;

How to leverage server virtualization to cost-effectively supplement your disaster-recovery or business-continuity plan;

How to identify “low hanging fruit” for your agency’s green initiatives while achieving a substantial return on your investment; and

Moving computing from the desktop to the data center to enhance your agency’s security.

Sounds like a good place to learn about how some really smart people implement virtualization. I certainly plan on being there. You can read more about it here.

-j

Red Hat and Microsoft enter virtualization support agreement

Red Hat and Microsoft have entered into a virtualization agreement. The agreement is designed so that Red Hat and Microsoft customers using virtualization from both companies can get support from either group.

Red Hat and Microsoft both emphasize that this agreement is not the same as the agreement that Microsoft has with Novell. That agreement covers such things as intellectual property, code indemnification and licensing.

Red Hat and Microsoft still very much remain competing platform vendors. Red Hat’s GM of virtualization Mike Neil emphasizes that “these agreements do not include any patent or other IP licensing rights.” The agreement with Novell is more of a partnership agreement where Microsoft gives it’s customers coupons to purchase SuSE Linux Enterprise Server from Novell.

Red Hat and Microsoft will enter into each other’s validation phases for their respective virtualization technologies. The results of these tests will be posted throughout the year on the Red Hat and Microsoft websites.

With the base of heterogenous hosts and virtual guests I suspect this is not the only agreement like this that we will see.

-j

Installing Openfiler NAS as a Xen virtual host

If you’ve ever tried to install the Openfiler NAS frontend as a Xen virtual host you probably found that 1) it was not as straightforward as it would seem, 2)getting the right combination of information that is on the web correct in the openfiler config file is difficult and 3)documentation at the Openfiler web site is nonexistent for this setup.

The OS that I use is Centos 5.2 with the native Red Hat Xen virtualization utilities. This should also be good for RHEL 5.2 as well. I downloaded and installed openfiler-2.3-x86.tar.gz as the NAS frontend for my Xen virtual server. The information that I provide here is good for that version. I have a feeling that it may change some from version to version as many of the config files that I found were very similar but enough off to keep the machine from booting.

I use physical volumes for all of my Xen virtual machines. All of this should apply except for the way that file images are defined in the config file.

You will need approximately 3 GB of disk space for the Openfiler front end and swap. You probably should add a little more in case you find that you want some applications installed that don’t come with Openfiler. By default Openfiler will use about 1 GB of that space for swap.

[root@openfiler ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 6.0G 2.3G 3.4G 40% /
none 130M 0 130M 0% /dev/shm


[root@openfiler ~]# free
total used free shared buffers cached
Mem: 264596 218432 46164 0 84448 55240
-/+ buffers/cache: 78744 185852
Swap: 1048568 0 1048568


After deciding how much disk space that you want to use create it using your desired method-physical disk or file image. Once you have it created format it with ext3 or with your favorite file system. Now mount it and extract the tar ball from where you saved it into mounted directory. I created the directory openfiler under /mnt and mounted the disk there.

lvcreate -L+6G -n openfiler linux-virtuals.hdd1
mkfs -j -L openfiler /dev/linux-virtuals.hdd1/openfiler
mkdir /mnt/openfiler
mount /dev/linux-virtuals.hdd1/openfiler /mnt/openfiler
cd /mnt/openfiler
tar xzvf /srv/secure/openfiler-2.3-x86.tar.gz #path to saved tarball


You need to create a directory to hold the boot files on your Xen host machine. I called mine boot-openfiler. Copy all of the files from /mnt/openfiler/boot/* to /boot-openfiler. Next copy the openfiler kernel modules to /lib/modules.

mkdir /boot-openfiler
cp -a /mnt/openfiler/boot/* /boot-openfiler/
cp -a /mnt/openfiler/lib/modules/2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU /lib/modules/


Now we need to create our openfiler NAS config file. cd to /etc/xen and create the config file for openfiler. You can use the one shown below as a template. Be sure that you create a unique uuid, MAC address and adjust your disk paths.

cd /etc/xen
vim openfiler

Notice that the kernel= and and ramdisk= point to the /mnt/openfiler/boot/* files that we copied point to the boot-openfiler directory that we created. You will also need to add your root=xvda1 ro to the config file. The “phy:/dev/sda,xvdb,w” entry that you see above is the 1TB SimpleTech external drive for which I am using the Openfiler NAS frontend.

Now unmount the partition that you mounted for the Openfiler files and start the domain with the typical xm create -c openfiler or the virsh command.

A cautionary note: Don’t copy the config file from above. WordPress seems to do something to the lines and they don’t start and end correctly ie. they won’t be parsed correctly when starting the domain.

Have fun with your new Openfile NAS!

-j

Adding the iptables firewall to the Xen domU (part 2)

In my last column we set up a physical NIC in our Xen domU to expose it to the internet and setup our iptables firewall.

At this point you should have 2 interfaces in your domU. One should be facing the internet and have an IP Address assigned from your ISP. The other should be a typical Xen interface with a static IP that connects to the rest of your network.

To start off our iptables network let’s open up the system-config-security application and make sure that iptables is enabled. Go ahead and close this once that is done. That should create a standard Red Hat\CentOS firewall setup as a starting point. You can check this by issuing the command:

iptables -L


Notice the chain that Red Hat\Centos adds to the typical iptables -L output. It is referenced by the input and forward chains. Generally when you put in a reference to the input chain you need a corresponding reference to the forward chain. This extra chain is the one that we will work with the most. Since it is referenced by both the forward and input chains we don’t need to put corresponding rules in both chains It is called:

RH-Firewall-1-INPUT


The first thing that we want to do is get the machines on our network out to the internet. We do this by using the nat table and the postrouting chain. This is the command to accomplish that:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

This will let any internet request from your internal network access the internet. My internet facing NIC is eth1. Your’s may vary. Notice the -o eth1. This indicates that it is looking for outbound packets on eth1.

By default anything coming in from the internet is blocked. You’re probably going to want to let ssh and maybe openvpn come in from the internet. The solution that I use for this is to use domUs behind the firewall so that these requests land there rather than on the firewall machine. Here is how to setup an inbound request and have it directed to the landing server. From there you can go where you need on the network.

##ssh
iptables -t nat -A PREROUTING -i eth1 -p tcp –dport 22 -j DNAT –to 172.16.0.201
##openvpn
iptables -t nat -A PREROUTING -i eth1 -p tcp –dport 1194 -j DNAT –to 172.16.0.201

Any port that you need uses the exact same syntax except for the port number.

We also need to enable port forwarding so that it will survive a reboot. Use the following commands to enable it for your current session and set it up to survive a reboot:

[root@virtual-host ~]# sysctl -w net.ipv4.ip_forward=1
[root@virtual-host ~]# sysctl -p
#output
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
[root@virtual-host ~]#

As we can see from the first line under #output ip forwarding is set to 1 which means that it is turned on.

Note that if you go back and use any of the firewall GUIs provided you will lose all of the settings that used the nat table. I suggest that you stick with the command line after making your initial setup.

Here is what my iptables output looks like:

[root@fw0 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all — anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all — anywhere anywhere
ACCEPT tcp — anywhere anywhere tcp dpt:servicetag
ACCEPT udp — anywhere anywhere udp dpt:servicetag

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain RH-Firewall-1-INPUT (2 references)
target prot opt source destination
DROP tcp —  yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:https
DROP tcp —  yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:http
ACCEPT all — anywhere anywhere
ACCEPT all — anywhere anywhere
ACCEPT icmp — anywhere anywhere icmp any
ACCEPT esp — anywhere anywhere
ACCEPT ah — anywhere anywhere
ACCEPT udp — anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp — anywhere anywhere udp dpt:ipp
ACCEPT udp — anywhere anywhere state NEW,RELATED,ESTABLISHED udp dpt:servicetag
ACCEPT tcp — anywhere anywhere state NEW,RELATED,ESTABLISHED tcp dpt:servicetag
ACCEPT tcp — anywhere anywhere tcp dpt:ipp
ACCEPT all — anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:domain
ACCEPT udp — anywhere anywhere state NEW udp dpt:domain
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:smtp
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp — anywhere anywhere state NEW tcp dpt:https
REJECT all — anywhere anywhere reject-with icmp-host-prohibited
[root@fw0 ~]#

The two drops that you see at the top of the input chain are from somebody that kept hitting on my web server. Usually if you want to put a drop in against a specific target your will want to insert (I) it at the top of the chain like so:

iptables -I RH-Firewall-1-INPUT 1 -p tcp –dport 80 –source 11.22.33.444 -j DROP

The 1 just after INPUT instructs iptables to make that the first rule in the chain. Since both the input and forward chains are reference by the RH-Firewall-1-INPUT chain we don’t have to concern ourselves with putting the same rule in the forward chain.

I hope this helps you get started with your domU firewall.

-j

Setting up a physical NIC for a firewall on a Xen domU (Part 1)

Recently I brought up a new Xen server that needed an iptables firewall on a domU. My first thought had been to setup the firewall on dom0 but that turned out to be a difficult task because of all of the virtual interfaces that are created. Red Hat/Centos also installs a set of rules by default to make sure that all of these interfaces will interact with each other properly. Onward to domU.

The first thing necessary to setting up a domU firewall that is exposed to the internet is to “hide” an interface from dom0 and import it into the domU firewall machine. To start we need to do a few things. Ultimately this is going to cause of reboot of dom0 so consider if this is feasible for your situation.

Let’s get started. First we need to get some numbers from the interface To do this use the lspci command.

[root@virtual-host ~]# lspci |grep -i ethernet
==>01:02.0 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:02.1 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

As you can see I have three interfaces on this machine. The marked interface requires an entry into modprobe.conf and the xen firewall configuration file.

##modprobe.conf
options pciback hide=(01:02.0)

##xen firewall configuration
pci = [ "01:02.0" ]


Now we need to use the lspci -n command and use this entry in the xend-pci-permissive.sxp file under /etc/xen.

[root@virtual-host xen]# lspci -n
==>01:02.0 0200: 8086:1079 (rev 03)
01:02.1 0200: 8086:1079 (rev 03)
1:06.0 0200: 8086:100e (rev 02)

Match the pci numbers from the lspci command to find the correct line. You’ll want the last 8 characters of this line. In the code above we want the 8086:1079 part of the output.

Open the xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
(’8086:1079′)
)


Once we have this done we need to make a new initrd image that preloads the pciback module. Before running the following code you should make a copy of your current initrd. If you run into problems you can use this to replace the one that you created and try again. Use the following code to create the new initrd:

cd /boot
mkinitrd -f –preload pciback initrd-$(uname -r).img $(uname -r)


After creating the new initrd it’s time to reboot and check your work.

Once dom0 is up we need to look for certain entries in /var/log/messages:

[root@virtual-host ~]# grep pciback /var/log/messages
vpci: 0000:01:02.0: assign to virtual slot 0
virtual-host kernel: pciback 0000:01:02.0: seizing device
virtual-host kernel: pciback 0000:01:02.0: enabling permissive mode configuration space accesses!
virtual-host kernel: pciback 0000:01:02.0: permissive mode is potentially unsafe!
virtual-host kernel: pciback: vpci: 0000:01:02.0: assign to virtual slot 0


Once you see that the device is seized and assigned to a virtual slot check your firewall machine to make sure it is getting an ip from your ISP as well as connected to your local lan IP.

[root@fw0 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3E:36:73:82
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::216:3eff:fe36:7382/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:548690 errors:0 dropped:0 overruns:0 frame:0
TX packets:291190 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:486044371 (463.5 MiB) TX bytes:47023339 (44.8 MiB)

collisions:0 txqueuelen:100
RX bytes:50257788 (47.9 MiB) TX bytes:487593757 (465.0 MiB)
Base address:0xb400 Memory:fea40000-fea60000


As you can see from the above output eth0 is connected to my lan and eth1 has received it’s internet address so that we are connected to the internet. The OS (Red Hat/CentOS) should create the entry for eth1 without any input on your part.

Please read my next post for setting up iptables in your domU.

-j

Setting up your firewall on domU with iptables

As discussed in an earlier post you must first hide your NIC from dom0 to set up your iptables firewall on your domU. After you have successfully hidden the NIC from dom0 then we can proceed to our domU firewall setup.

You must first decide which domU that you are going to use for a firewall. Personally I prefer my firewall domU to have nothing on it but iptables. I can then use POSTROUTING and PREROUTING to nat my outbound packets and redirect the new inbound packets to their correct destinations. After you have your domU built and working properly you need to make the following entry into the configuration file:

name = “fw0″
uuid = “203e2874-a08b-4065-7155-cdad1b5b7341″
maxmem = 256
memory = 256
vcpus = 1
bootloader = “/usr/bin/pygrub”
on_poweroff = “destroy”
on_reboot = “restart”
on_crash = “restart”
vfb = [ "type=vnc,vncunused=1,keymap=en-us" ]
disk = [ "phy:/dev/linux-virtuals/secure,xvda,w" ]
vif = [ "mac=00:16:3e:36:73:82,bridge=xenbr0" ]
pci = [ '01:02.0' ] =====Should be the same as obtained from your lspci command


Now start your domU. You should see a second interface, eth1, show up when you use ifconfig. There is no need to build an ifcfg-eth1 file for this as the operating system will take care of it for you. This is the interface that is connected to your DSL\Cable connection to the internet. Make sure that you have a cable plugged into the physical interface that [ '01:02.0' ] represents and the other end into your Cable or DSL modem. You should see that it gets a publicly routed interface like this:

[root@fw0 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3E:36:73:82
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::216:3eff:fe36:7382/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:37856 errors:0 dropped:0 overruns:0 frame:0
TX packets:27763 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7935825 (7.5 MiB) TX bytes:11696196 (11.1 MiB)

[root@fw0 ~]#


The x’s are place in the last two octets for security reasons. However you can see by the first two octets that this is a publicly routable interface that got it’s address from my ISP provider.

Now to get your machines on your LAN out to the internet two things must happen. Their default gateway must be set to the ip address of eth0 on your domU. In my case this is 172.16.0.254. This is quite simple if you are using DHCP. Just make an entry like this into the dhcpd.conf file:

subnet 172.16.0.0 netmask 255.255.0.0 {
range 172.16.0.111 172.16.0.150;
option routers 172.16.0.254;=====set this option for your default gateway
option broadcast-address 172.16.255.255;
default-lease-time 259200;
max-lease-time 604800;
option domain-name-servers 172.16.0.205, xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
}


If you’re not using DHCP then you can make an entry either into /etc/sysconfig/network or /etc/sysconfig/network-scripts/ifcfg-eth* where the * is replaced by whatever your interface number is:

GATEWAY=172.16.0.254


Once that is done now we need to set up our masquerade so that our outbound packets are nat’d and we can browse the internet. On the firewall machine issue the following commands:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
service iptables save
service iptables restart


There you have it. Your domU is now connected to the internet, firewalling your network and allowing your internal machines on your LAN to browse the internet. This setup was done on CentOS 5.2 with the native virtualization that is built in.

-j

Finally, Virtualization testing for Xen, ESX, Hyper-V and more

Information Week has announced a test among several virtual machine vendors including Citrix, ESX, Microsoft’s Hyper-V, Parallels and Virtual Iron.

This comparison is what Information Week calls a rolling review. This is where, over a period of time, all of the products are pitted against one another.

The testing starts from bare metal and includes four VM Hosts. Two are on identical high end servers newly purchased and two on lesser powered servers that have been repurposed for virtual host use.

The evaluation will consist of each vendor’s ease of setup, configuration, data and network connectivity. Conversion tools supplied by each of the vendors will be used to migrate real world servers running Windows 2000, 2003, 2008, Windows XP, and Debian Linux.

The first reviews will begin with Citrix XenServer. Identical runs of Microsfts’ Hyper-V and VMWare’s ESX server will follow. After these tests are completed Information Week’s Lab will provide a comprehensive overview of the smaller vendors. Following the testing a comprehensive wrapup detailing the features, performance, and price differences among the different Virtualization Vendors will be provided.

Follow the results of the rolling review here.

-j

CentOS 5 and pciback aka hiding pci card from Xen DOM0

I just recently spent more hours than I care to think about getting a NIC in Xen Dom0 hidden and passed to a DomU via pciback in CentOS 5.

Lesson number 1: pciback or pciback.hide is now a module. Putting it on GRUB’s kernel line is no longer sufficient.

Lesson number 2: If you NIC module does not load until late in the boot process either use another one or put it in your initrd. Davicom cards are a good example of this. Use an Intel.

Lesson number 3: Much of the information that you find online about what to put in your /etc/modprobe.config incorrect.

So now that we’ve covered all of that here is what you need to do to use pciback in CentOS 5.

Step 1. Put the following and only the following in your /etc/modprobe.conf:

options pciback hide=(01:06.0)

The numbers that you see there are found by running lspci |grep ethernet (or whatever you want to hide such as vga, usb, etc).

[root@virtual-host xen]# lspci |grep -i ethernet
01:02.0 Ethernet controller: Davicom Semiconductor, Inc. 21×4x DEC-Tulip compatible 10/100 Ethernet (rev 31)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
[root@virtual-host xen]#


Now you need to create a new initrd image. Before doing so make a backup copy of the one that you are currently using. Then create a new initrd so that the pciback module is loaded early on.

mkinitrd -f –preload=pciback /boot/initrd-$(uname -r).img $(uname -r)

This will place the new initrd into your boot directory and overwrite the old one.

Moving on to the /etc/xen directory we have a little work do here. In the domU config file make two entries like the following:

pci = [ "01:06.0"]
vif = [ '' ]


Now open /etc/xen/xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
#(’0123:4567:89AB:CDEF’)
(’8086:100e’) ##Everything but this entry is already in here for an example.
)


You get the 8086:100e number from running lspci -n:

[root@virtual-host xen]# lspci -n
00:00.0 0600: 8086:254c (rev 01)
00:00.1 ff00: 8086:2541 (rev 01)
00:1d.0 0c03: 8086:2482 (rev 02)
00:1d.1 0c03: 8086:2484 (rev 02)
00:1e.0 0604: 8086:244e (rev 42)
00:1f.0 0601: 8086:2480 (rev 02)
00:1f.1 0101: 8086:248b (rev 02)
00:1f.3 0c05: 8086:2483 (rev 02)
01:02.0 0200: 1282:9102 (rev 31)
01:04.0 0300: 1002:4752 (rev 27)
01:06.0 0200: 8086:100e (rev 02)
As you can see those number match up with the 01:06.0 number that we used in modprobe.conf and the domU config file.