At this point you should have 2 interfaces in your domU. One should be facing the internet and have an IP Address assigned from your ISP. The other should be a typical Xen interface with a static IP that connects to the rest of your network.

To start off our iptables network let’s open up the system-config-security application and make sure that iptables is enabled. Go ahead and close this once that is done. That should create a standard Red Hat\CentOS firewall setup as a starting point. You can check this by issuing the command:

iptables -L


Notice the chain that Red Hat\Centos adds to the typical iptables -L output. It is referenced by the input and forward chains. Generally when you put in a reference to the input chain you need a corresponding reference to the forward chain. This extra chain is the one that we will work with the most. Since it is referenced by both the forward and input chains we don’t need to put corresponding rules in both chains It is called:

RH-Firewall-1-INPUT


The first thing that we want to do is get the machines on our network out to the internet. We do this by using the nat table and the postrouting chain. This is the command to accomplish that:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

This will let any internet request from your internal network access the internet. My internet facing NIC is eth1. Your’s may vary. Notice the -o eth1. This indicates that it is looking for outbound packets on eth1.

By default anything coming in from the internet is blocked. You’re probably going to want to let ssh and maybe openvpn come in from the internet. The solution that I use for this is to use domUs behind the firewall so that these requests land there rather than on the firewall machine. Here is how to setup an inbound request and have it directed to the landing server. From there you can go where you need on the network.

##ssh
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 22 -j DNAT --to 172.16.0.201
##openvpn
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 1194 -j DNAT --to 172.16.0.201

Any port that you need uses the exact same syntax except for the port number.

We also need to enable port forwarding so that it will survive a reboot. Use the following commands to enable it for your current session and set it up to survive a reboot:

[root@virtual-host ~]# sysctl -w net.ipv4.ip_forward=1
[root@virtual-host ~]# sysctl -p
#output
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
[root@virtual-host ~]#

As we can see from the first line under #output ip forwarding is set to 1 which means that it is turned on.

Note that if you go back and use any of the firewall GUIs provided you will lose all of the settings that used the nat table. I suggest that you stick with the command line after making your initial setup.

Here is what my iptables output looks like:

[root@fw0 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all -- anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere tcp dpt:servicetag
ACCEPT udp -- anywhere anywhere udp dpt:servicetag

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain RH-Firewall-1-INPUT (2 references)
target prot opt source destination
DROP tcp -- yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:https
DROP tcp -- yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:http
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT udp -- anywhere anywhere state NEW,RELATED,ESTABLISHED udp dpt:servicetag
ACCEPT tcp -- anywhere anywhere state NEW,RELATED,ESTABLISHED tcp dpt:servicetag
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain
ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:smtp
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
[root@fw0 ~]#

The two drops that you see at the top of the input chain are from somebody that kept hitting on my web server. Usually if you want to put a drop in against a specific target your will want to insert (I) it at the top of the chain like so:

iptables -I RH-Firewall-1-INPUT 1 -p tcp --dport 80 --source 11.22.33.444 -j DROP

The 1 just after INPUT instructs iptables to make that the first rule in the chain. Since both the input and forward chains are reference by the RH-Firewall-1-INPUT chain we don't have to concern ourselves with putting the same rule in the forward chain.

I hope this helps you get started with your domU firewall.

-j

The first thing necessary to setting up a domU firewall that is exposed to the internet is to “hide” an interface from dom0 and import it into the domU firewall machine. To start we need to do a few things. Ultimately this is going to cause of reboot of dom0 so consider if this is feasible for your situation.

Let’s get started. First we need to get some numbers from the interface To do this use the lspci command.

[root@virtual-host ~]# lspci |grep -i ethernet
==>01:02.0 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:02.1 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

As you can see I have three interfaces on this machine. The marked interface requires an entry into modprobe.conf and the xen firewall configuration file.

##modprobe.conf
options pciback hide=(01:02.0)

##xen firewall configuration
pci = [ "01:02.0" ]


Now we need to use the lspci -n command and use this entry in the xend-pci-permissive.sxp file under /etc/xen.

[root@virtual-host xen]# lspci -n
==>01:02.0 0200: 8086:1079 (rev 03)
01:02.1 0200: 8086:1079 (rev 03)
1:06.0 0200: 8086:100e (rev 02)

Match the pci numbers from the lspci command to find the correct line. You’ll want the last 8 characters of this line. In the code above we want the 8086:1079 part of the output.

Open the xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
('8086:1079')
)


Once we have this done we need to make a new initrd image that preloads the pciback module. Before running the following code you should make a copy of your current initrd. If you run into problems you can use this to replace the one that you created and try again. Use the following code to create the new initrd:

cd /boot
mkinitrd -f --preload pciback initrd-$(uname -r).img $(uname -r)


After creating the new initrd it’s time to reboot and check your work.

Once dom0 is up we need to look for certain entries in /var/log/messages:

[root@virtual-host ~]# grep pciback /var/log/messages
vpci: 0000:01:02.0: assign to virtual slot 0
virtual-host kernel: pciback 0000:01:02.0: seizing device
virtual-host kernel: pciback 0000:01:02.0: enabling permissive mode configuration space accesses!
virtual-host kernel: pciback 0000:01:02.0: permissive mode is potentially unsafe!
virtual-host kernel: pciback: vpci: 0000:01:02.0: assign to virtual slot 0


Once you see that the device is seized and assigned to a virtual slot check your firewall machine to make sure it is getting an ip from your ISP as well as connected to your local lan IP.

[root@fw0 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3E:36:73:82
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::216:3eff:fe36:7382/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:548690 errors:0 dropped:0 overruns:0 frame:0
TX packets:291190 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:486044371 (463.5 MiB) TX bytes:47023339 (44.8 MiB)

collisions:0 txqueuelen:100
RX bytes:50257788 (47.9 MiB) TX bytes:487593757 (465.0 MiB)
Base address:0xb400 Memory:fea40000-fea60000


As you can see from the above output eth0 is connected to my lan and eth1 has received it’s internet address so that we are connected to the internet. The OS (Red Hat/CentOS) should create the entry for eth1 without any input on your part.

Please read my next post for setting up iptables in your domU.

-j

You must first decide which domU that you are going to use for a firewall. Personally I prefer my firewall domU to have nothing on it but iptables. I can then use POSTROUTING and PREROUTING to nat my outbound packets and redirect the new inbound packets to their correct destinations. After you have your domU built and working properly you need to make the following entry into the configuration file:

name = "fw0"
uuid = "203e2874-a08b-4065-7155-cdad1b5b7341"
maxmem = 256
memory = 256
vcpus = 1
bootloader = "/usr/bin/pygrub"
on_poweroff = "destroy"
on_reboot = "restart"
on_crash = "restart"
vfb = [ "type=vnc,vncunused=1,keymap=en-us" ]
disk = [ "phy:/dev/linux-virtuals/secure,xvda,w" ]
vif = [ "mac=00:16:3e:36:73:82,bridge=xenbr0" ]
pci = [ '01:02.0' ] =====Should be the same as obtained from your lspci command


Now start your domU. You should see a second interface, eth1, show up when you use ifconfig. There is no need to build an ifcfg-eth1 file for this as the operating system will take care of it for you. This is the interface that is connected to your DSL\Cable connection to the internet. Make sure that you have a cable plugged into the physical interface that [ '01:02.0' ] represents and the other end into your Cable or DSL modem. You should see that it gets a publicly routed interface like this:

[root@fw0 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3E:36:73:82
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::216:3eff:fe36:7382/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:37856 errors:0 dropped:0 overruns:0 frame:0
TX packets:27763 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:7935825 (7.5 MiB) TX bytes:11696196 (11.1 MiB)

[root@fw0 ~]#


The x’s are place in the last two octets for security reasons. However you can see by the first two octets that this is a publicly routable interface that got it’s address from my ISP provider.

Now to get your machines on your LAN out to the internet two things must happen. Their default gateway must be set to the ip address of eth0 on your domU. In my case this is 172.16.0.254. This is quite simple if you are using DHCP. Just make an entry like this into the dhcpd.conf file:

subnet 172.16.0.0 netmask 255.255.0.0 {
range 172.16.0.111 172.16.0.150;
option routers 172.16.0.254;=====set this option for your default gateway
option broadcast-address 172.16.255.255;
default-lease-time 259200;
max-lease-time 604800;
option domain-name-servers 172.16.0.205, xxx.xxx.xxx.xxx, xxx.xxx.xxx.xxx;
}


If you’re not using DHCP then you can make an entry either into /etc/sysconfig/network or /etc/sysconfig/network-scripts/ifcfg-eth* where the * is replaced by whatever your interface number is:

GATEWAY=172.16.0.254


Once that is done now we need to set up our masquerade so that our outbound packets are nat’d and we can browse the internet. On the firewall machine issue the following commands:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE
service iptables save
service iptables restart


There you have it. Your domU is now connected to the internet, firewalling your network and allowing your internal machines on your LAN to browse the internet. This setup was done on CentOS 5.2 with the native virtualization that is built in.

-j

Lesson number 1: pciback or pciback.hide is now a module. Putting it on GRUB’s kernel line is no longer sufficient.

Lesson number 2: If you NIC module does not load until late in the boot process either use another one or put it in your initrd. Davicom cards are a good example of this. Use an Intel.

Lesson number 3: Much of the information that you find online about what to put in your /etc/modprobe.config incorrect.

So now that we’ve covered all of that here is what you need to do to use pciback in CentOS 5.

Step 1. Put the following and only the following in your /etc/modprobe.conf:

options pciback hide=(01:06.0)

The numbers that you see there are found by running lspci |grep ethernet (or whatever you want to hide such as vga, usb, etc).

[root@virtual-host xen]# lspci |grep -i ethernet
01:02.0 Ethernet controller: Davicom Semiconductor, Inc. 21x4x DEC-Tulip compatible 10/100 Ethernet (rev 31)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)
[root@virtual-host xen]#


Now you need to create a new initrd image. Before doing so make a backup copy of the one that you are currently using. Then create a new initrd so that the pciback module is loaded early on.

mkinitrd -f --preload=pciback /boot/initrd-$(uname -r).img $(uname -r)

This will place the new initrd into your boot directory and overwrite the old one.

Moving on to the /etc/xen directory we have a little work do here. In the domU config file make two entries like the following:

pci = [ "01:06.0"]
vif = [ '' ]


Now open /etc/xen/xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
#('0123:4567:89AB:CDEF')
('8086:100e') ##Everything but this entry is already in here for an example.
)


You get the 8086:100e number from running lspci -n:

[root@virtual-host xen]# lspci -n
00:00.0 0600: 8086:254c (rev 01)
00:00.1 ff00: 8086:2541 (rev 01)
00:1d.0 0c03: 8086:2482 (rev 02)
00:1d.1 0c03: 8086:2484 (rev 02)
00:1e.0 0604: 8086:244e (rev 42)
00:1f.0 0601: 8086:2480 (rev 02)
00:1f.1 0101: 8086:248b (rev 02)
00:1f.3 0c05: 8086:2483 (rev 02)
01:02.0 0200: 1282:9102 (rev 31)
01:04.0 0300: 1002:4752 (rev 27)
01:06.0 0200: 8086:100e (rev 02)
As you can see those number match up with the 01:06.0 number that we used in modprobe.conf and the domU config file.