Red Hat and Microsoft both emphasize that this agreement is not the same as the agreement that Microsoft has with Novell. That agreement covers such things as intellectual property, code indemnification and licensing.

Red Hat and Microsoft still very much remain competing platform vendors. Red Hat’s GM of virtualization Mike Neil emphasizes that “these agreements do not include any patent or other IP licensing rights.” The agreement with Novell is more of a partnership agreement where Microsoft gives it’s customers coupons to purchase SuSE Linux Enterprise Server from Novell.

Red Hat and Microsoft will enter into each other’s validation phases for their respective virtualization technologies. The results of these tests will be posted throughout the year on the Red Hat and Microsoft websites.

With the base of heterogenous hosts and virtual guests I suspect this is not the only agreement like this that we will see.

-j

There are many excellent 3rd party security tools out there for you to install on your system. Prior to installing these though you should review the tools that are already on your system. There is probably already a package included with the system that will accomplish what you need.

Why not use these tools? The major Linux distributions have gone to considerable expense to test these tools and make sure that they will not break anything on your system. When you consider the many 3rd party applications that are certified for a distribution such as Lotus Domino and JBoss this becomes even more critical. These applications are generally installed because they are mission critical. You don’t want to install a non certified security application only to find that it breaks or creates a security flaw in your certified mission critical application. Don’t do this.

A pet peeve of mine has always been the idea of “point and click and know not what I just did” that many administrators perform. While this seems to be more prevalent in the Windows world it exists in the *nix world as well. Generally the idea of text configuration files can overcome this but not always. Take, for example, the website securecentos.com (not affiliated with CentOS). One of the things that they want you to do is patch your kernel with a patch from http://www.grsecurity.com/. Doing something like this should raise a red flag immediately. Do you know what the patch is fixing and/or how it is making your machine more secure? If you can’t answer yes to this then don’t do it with this or any other patch except one from your vendor.

Aside from that when your vendor releases a kernel update you are going to have to go and redo the whole process again. This can quite quickly become heavy with administrative costs. If your machines are duplicated across the network now you have to go and install this on all of them. And again when you run a kernel update. Don’t do this.

You should never download a configuration file that affects the core of your machine without knowing exactly what it does. Using the same site above they have many configuration files that they want you to download and put into production on your machine(s). There is even a sysctl.conf file which affects many core processes of your machine and how they operate. At the time of this post comments in this file are non existent. This amounts to the notion of “point and click and know not what I just did” mentioned above. Don’t do this.

I don’t mean to single out securecentos.com. It just happens to be the one that I ran across today among the many out there asking administrators to do some things that they should think twice about.. I’m sure that they mean well. If I got out my sysctl manual I could find out what each of those changes would to do to my machine. However I’m not going to..if they want me to use their product/advice then those should be clearly documented either in the file or with a url embedded in the file that leads to that information.

Be smart with your machines! Don’t go putting configuration files in service, clicking on buttons that affect the security or core services of your machine or installing 3rd party applications that may already have the equivalent tested on your machine without knowing exactly what other files and applications they are going to affect.

-j

Red Hat released version 5.3 on January 21st of this year. The CentOS developers generally follow with a CentOS release about 3-5 weeks after Red Hat. This should put the release as generally available around March 1st.

We can expect to see some very nice feature enhancements on this release. NetWorkManager and wpa_supplicant have a whole host of updates listed. This means improved wireless security and better driver support. For those of us using Broadcom wireless drivers the b43 driver from linuxwireless.org has been backported. Following the links on that page should lead you to the proper firmware as well.

The new ext4 filesystem is also incuded in the new release. Laptop users like myself will be glad to know that anaconda now supports encrypted block devices during installation. Red Hat continues their commitment to Xen and has released many updates for virtualizaton including support for up to 126 CPUs in the x86_64 Xen-based hypervisor (up to 32 CPUs per virtual server) and support for up to 1TB memory per host on x86_64 (up to 80GB per virtual server).

Other enhancements include 802.1q VLAN tagging support for kickstart, iSCSI installation and boot support, ability to install Xen and KVM guests and for fibre channel users Emulex FCoE HBA support through the lpfc driver and QLogic FCoE HBA support through qla2xxx driver. See a full list of new features here.

With all of these new enhancements desktop users and server administrators are sure to be pleased.

-j

I recently accepted a 3-5 week consulting job about 400 miles from where I live. I had to leave in the late afternoon because I had an interview for another job, this one closer to home and about 60 miles in the wrong direction. This meant that I wouldn’t actually be on the road until 1500 EST or later.

After the interview off I went. About 90 miles into the drive my truck started missing so badly that it was shaking as I drove and the “Service Engine Soon” light came on.

I called my wife and asked her to locate an auto parts store in the town that was coming up, Champaign, IL. 60 miles of a teeth rattling miss in my engine. This was fun. She called back with directions to a store that was about 10 minutes off of the highway.

I arrived there about 1700 CST. If you’ve ever been in the midwest in January you know that it starts getting dark between 1730-1800. I had them put a tester on the truck and sure enough the Coil on Plug 3 was bad. To top off everything it was 10 degrees F outside.

I had somewhat expected something like this as my truck had been acting up for the last few days. With that in mind I had brought along enough tools to pull a spark plug which meant I had the necessary tools to remove the coil.

I don’t know if you’ve ever been under the hood of an F150 4 wheel drive but some of the plugs and coils can be difficult to reach. It is also high enough that it is helpful if you have a step stool of some sort. Fortunately the coil/plug 3 is not under the firewall so it wasn’t too bad. I did have to crawl up onto the grill housing to get to the coil though.

I bought the coil and went to work. It took me about 10 or 15 minutes to get the old coil off and another 20-30 to get the new one on. By now I’m working by the light of the auto parts store and the temperature is dropping. Cold, very cold. I’m glad the wind wasn’t blowing or it would have almost been unbearable.

The rest of the drive was uneventful although driving through St. Louis wasn’t much fun. Finally I reach the hotel in Rolla, MO about 2200 CST. This puts me about 4-5 hours from my destination. Ahh, I say to myself, the hotel has wireless and breakfast so I’ll just hit the sack and check email and what have you tomorrow morning.

So much for day 1.

-j

The OS that I use is Centos 5.2 with the native Red Hat Xen virtualization utilities. This should also be good for RHEL 5.2 as well. I downloaded and installed openfiler-2.3-x86.tar.gz as the NAS frontend for my Xen virtual server. The information that I provide here is good for that version. I have a feeling that it may change some from version to version as many of the config files that I found were very similar but enough off to keep the machine from booting.

I use physical volumes for all of my Xen virtual machines. All of this should apply except for the way that file images are defined in the config file.

You will need approximately 3 GB of disk space for the Openfiler front end and swap. You probably should add a little more in case you find that you want some applications installed that don’t come with Openfiler. By default Openfiler will use about 1 GB of that space for swap.

[root@openfiler ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/xvda1 6.0G 2.3G 3.4G 40% /
none 130M 0 130M 0% /dev/shm


[root@openfiler ~]# free
total used free shared buffers cached
Mem: 264596 218432 46164 0 84448 55240
-/+ buffers/cache: 78744 185852
Swap: 1048568 0 1048568


After deciding how much disk space that you want to use create it using your desired method-physical disk or file image. Once you have it created format it with ext3 or with your favorite file system. Now mount it and extract the tar ball from where you saved it into mounted directory. I created the directory openfiler under /mnt and mounted the disk there.

lvcreate -L+6G -n openfiler linux-virtuals.hdd1
mkfs -j -L openfiler /dev/linux-virtuals.hdd1/openfiler
mkdir /mnt/openfiler
mount /dev/linux-virtuals.hdd1/openfiler /mnt/openfiler
cd /mnt/openfiler
tar xzvf /srv/secure/openfiler-2.3-x86.tar.gz #path to saved tarball


You need to create a directory to hold the boot files on your Xen host machine. I called mine boot-openfiler. Copy all of the files from /mnt/openfiler/boot/* to /boot-openfiler. Next copy the openfiler kernel modules to /lib/modules.

mkdir /boot-openfiler
cp -a /mnt/openfiler/boot/* /boot-openfiler/
cp -a /mnt/openfiler/lib/modules/2.6.21.7-3.20.smp.pae.gcc3.4.x86.i686.xen.domU /lib/modules/


Now we need to create our openfiler NAS config file. cd to /etc/xen and create the config file for openfiler. You can use the one shown below as a template. Be sure that you create a unique uuid, MAC address and adjust your disk paths.

cd /etc/xen
vim openfiler

Notice that the kernel= and and ramdisk= point to the /mnt/openfiler/boot/* files that we copied point to the boot-openfiler directory that we created. You will also need to add your root=xvda1 ro to the config file. The “phy:/dev/sda,xvdb,w” entry that you see above is the 1TB SimpleTech external drive for which I am using the Openfiler NAS frontend.

Now unmount the partition that you mounted for the Openfiler files and start the domain with the typical xm create -c openfiler or the virsh command.

A cautionary note: Don’t copy the config file from above. WordPress seems to do something to the lines and they don’t start and end correctly ie. they won’t be parsed correctly when starting the domain.

Have fun with your new Openfile NAS!

-j

At this point you should have 2 interfaces in your domU. One should be facing the internet and have an IP Address assigned from your ISP. The other should be a typical Xen interface with a static IP that connects to the rest of your network.

To start off our iptables network let’s open up the system-config-security application and make sure that iptables is enabled. Go ahead and close this once that is done. That should create a standard Red Hat\CentOS firewall setup as a starting point. You can check this by issuing the command:

iptables -L


Notice the chain that Red Hat\Centos adds to the typical iptables -L output. It is referenced by the input and forward chains. Generally when you put in a reference to the input chain you need a corresponding reference to the forward chain. This extra chain is the one that we will work with the most. Since it is referenced by both the forward and input chains we don’t need to put corresponding rules in both chains It is called:

RH-Firewall-1-INPUT


The first thing that we want to do is get the machines on our network out to the internet. We do this by using the nat table and the postrouting chain. This is the command to accomplish that:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

This will let any internet request from your internal network access the internet. My internet facing NIC is eth1. Your’s may vary. Notice the -o eth1. This indicates that it is looking for outbound packets on eth1.

By default anything coming in from the internet is blocked. You’re probably going to want to let ssh and maybe openvpn come in from the internet. The solution that I use for this is to use domUs behind the firewall so that these requests land there rather than on the firewall machine. Here is how to setup an inbound request and have it directed to the landing server. From there you can go where you need on the network.

##ssh
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 22 -j DNAT --to 172.16.0.201
##openvpn
iptables -t nat -A PREROUTING -i eth1 -p tcp --dport 1194 -j DNAT --to 172.16.0.201

Any port that you need uses the exact same syntax except for the port number.

We also need to enable port forwarding so that it will survive a reboot. Use the following commands to enable it for your current session and set it up to survive a reboot:

[root@virtual-host ~]# sysctl -w net.ipv4.ip_forward=1
[root@virtual-host ~]# sysctl -p
#output
net.ipv4.ip_forward = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
[root@virtual-host ~]#

As we can see from the first line under #output ip forwarding is set to 1 which means that it is turned on.

Note that if you go back and use any of the firewall GUIs provided you will lose all of the settings that used the nat table. I suggest that you stick with the command line after making your initial setup.

Here is what my iptables output looks like:

[root@fw0 ~]# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all -- anywhere anywhere

Chain FORWARD (policy ACCEPT)
target prot opt source destination
RH-Firewall-1-INPUT all -- anywhere anywhere
ACCEPT tcp -- anywhere anywhere tcp dpt:servicetag
ACCEPT udp -- anywhere anywhere udp dpt:servicetag

Chain OUTPUT (policy ACCEPT)
target prot opt source destination

Chain RH-Firewall-1-INPUT (2 references)
target prot opt source destination
DROP tcp -- yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:https
DROP tcp -- yktgi01e0-s4.watson.ibm.com anywhere tcp dpt:http
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
ACCEPT icmp -- anywhere anywhere icmp any
ACCEPT esp -- anywhere anywhere
ACCEPT ah -- anywhere anywhere
ACCEPT udp -- anywhere 224.0.0.251 udp dpt:mdns
ACCEPT udp -- anywhere anywhere udp dpt:ipp
ACCEPT udp -- anywhere anywhere state NEW,RELATED,ESTABLISHED udp dpt:servicetag
ACCEPT tcp -- anywhere anywhere state NEW,RELATED,ESTABLISHED tcp dpt:servicetag
ACCEPT tcp -- anywhere anywhere tcp dpt:ipp
ACCEPT all -- anywhere anywhere state RELATED,ESTABLISHED
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:domain
ACCEPT udp -- anywhere anywhere state NEW udp dpt:domain
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:ssh
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:smtp
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:https
REJECT all -- anywhere anywhere reject-with icmp-host-prohibited
[root@fw0 ~]#

The two drops that you see at the top of the input chain are from somebody that kept hitting on my web server. Usually if you want to put a drop in against a specific target your will want to insert (I) it at the top of the chain like so:

iptables -I RH-Firewall-1-INPUT 1 -p tcp --dport 80 --source 11.22.33.444 -j DROP

The 1 just after INPUT instructs iptables to make that the first rule in the chain. Since both the input and forward chains are reference by the RH-Firewall-1-INPUT chain we don't have to concern ourselves with putting the same rule in the forward chain.

I hope this helps you get started with your domU firewall.

-j

The first thing necessary to setting up a domU firewall that is exposed to the internet is to “hide” an interface from dom0 and import it into the domU firewall machine. To start we need to do a few things. Ultimately this is going to cause of reboot of dom0 so consider if this is feasible for your situation.

Let’s get started. First we need to get some numbers from the interface To do this use the lspci command.

[root@virtual-host ~]# lspci |grep -i ethernet
==>01:02.0 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:02.1 Ethernet controller: Intel Corporation 82546GB Gigabit Ethernet Controller (rev 03)
01:06.0 Ethernet controller: Intel Corporation 82540EM Gigabit Ethernet Controller (rev 02)

As you can see I have three interfaces on this machine. The marked interface requires an entry into modprobe.conf and the xen firewall configuration file.

##modprobe.conf
options pciback hide=(01:02.0)

##xen firewall configuration
pci = [ "01:02.0" ]


Now we need to use the lspci -n command and use this entry in the xend-pci-permissive.sxp file under /etc/xen.

[root@virtual-host xen]# lspci -n
==>01:02.0 0200: 8086:1079 (rev 03)
01:02.1 0200: 8086:1079 (rev 03)
1:06.0 0200: 8086:100e (rev 02)

Match the pci numbers from the lspci command to find the correct line. You’ll want the last 8 characters of this line. In the code above we want the 8086:1079 part of the output.

Open the xend-pci-permissive.sxp and make an entry like the following:

(unconstrained_dev_ids
('8086:1079')
)


Once we have this done we need to make a new initrd image that preloads the pciback module. Before running the following code you should make a copy of your current initrd. If you run into problems you can use this to replace the one that you created and try again. Use the following code to create the new initrd:

cd /boot
mkinitrd -f --preload pciback initrd-$(uname -r).img $(uname -r)


After creating the new initrd it’s time to reboot and check your work.

Once dom0 is up we need to look for certain entries in /var/log/messages:

[root@virtual-host ~]# grep pciback /var/log/messages
vpci: 0000:01:02.0: assign to virtual slot 0
virtual-host kernel: pciback 0000:01:02.0: seizing device
virtual-host kernel: pciback 0000:01:02.0: enabling permissive mode configuration space accesses!
virtual-host kernel: pciback 0000:01:02.0: permissive mode is potentially unsafe!
virtual-host kernel: pciback: vpci: 0000:01:02.0: assign to virtual slot 0


Once you see that the device is seized and assigned to a virtual slot check your firewall machine to make sure it is getting an ip from your ISP as well as connected to your local lan IP.

[root@fw0 ~]# ifconfig
eth0 Link encap:Ethernet HWaddr 00:16:3E:36:73:82
inet addr:172.16.0.254 Bcast:172.16.255.255 Mask:255.255.0.0
inet6 addr: fe80::216:3eff:fe36:7382/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:548690 errors:0 dropped:0 overruns:0 frame:0
TX packets:291190 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:486044371 (463.5 MiB) TX bytes:47023339 (44.8 MiB)

collisions:0 txqueuelen:100
RX bytes:50257788 (47.9 MiB) TX bytes:487593757 (465.0 MiB)
Base address:0xb400 Memory:fea40000-fea60000


As you can see from the above output eth0 is connected to my lan and eth1 has received it’s internet address so that we are connected to the internet. The OS (Red Hat/CentOS) should create the entry for eth1 without any input on your part.

Please read my next post for setting up iptables in your domU.

-j

Running this command however gave some strange output, primarily ? marks. I thought this was a little strange but had other things that I needed to do at the time and decided that I would look into it later.

This weekend was that time. Let me say that I use SELinux on my Red Hat and CentOS machines and think that it is a very good way to help secure a machine. However it is anything but intuitive. If it weren’t for some very good documentation at Red Hat I probably never would have been successful at using this security tool. Mind you I’m no guru with it but I have six servers using it and I know how to troubleshoot SeLinux problems.

Which brings me to the part about Ubuntu and SELinux that I find disturbing. Doing some Google searching I ran across two pages regarding Ubuntu and SELinux. Both of them had no usable information in them other than how to install SELinux. Nothing about what to expect, how to troubleshoot, what a context or a boolean is nor did it mention if Ubuntu provided any troubleshoooting tools like setroubleshoot. You can find these two pages here and here.

The documentation only warned that SELinux is for experienced users. While that is an understatement how do they expect people to start using it to protect their machines? It would seem to indicate that they have no real interest in their users having the ability to use SELinux. I personally think that is a shame. I also believe that it is going to hurt their efforts at becoming enterprise ready especially with their server product. I certainly won’t be installing Ubuntu on any of my critical machines.

-j

The Sun Inventory application tracks these items by installing a small application on the machine that you want to inventory. Initially it will report back the hardware and operating system. As qualified applications are installed the agent will report these back to the Sun Inventory application without any interaction on your part.

Getting started is simple. Go here to get started. If you don’t have a Sun account go ahead and sign up. Once you are signed in it is a 3 step process to get started.

Step one is to download what are known as service tags. This is the application that you will install to “tag” your inventory so that it can be put into the application. Tags are available for Red Hat Enterprise Linux, Suse Enterprise Linux, Solaris and Windows. Download the appropriate tag for your operating system and install it on the machine on which you want to inventory. The tagging also works on Virtualized Machines from Red Hat Virtualization and from VMs using Virtual Box. I didn’t check any other virtualization applications.

Steps two and three are discovering and registering your “gear” as Sun calls it. This downloads a small java program onto your machine to help in finding and registering tag ready machines. With this application you can find your machines in various ways such as hostname, subnet and ip address. Below is a screen shot of the information that you can use to find your tagged your machines.

Once you have done this a screen will pop up showing the gear that the registration client found. You will then login to your Sun Account and choose which products that you want to register. Once they are registered what you will see is like the following screen shot.

As you can see I have my 1u server tagged along with the host and virtual operatings systems. The OpenSolaris machine is running on Virtual Box. The OpenOffice application was installed after I tagged and registered the machine. Since the tag runs as a service it picked up the OpenOffice application and registered it as part of the OpenSolaris machine.

This is a great way to get your machines and related software inventoried and get control of it.

-j

First up are the /var/log/audit/audit.log, /var/log/security and /var/log/messages. If selinux is set to enforcing and you’ve just installed a new application or created a file or directory that is not allowing proper access these three files are the place to go. Before you do this make sure the following applications are installed:
setroubleshoot.noarch
setroubleshoot-plugins.noarch
setroubleshoot-server.noarch

After installing these make sure that you start the setroubleshoot application and set it to start on reboot:

/etc/init.d/setroubleshoot start
Starting setroubleshootd: [ OK ]
chkconfig setroubleshoot on


Watch the logs in real time as you attempt to access the application, file or directory like this:

cd /var/logs
tail -f security audit/audit.log messages

After doing this hit enter three times to give you some white space between the old messages and the new ones that are generated. If selinux is giving you a problem you will see something like the following in the messages log:

Nov 5 08:18:44 centos5-dev setroubleshoot: SELinux is preventing access to files with the label, file_t. For complete SELinux messages. run sealert -l d102b5a4-ac6f-470f-aa34-55ac37dafa37


To find out not only what is going on but how to fix it run the sealert -l d102b5a4-ac6f-470f-aa34-55ac37dafa37 command described in the message.

[root@centos5-dev ~]# sealert -l d102b5a4-ac6f-470f-aa34-55ac37dafa37

host=centos5-dev.hendricks.org type=SYSCALL msg=audit(1225891119.851:12): arch=40000003 syscall=5 success=no exit=-13 a0=b7fb2b4b a1=0 a2=bfd8a2b4 a3=8 items=0 ppid=2633 pid=2634 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="hpssd.py" exe="/bin/env" subj=system_u:system_r:hplip_t:s0 key=(null)
[root@centos5-dev ~]#

The part that we are interested in is under the above heading Allowing Access: You can execute the following command as root to relabel your computer system:
"touch /.autorelabel; reboot"
When we run this command this will fix our problem. Note that these problems could run from accessing html pages to allowing a public web directory in your home directory.

Next up we have the command:

chcon --reference

Lets say you are using your localhost as your web server. You decide that you want to add some virtual hosts. You then add the virtual host directories outside of the normal /var/www/html directory. You build your virtual hosts but now you can’t access them. Watching your messages you see that this is definitely an selinux problem. Using the above command we can fix our problem like this:

chcon --reference /var/www/html /srv/www/vhosts #This will fix the selinux properties on the root directory of the virtual hosts
chcon -- reference /var/www/html/* /srv/www/vhosts/* # This will fix the properties on the files in case they are different from the directory

This code references the contexts of the given files or directories and applies them to the new files and directories. Now every time that you add a file or directory under /srv/www/vhosts it will get the proper selinux context.

The last way that we are going to discuss is restorecon. Taking the above scenario under either of the directories you find that some files or directories did not pick up the correct context or maybe none at all. Easy enough to fix:

restorecon /var/www/html

The reason this works is because the restorecon looks at the current contexts of the other files and directories and applies that context to the ones with the incorrect or no context.

There you have it. Keep your sanity and still use SELinux.

-j