At Blue Sky every once in a while we set up Shorewall servers for customers, Linux Based Open Source firewall software, which is actually pretty good for what it can do. They have been running fine on physical boxes, and as virtual machines on ESX 3.x environments for a while now, however an issue arose when we started to install vSphere (ESX/ESXi 4.0 / 4.1) servers over the past year, where suddenly these firewalls just were not moving the data from one network to the other. Typically for customers we install Shorewall in a NAT configuration mode.

This stumped us for a while, it stumped VMware for a while, even the community came along and had a go, and while got me half the solution, it never quite worked.

So for those having any issues, here are a few more things to try, which worked for Blue Sky! (Although I do keep forgetting to run the final part of this!)

The first thing to look out for is the following. As a default, when you create a VM, most of the options will lend you towards setting up a network card as “Flexible”. This is really handy, as in most cases, this is an emulated network card which the drivers are available / installed on most OS’s out there. Handy! And technically you could run with this type of network card on any server. You wouldn’t get the full feature set you get with the other type of cards, but it will do. VMware also offers you a number of other types, but in the new vSphere world, they have a new type called “VMXNET3″. This is only available on version 7 VM’s (which will only go on 4.x ESX servers), there are the other VMXNET drivers, which will work on other systems, but for this case, I a going to stick with VMXNET3.

What we end up doing is, once we have installed the OS with the Flexible driver, and installed the Guest VMware Tools (they contain the drivers for the VMXNET3 network card), I shutdown the VM and re-add the network cards, ensuring they are VMXNET3.

Once the OS boots up, everything seems fine (and though changes in the config to ensure that you are pointing to the new MAC addresses of these network cards!), however, Shorewall still has an issue.

The final bit of the puzzle, lay in something called TCP Segmentation Offloading. Something which virtually, is not needed, which on a physical level is very handy. As this is not needed, and brings a large overhead to the vm, I downloaded a tool called “ethtool” (in the case of Fedora, you can get it using yum), and typed in the following command for all interfaces.

ethtool -K ethX tso off gso off

Without any rebooting, all the systems behind the Shorewall works.

It’s not much at the end of the day, but I thought I would share this with the world!