ESX doesn’t have NAT inbuilt, so here’s how to configure it with the help of a VMware appliance called pfSense (an Open Source [free!] firewall/router).

There are three components in this setup:

1. Host
2. router/pfSense
3. NAT Client

Host

A great read for beginners and those refreshing is the VMware Virtual Networking Concepts whitepaper

Now lets create a network that our NAT’ed VMs will be using.

When prompted under Connection Type, select a ‘VM Network’, as this is for the typical traffic within the Virtual Machine (not IO or management of your machines).

Lets create a vSwitch that doesn’t connect to anything, a dud, a blankie. This will be our NAT’ed environment. It’s quite important that you DON’T connect a network card to this vSwitch to prevent any inadvertent DHCP leakage. Make sure you have nothing selected.

Give it a name to differentiate.

Once you’re done, click finish and you will have something two network available to your VMs:

• VM Network
• NAT Network

Time to setup pfSense. Once you’ve downloaded and extracted it. You have the option of either copying it directly to your datastore and then adding directly to inventory, or importing via the Standalone Converter. I find the latter is always faster.

router/pfSense

Incase you’re converting pfSense first (like I did whilst re-doing it for this post), I recommend you disable the network interfaces until you’ve finished setting up the host networks. We’ll enable these in a later step.

Disabled interfaces

Once the conversion is complete, time to configure our virtual router. pfSense is provided with two NICs out of the box. One for the WAN interface (which is your internal LAN), and one for ‘its’ LAN – the one on which it will be servicing DHCP requests.

Mark down the last 4 digits of the MAC address, these will help to validate the following step.

Configuring pfSense NICs

Start the pfSense VM. You will be guided through the mapping of the interfaces, and just to make sure – check to see the MAC addresses matching to the VM Network (in my case 67:3c) and NAT Network (67:46).

Upon following the wizard, and if you’ve followed everything accordingly (or rather I documented the steps properly) you will be shown the interfaces within pfSense, their mapping (WAN vs LAN) and IP addresses.

NAT Client[s]

You are now ready to assign clients on this host to the NAT Network and have them pick up addresses dished out by your shiny new appliance.

The whole setup takes just under 5 minutes from start to finish to complete.

Exception when adding a network - click to enlarge

Adding a new port (i.e. a vMotion interface) to a vSwitch on vSphere/ESX leads to this lovely error message. If you check your vpxd.log you’ll see something the image verbalised.

[2010-06-04 15:30:52.411 03152 info 'App'] [VpxLRO] — ERROR task-54417 — host-36589 — vim.host.NetworkSystem.updateNetworkConfig: vim.fault.PlatformConfigFault:
(vim.fault.PlatformConfigFault) {
dynamicType = ,
faultCause = (vmodl.MethodFault) null,
text = “SysinfoException: Node (VSI_NODE_net_tcpip_plumb) ; Status(bad0017)= Out of resources; Message= Instance(0): Input(4) if=0 portset=VMkernel macaddr=00:50:56:76:16:67 tsomss=65535 “,
msg = “Error during the configuration of the host: SysinfoException: Node (VSI_NODE_net_tcpip_plumb) ; Status(bad0017)= Out of resources; Message= Instance(0): Input(4) if=0 portset=VMkernel macaddr=00:50:56:76:16:67 tsomss=65535 “,
}

Key here is the Out of resources; message. The reason for this is none other than the default number of ports for the vSwitch on ESX is 24, and if you have VM’s or other interfaces using up ports (such as AppSpeed probes), you will quickly run out. Switch this easily by going to Configuration -> Networking -> Properties [for the vSwitch in question] and up the value up to and including your growth requirements for the future.

NOTE: After you setup a new host, set it to more ports than you require, as you’ll need to restart the host for the ports to be provisioned; best to do this immediately after installation.

I think a better message would be to explicitly say “Out of available ports on vSwitch – [blah]“; instead of the semi-cryptic one presented.

… go with me here.

You are going to have black caviar (highly recommended). You were provided with 50 grams; you figure you can fit at most 45 grams onto the piece of that delicious dark-rye. So what do you do? Do you disregard and thus throwout the 5 grams? … Well it’s not important what YOU would do, its important what VMware does!

Mr. ESX tends to look at the excruciatingly expensive, sustenance-providing caviar, and throws out the majority that it can’t handle, and opts for the remains in the hard-to-reach crevices of the jar.

What does this all mean for the geek?

There’s a decrepit limit of 2TB minus 512 bytes for each LUN that you can present to ESX. Anything larger, it has no love for. So if you were to present it with a 4TB LUN, you would naïvely assume that you would get the bastardised version of 2TB and the rest would be lost in the ether. I guess that would be somewhat logical.

Lets try it:

Capacity vs. Available Space

There you have it. Instead of actually using up as much as ESX’ly possible (~2TB) from a LUN that has been allocated, VMware chose to only pick up the left-overs (~500GB).

“FAILED: The object or item referred to could not be found” is the extremely helpful message that VMWare converter displays when it fails.

Digging deeper, within the logs we can see that there are multiple instances of

“Warning: failed to create directory” and “Warning: failed to clone directory tree”.

The simple work-around is to convert the vmdk disk to a monolithic-sparse.

You can do this by issuing:

$ vmware-vdiskmanager -r original.vmdk -t 0 destination.vmdk

This will clone the disk image as well as modify it from being composed of 2GB files for the entirety of your VM to a single vmdk referred to as a ‘monolithic-sparse’ (merely referring to the fact that it will increase in size automatically to encompass the the VM partition).

After completing the cloning process, you should have no problems in restarting the conversion process, and it should complete as advertised.