Amazon did a great job of not calling it ‘amazon cloud services’ when it first launched, this way they stay away from having to define it [cloud]. Once we remember that ‘cloud’ is really an abstraction of abstractions (even from definitions), we soon realise that apps written for tomorrows consumption don’t have to be tied to processor architecture or even a specific OS to run.

Now AWS are going up the stack and developing their platform play atop the IaaS they already have. Most importantly not concentrating purely on a single facet as performance (like the freshly announced Australian startup OrionVM), but useability and management through an extensive set of API’s. The differentiator is a little further up from a mere hypervizor.

Along the same path of execution in the past year we’ve seen VMware reach … up, and look to fill the gaps in Virtual Infrastructure Management+awareness solutions through vSM/Request-Manager for vCD, and more recently vCenter Operations.

The wonderful thing about technology is that by focusing on the value-add, and services further up, you can actually replace the foundation without the need to take down and rebuild the whole offering; as Apple did with Rosetta in their migration away from PowerPC to Intel, shows us you can provide a platform which negates the tight-coupling of CPU > OS > App of the yesteryear. In the current incarnation of cloud deployments across all worthwhile platforms, we can see a single constant – base compute.

. . . just throwing this out there, but it is not only possible, but most probable that within 24 months, not only AWS but other players will launch an Atom or even ARM-based service publicly, which will do away with offering infrastructure services and concentrate from the get-go on platform; all through the same management interface.

Beyond the veil of security, and sense of calm I get when leaving my laptop in the car or airport lobbies; the next most important thing to me is performance (after all, running my VMware lab on a laptop with a single drive isn’t exactly cheap). Thus mid last year I invested in a 240 GB OCZ Vertex 2 SSD drive, to run inside my i7 MBP. Encrypting your drive absolutely kills performance, so I did some very un-scientific tests today upon my upgrade to Lion that I wish to share.

Below are several Xbench results for HDD performance only, covering ‘fresh’ HDD, and encrypted. Within Snow Leopard (10.6) with PGP WDE, and in Lion (10.7) with the inbuilt Disk Encryption.

First up as you can see we have the fresh install, and nicely enough, with just an OS upgrade, there’s an increase in the disk throughput. More importantly though, is the encrypted performance. Averaging out PGP vs Lion, we witness a doubling of the MB/s capability within the latter.

On the one hand, its a shame I have a license for PGP that I spent ~$175AUD, on the other, it served its purpose, and is now going to be replaced by something that comes free with the OS, and performs better.

Symantec has a talent for buying dying horses.

What

At its core vShield Edge is a very lightweight appliance mostly based on LinuxFromScratch and the rest comes from VMware’s BlueLane acquisition.

If you want to skip the verbosity, skip straight to “How”

We are hearing more about vShield recently, because it is a prerequisite component of vCloud Director. It facilitates the critical piece that makes virtual datacenters work; network connectivity. Unfortunately it is still a young product, and as such feature-poor.

In the last two posts I covered two preludes. The first is how to bake your own NAT forwarding router and use it inplace of vShield within the App Network. Second being how to get inside the vSE appliance.

… which is where we’ll start today.

–

Why

The unfortunate repercussion of ‘rolling-your-own’ is that it simply doesn’t scale, and that’s what vCD + Cloud is all about. I simply can’t deploy my own NAT solution with every vApp, and have it integrate with vCloud API’s. So I started at the other end. Customizing vShield Edge itself.

For this piece, I wanted to add only two extra features

1. DNS resolution

Simply put, this will allow for local DNS resolution within each vApp/Organization. More importantly can be made to work with the currently unused DNS Suffix, when creating networks.

2. Dynamic DNS Registration

I have a dream… that one day, when you deploy a resource, it will be able to resolve and ‘see’ others within the same vApp/Organization or vSE ‘Network Boundary’.

How

SUMMARY: Get inside vShield Manager, search for vShieldEdge.ovf. Modify it until you’re satisfied, and you’re done.


-----
/common/em/components/zones/vShield.ovf
/common/em/components/ni/ovf/vshield-ni-infra.ovf
/common/em/components/edge/vShieldEdge.ovf
-----

Note that the other vShield products are also available to you.

Am not going to go through every command required to get this to work. If you’re stuck at any point whilst attempting this, feel free to drop me a line.

First, lets fix the dhcpd.conf generating file – /root/vSEdge/vse_dhcp_config.pl

Search for “ad-hoc” and replace that line with:

$cmd = "include \"/etc/rndc.key\";\nddns-update-style interim;\nddns-updates on;\nddns-domainname \"wcm.lab\";\nddns-rev-domainname \"in-addr.arpa\";\n";

This will do several things for us:

• Setup a trust mechanism between the dhcp and named daemons via the rndc.key
• Set the zone names and allow dhcpd to update them

For bind/named itself – since we’re dealing with LinuxFromScratch, you can get the code from the LFS project page

Compiling named is done via: ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-threads --with-libtool --disable-shared ; make

You can then find the executable in ./bin/named/named

Unfortunately I haven’t been able to figure out how to get it to work with the current libraries present within vShield, so you’ll also have to copy some libs across as well.

On vShield:
$ mkdir -p /var/named /named_libs

On your compiling machine:
$ scp /lib/{ld-2.5.so,ld-linux.so.2,libc.so.6,libnsl.so.1,libpthread.so.0} root@vse:/named_libraries

Once that is all set and done, you’re just left with having to configure named like on any other machine, which means you will require at the very least:

• named.conf
• named.ca
• rndc.conf + key
• zone files

I will be using the ones I’ve prepared earlier for other projects.

… and execute it using:

$ /named_libs/ld-2.5.so --library-path /named_libs/ /usr/sbin/named -g -c /etc/named.conf

I wrapped the above and placed it into /etc/inittab.

After you’re done, you overwrite the vShieldEdge.ovf within the manager VM, and have vSE Manager provision your own, custom vShield Edge appliance.

–

The above is meant for those that like to get their fingers a little dirty trying to improve things, this sometimes will break expected functionality.

Some gotchas, and debugging tips:

• If for some reason vSE isn’t starting, and by looking at the console, you can see that ifconfig doesn’t return any interfaces with an IP, run /root/vSEdge/edge_configure.pl – this will nearly always tell you where you went wrong in your vse_ script modifications
• To find linked libraries for any other packages you wish to inject, use export LD_DEBUG=<type> before running a binary where the <type> is referring to [versions | files | libs]
• Don’t forget to disable/modify iptables, otherwise you’ll have trouble getting inside vSE
• For anything else, there’s my contact page

I wanted to learn more about how vShield Edge + Manager appliance works under the covers. By default, no inbound SSH capability exists. In the video I show you how you can enable it, so that you can extract the innards of the appliance, and hack away to create something beautiful.

Initially started writing this as a static blog-post, unfortunately too many screenshots + and explanations would be required. Hence I simply recorded some of the steps.

Enabling SSH in vShield Edge

Caveats

• There is no possibility that this is in any way supported by our VMware friends, please don’t raise SR’s on a vSE that you’ve Roman’ed up!
• In the last step, I replace the root user password. Be sure to restore it from the backup immediately upon ssh’ing back in. As vCloud Director uses that password to interact with vSE, and will be unable to do so.

Upon trying to create a NAT-Routed network within vCD, you can be faced with a

“Error creating Edge VM”

If you look within the debug logs, the excerpt of the stack-trace will look something like this:

com.vmware.ssdc.util.LMException: Error creating Shield network appliance.
… blah blah
at java.lang.Thread.run(Thread.java:619)
Caused by: com.vmware.ssdc.util.LMException: Error creating Edge VM.
... 17 more
Caused by: java.lang.NullPointerException
… yadda yadda


Click image to enlarge

The cause is a misconfigured ESX Host that is sitting in the same resource pool as other Hosts connected to the dVS against which vCD is trying to create the network group.

Solution being to either remove the host, or alternatively verify that all hosts are connected EQUALLY to the dVS. This is where Host Profiles come in very handy.

I apologise for the lack of a vCD screenshot, I fixed the problem before realising that it was one; at least there will always be logs. Once you do see this again, feel free to send in a screenie.

Related Local Files

logs/vcloud-container-debug.log

In order to demonstrate to some customers, partners and colleagues that haven’t had a chance to enjoy vCloud Director yet, I’ve recently prepared some videos which demonstrate how you setup a network within an Organization on top of vCD.

They’re designed to be completely self-sufficient, so the ‘intro’ is re-used.

Direct Network

This is by far the most … straight forward way of providing your VM’s with an outside connection.

vCloud Director Networking : Part 1 : Direct External

Routed Network

The elegance of vShield Edge allows you to configure DHCP, Firewall, and NAT IP Mapping through an easy to click-through UI. Be it for your Organization or vApp. Here I’ll show you how to setup a NAT-Routed vShield backed internal network.

vCloud Director Networking : Part 2 : NAT-Routed

Custom’ish vApp Network

There are environments that have already invested a lot in customizing their own DHCP server rules, Dynamic DNS registrations, reverse lookups, firewalls… the list goes on. In essence, here I can show you how you can re-use your current soft-router, and have it used within the vCloud realm.

vCloud Director Networking : Part 3 : Custom Routing Appliance in a vApp

Attempted to bring up vCD and saw the following in vcloud-container-debug.log

| ERROR | Start Level Event Dispatcher | StartupUtils | Error starting application: Could not bind network port: 80 on host address: 172.16.225.131 |
java.net.BindException: Cannot assign requested address
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:365)
…

One of these days I’ll allocate some static IP’s to my whole Fusion-bound vLab. Until then – I will put up with DHCP assigned IP’s.

To fix, first stop vCD [ vmware-vcd stop ], the modify cloud-director/etc/global.properties to reflect the IP changes for:

vcloud.cell.ip.primary
consoleproxy.host.https
vcloud.cell.ips

or anywhere you have the old address[es]. Some love from sed will do the trick here.

Now back to it …