According to IDC, the mere growth of virtual private cloud services by 2014, will be in excess of 230% from a base of 2010, or be worth $46USD billion as an industry. This referring to the homogeneous, logical environment that will span public + private and be isolated for an organisation. Of the current public cloud providers supporting this, how many can actually take your existing environment and extend it into their own to allow you to use their capacity as needed?

Source: IDC 2010 SMO Analysis | repurposed

It is important to note that within a discussion on cloud, several attributes are always considered: trust, availability, flexibility, inevitably – cost; but most importantly the vital tenant, scale.

Arguably most of these are actually better catered for within the cloud-sphere. You have complete stateful, and migratory capability of a security policy which can reflect business rules and be as dynamic as required. Technologies like VMware’s HA, and FT can provide guarantees around operational state of the OS, integration with Intel’s TXT can further provide assurances around GRC and around execution on foreign tin; thus introducing capability the original OS + App need-not design and maintain. Lastly, through significant consolidation and utilization adjustment, an organisation can drive significant cost savings to a pure hardware-OS-app stack. How do we then grow this environment?

The problem of scale within an enterprise is normally solved vertically with mainframes, and horizontally with oft-too-expensive clustered approach. What happens to all your old apps that are mission critical? How do you ‘cloudify’ them? Typical response is re-architect. Reality is that the lifecycle of a typical enterprise-grade application is around 15-20 years (or considerably more if you look at Sabre). Thus any suggested changes to these monoliths will undoubtedly not be greeted with open arms by the application and business owners.
Therefore re-architecture is simply not an option. We have designed an elegant abstraction for each of these silos, called Virtual Machines. Which transform the old and senile into something more malleable and agile, or so you would think.

At the end of the day, we’ve just injected some functionality into the sarcophagus of an ever-decaying technological silo. Now we have to figure out a way to move it around. How do we optimize this further? How do we migrate these workloads and expand the stacks’ capability? Surely there are service providers out there that can help?!

There do exist “cloud-providers” that can cater for, and ingest your applications to be run within their own DC’s. Whether you’re moving to or from their premises, they will support you; in a very manual fashion. Herein lies the issue, under the guise of ‘cloud’; they’re no more than a hosting service provider with a brand new brochure.

In order for a provider to be truly cloud-enabled, and allow for Enterprises to scale outside their physical boundary. They must support the same abstraction-architecture as you use internally, and federate your application and data in a seamless and live manner, thus delivering on ‘true-cloud’ promise, and not the same way that you have always attacked this by replication without state.

Technically, the problem of migrating live workloads across distances has been solved. We encapsulate the OS+App within a VM, all networks and their configuration for a group of VM’s that is further packed within a vApp; is provided through vShield. Inter-DC communication can be achieved with Cisco’s OVT or stretched L2 networks (ugh), thus giving us the mobility of the state. Lastly, the moving of the storage can be facilitated through with VPLEX by further abstracting the underlying storage arrays to achieve the fully virtualized, completely federated, live workload migration.

The unfortunate state of affairs is that whilst we continue to have the baggage of old-school applications, for a solution to be Enterprise-ready, it must provide for ‘warts and all’ encapsulation and transmission. Until that is the case, and organisations as well as the service providers standardise on a mechanism and technological cocktail, we can’t really be calling some clouds, clouds.

Amazon did a great job of not calling it ‘amazon cloud services’ when it first launched, this way they stay away from having to define it [cloud]. Once we remember that ‘cloud’ is really an abstraction of abstractions (even from definitions), we soon realise that apps written for tomorrows consumption don’t have to be tied to processor architecture or even a specific OS to run.

Now AWS are going up the stack and developing their platform play atop the IaaS they already have. Most importantly not concentrating purely on a single facet as performance (like the freshly announced Australian startup OrionVM), but useability and management through an extensive set of API’s. The differentiator is a little further up from a mere hypervizor.

Along the same path of execution in the past year we’ve seen VMware reach … up, and look to fill the gaps in Virtual Infrastructure Management+awareness solutions through vSM/Request-Manager for vCD, and more recently vCenter Operations.

The wonderful thing about technology is that by focusing on the value-add, and services further up, you can actually replace the foundation without the need to take down and rebuild the whole offering; as Apple did with Rosetta in their migration away from PowerPC to Intel, shows us you can provide a platform which negates the tight-coupling of CPU > OS > App of the yesteryear. In the current incarnation of cloud deployments across all worthwhile platforms, we can see a single constant – base compute.

. . . just throwing this out there, but it is not only possible, but most probable that within 24 months, not only AWS but other players will launch an Atom or even ARM-based service publicly, which will do away with offering infrastructure services and concentrate from the get-go on platform; all through the same management interface.

Beyond the veil of security, and sense of calm I get when leaving my laptop in the car or airport lobbies; the next most important thing to me is performance (after all, running my VMware lab on a laptop with a single drive isn’t exactly cheap). Thus mid last year I invested in a 240 GB OCZ Vertex 2 SSD drive, to run inside my i7 MBP. Encrypting your drive absolutely kills performance, so I did some very un-scientific tests today upon my upgrade to Lion that I wish to share.

Below are several Xbench results for HDD performance only, covering ‘fresh’ HDD, and encrypted. Within Snow Leopard (10.6) with PGP WDE, and in Lion (10.7) with the inbuilt Disk Encryption.

First up as you can see we have the fresh install, and nicely enough, with just an OS upgrade, there’s an increase in the disk throughput. More importantly though, is the encrypted performance. Averaging out PGP vs Lion, we witness a doubling of the MB/s capability within the latter.

On the one hand, its a shame I have a license for PGP that I spent ~$175AUD, on the other, it served its purpose, and is now going to be replaced by something that comes free with the OS, and performs better.

Symantec has a talent for buying dying horses.

What

At its core vShield Edge is a very lightweight appliance mostly based on LinuxFromScratch and the rest comes from VMware’s BlueLane acquisition.

If you want to skip the verbosity, skip straight to “How”

We are hearing more about vShield recently, because it is a prerequisite component of vCloud Director. It facilitates the critical piece that makes virtual datacenters work; network connectivity. Unfortunately it is still a young product, and as such feature-poor.

In the last two posts I covered two preludes. The first is how to bake your own NAT forwarding router and use it inplace of vShield within the App Network. Second being how to get inside the vSE appliance.

… which is where we’ll start today.

–

Why

The unfortunate repercussion of ‘rolling-your-own’ is that it simply doesn’t scale, and that’s what vCD + Cloud is all about. I simply can’t deploy my own NAT solution with every vApp, and have it integrate with vCloud API’s. So I started at the other end. Customizing vShield Edge itself.

For this piece, I wanted to add only two extra features

1. DNS resolution

Simply put, this will allow for local DNS resolution within each vApp/Organization. More importantly can be made to work with the currently unused DNS Suffix, when creating networks.

2. Dynamic DNS Registration

I have a dream… that one day, when you deploy a resource, it will be able to resolve and ‘see’ others within the same vApp/Organization or vSE ‘Network Boundary’.

How

SUMMARY: Get inside vShield Manager, search for vShieldEdge.ovf. Modify it until you’re satisfied, and you’re done.


-----
/common/em/components/zones/vShield.ovf
/common/em/components/ni/ovf/vshield-ni-infra.ovf
/common/em/components/edge/vShieldEdge.ovf
-----

Note that the other vShield products are also available to you.

Am not going to go through every command required to get this to work. If you’re stuck at any point whilst attempting this, feel free to drop me a line.

First, lets fix the dhcpd.conf generating file – /root/vSEdge/vse_dhcp_config.pl

Search for “ad-hoc” and replace that line with:

$cmd = "include \"/etc/rndc.key\";\nddns-update-style interim;\nddns-updates on;\nddns-domainname \"wcm.lab\";\nddns-rev-domainname \"in-addr.arpa\";\n";

This will do several things for us:

• Setup a trust mechanism between the dhcp and named daemons via the rndc.key
• Set the zone names and allow dhcpd to update them

For bind/named itself – since we’re dealing with LinuxFromScratch, you can get the code from the LFS project page

Compiling named is done via: ./configure --prefix=/usr --sysconfdir=/etc --localstatedir=/var --enable-threads --with-libtool --disable-shared ; make

You can then find the executable in ./bin/named/named

Unfortunately I haven’t been able to figure out how to get it to work with the current libraries present within vShield, so you’ll also have to copy some libs across as well.

On vShield:
$ mkdir -p /var/named /named_libs

On your compiling machine:
$ scp /lib/{ld-2.5.so,ld-linux.so.2,libc.so.6,libnsl.so.1,libpthread.so.0} root@vse:/named_libraries

Once that is all set and done, you’re just left with having to configure named like on any other machine, which means you will require at the very least:

• named.conf
• named.ca
• rndc.conf + key
• zone files

I will be using the ones I’ve prepared earlier for other projects.

… and execute it using:

$ /named_libs/ld-2.5.so --library-path /named_libs/ /usr/sbin/named -g -c /etc/named.conf

I wrapped the above and placed it into /etc/inittab.

After you’re done, you overwrite the vShieldEdge.ovf within the manager VM, and have vSE Manager provision your own, custom vShield Edge appliance.

–

The above is meant for those that like to get their fingers a little dirty trying to improve things, this sometimes will break expected functionality.

Some gotchas, and debugging tips:

• If for some reason vSE isn’t starting, and by looking at the console, you can see that ifconfig doesn’t return any interfaces with an IP, run /root/vSEdge/edge_configure.pl – this will nearly always tell you where you went wrong in your vse_ script modifications
• To find linked libraries for any other packages you wish to inject, use export LD_DEBUG=<type> before running a binary where the <type> is referring to [versions | files | libs]
• Don’t forget to disable/modify iptables, otherwise you’ll have trouble getting inside vSE
• For anything else, there’s my contact page

I wanted to learn more about how vShield Edge + Manager appliance works under the covers. By default, no inbound SSH capability exists. In the video I show you how you can enable it, so that you can extract the innards of the appliance, and hack away to create something beautiful.

Initially started writing this as a static blog-post, unfortunately too many screenshots + and explanations would be required. Hence I simply recorded some of the steps.

Enabling SSH in vShield Edge

Caveats

• There is no possibility that this is in any way supported by our VMware friends, please don’t raise SR’s on a vSE that you’ve Roman’ed up!
• In the last step, I replace the root user password. Be sure to restore it from the backup immediately upon ssh’ing back in. As vCloud Director uses that password to interact with vSE, and will be unable to do so.

Upon trying to create a NAT-Routed network within vCD, you can be faced with a

“Error creating Edge VM”

If you look within the debug logs, the excerpt of the stack-trace will look something like this:

com.vmware.ssdc.util.LMException: Error creating Shield network appliance.
… blah blah
at java.lang.Thread.run(Thread.java:619)
Caused by: com.vmware.ssdc.util.LMException: Error creating Edge VM.
... 17 more
Caused by: java.lang.NullPointerException
… yadda yadda


Click image to enlarge

The cause is a misconfigured ESX Host that is sitting in the same resource pool as other Hosts connected to the dVS against which vCD is trying to create the network group.

Solution being to either remove the host, or alternatively verify that all hosts are connected EQUALLY to the dVS. This is where Host Profiles come in very handy.

I apologise for the lack of a vCD screenshot, I fixed the problem before realising that it was one; at least there will always be logs. Once you do see this again, feel free to send in a screenie.

Related Local Files

logs/vcloud-container-debug.log

In order to demonstrate to some customers, partners and colleagues that haven’t had a chance to enjoy vCloud Director yet, I’ve recently prepared some videos which demonstrate how you setup a network within an Organization on top of vCD.

They’re designed to be completely self-sufficient, so the ‘intro’ is re-used.

Direct Network

This is by far the most … straight forward way of providing your VM’s with an outside connection.

vCloud Director Networking : Part 1 : Direct External

Routed Network

The elegance of vShield Edge allows you to configure DHCP, Firewall, and NAT IP Mapping through an easy to click-through UI. Be it for your Organization or vApp. Here I’ll show you how to setup a NAT-Routed vShield backed internal network.

vCloud Director Networking : Part 2 : NAT-Routed

Custom’ish vApp Network

There are environments that have already invested a lot in customizing their own DHCP server rules, Dynamic DNS registrations, reverse lookups, firewalls… the list goes on. In essence, here I can show you how you can re-use your current soft-router, and have it used within the vCloud realm.

vCloud Director Networking : Part 3 : Custom Routing Appliance in a vApp