Xen and EC2: make or buy?

the bottom line

The advantages of EC2 in terms of up-front investment are clear-cut: there's no up-front hardware investment involved. Capacity is available on-demand.

Building your own cloud, involves both up-front payment for the hardware plus a longer lead time to bring that capacity on-line. Spread out over a number of years however, investment costs are trumped by the monthly rental costs of rack space. Electrical power turns out to be much more expensive than hardware.

Without going too much into detail, if I run the numbers for a typical entry-level cluster with two powerful host servers in dedicated rackspace, against an equivalent EC2 configuration, the break-even point is around 50% capacity utilization. Running above 50% utilization, as you should, it's definitely cheaper to make (own a rack) than to buy (rent capacity). Put differently, at full utilization of a single cluster, the equivalent EC2 capacity would cost about twice as much.

This is actually a bit surprising. Amazon has advantages of scale and should be able, for example, to procure electrical power at rates far lower than the rates we're confronted with in the Amsterdam area. As they say:

Amazon EC2 passes on to you the financial benefits of Amazon's scale.

That statement just doesn't square with the hard facts: managing your own rack capacity utilization well can give you a substantial cost advantage over hosting with Amazon.

You should do your own math, as the outcome depends highly on your usage scenario. If you're running stable, long-term workloads with a time horizon of several years, do-it-yourself is cheaper. Conversely, if you don't have a long time horizon, or your workloads fluctuate strongly, the EC2 solution may well turn out to be cheaper for you. Also, this analysis just counts "normal" EC2 capacity costs and excludes additional fees for IP addresses and persistent storage volumes.

the root of it all

There's other factors to consider as well. If you're running your own Xen cloud, you have domain-0 access. You or your systems administrators will find that a highly addictive perspective. Much like the root super-user can manage normal users at will, the domain-0 super-server can manage normal Xen servers at will.

Ultimately, that's a security issue. Who controls the hardware controls everything. Having a locked cabinet that requires a physical break-in (triggering chassis intrusion detection), is obviously much more secure than running a virtual server inside a cloud environment where some sysadmin you don't know has super-root privileges and full access to your computing resources.

In summary, choosing make instead of buy as a virtualization strategy still makes a lot of sense if you're running long-term generic workloads.