Showing posts from December, 2010

Using libcloud to manage instances across multiple cloud providers

More and more organizations are moving to ‘the cloud’ these days. In most cases, using ‘the cloud’ means buying compute and storage capacity from a public cloud vendor such as Amazon, Rackspace, GoGrid, Linode, etc. I believe that the next step in cloud usage will be deploying instances across multiple cloud providers, mainly for high availability, but also for performance reasons (for example if a specific provider has a presence in a geographical region closer to your user base).

All cloud vendors offer APIs for accessing their services -- if they don’t, they’re not a genuine cloud vendor in my book at least. The onus is on you as a system administrator to learn how to use these APIs, which can vary wildly from one provider to another. Enter libcloud, a Python-based package that offers a unified interface to various cloud provider APIs. The list of supported vendors is impressive, and more are added all the time. Libcloud was started by Cloudkick but has since migrated to the Apache …

A Fabric script for striping EBS volumes

Here's a short Fabric script which might be useful to people who need to stripe EBS volumes in Amazon EC2. Striping is recommended if you want to improve the I/O of your EBS-based volumes. However, striping won't help if one of the member EBS volumes goes AWOL or suffers performance issues. In any case, here's the Fabric script:

import commands from fabric.api import * # Globals env.project='EBSSTRIPING' env.user = 'myuser' DEVICES = [ "/dev/sdd", "/dev/sde", "/dev/sdf", "/dev/sdg", ] VOL_SIZE = 400 # GB # Tasks def install(): install_packages() create_raid0() create_lvm() mkfs_mount_lvm() def install_packages(): run('DEBIAN_FRONTEND=noninteractive apt-get -y install mdadm') run('apt-get -y install lvm2') run('modprobe dm-mod') def create_raid0(): cmd = 'mdadm --create /dev/md0 --level=0 --chunk=256 --raid-devices=4 ' for de…

Automated deployments with LittleChef

See my post on the Sysadvent blog.