In part one of my cloud automation series on building a simple web service, I focused on details about the technology, the architecture, and the steps involved in the process. Part two covered automating the Puppet Labs automation system.

Now I’m building virtual machines on the Amazon Elastic Compute Cloud (EC2) platform using the new Amazon Web Services (AWS) Command Line Interface (CLI) tools. Commands that used to look like

ec2-describe-instances

now look like

aws ec2 describe-instances

OK, that doesn’t look like it’s worth the effort. Why bother
installing the new tools when the old toolkit still works? The new tools are easier to install, cover more
AWS services, and are more consistent.

The old toolkit

The old AWS EC2 API toolkit is still maintained and is widely deployed. The install
procedure for this old-timer is complicated because it requires Java, it’s
split into several parts, and it requires a set of environment variables. Here’s a
summary of the old install procedure.

1. Sign up for AWS.

2. Download the archive of EC2 API tools, ec2-api-tools.zip.

3. Perform security checks on AWS files using keys and signatures (ec2-api-tools.zip.asc is the
signature of ec2-api-tools.zip).

4. Unpack the verified archive.

5. Set up Java and the rest of your workstation environment.

The old AWS EC2 API toolkit requires a bunch of environment
variables, like these.

# environment variables for AWS on my OS X machine
export EC2_PRIVATE_KEY=~/.ec2/my-ec2-private-key.pem
export EC2_KEYPAIR_EU_WEST_1=$EC2_PRIVATE_KEY
export EC2_CERT=~/.ec2/my-cert.pem
export EC2_REGION=eu-west-1
export EC2_URL=https://ec2.eu-west-1.amazonaws.com/
export EC2_HOME=~/AWS/tools/ec2-api-tools-1.6.12.2
export JAVA_HOME=`/usr/libexec/java_home`
export PATH=$PATH:$EC2_HOME/bin

The problem is not whether the old toolkit works–it’s just
a bit fiddly. The install is fiddly, the configuration is fiddly, and the commands
aren’t as clear as they could be.

The new AWS CLI toolkit

In September 2013 Amazon released a new AWS CLI toolkit v1.0. It’s easier to
install and use than the old suites of tools. The new toolkit covers 28 AWS services–it’s vast. The EC2 section alone contains 148 EC2 commands–a complete overhaul of the old AWS EC2 API toolkit.

The install procedure for the new CLI toolkit is much
simpler. Amazon provides an MSI installation archive for Windows, another archive
for OS X, Linux, and UNIX, and even a cross-platform pip
archive
for Python users.

Install AWS CLI

I carried out this install procedure on my OS X machine to
install the latest version (1.2.13).

1. Sign up for AWS.

2. Open a terminal.

3. Download the archive.

ick:~ $ curl -O https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
% Total % Received % Xferd Average Speed Time Time Time Current
1. Dload Upload Total Spent Left Speed
31 5130k 31 1616k 0 0 76697 0 0:01:08 0:00:21 0:00:47 119k

4. Unpack the archive.

nick:~ $ unzip awscli-bundle.zip
Archive: awscli-bundle.zip
inflating: awscli-bundle/install

inflating: awscli-bundle/packages/virtualenv1.10.1.tar.gz
nick:~ $

5. Run the install script.

nick:~ $ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws
Password:
Running cmd: /usr/bin/python virtualenv.py –python /usr/bin/python /usr/local/aws
Running cmd: /usr/local/aws/bin/pip install –no-index –find-links file:///Users/nick/awscli-bundle/packages awscli-1.2.13.tar.gz
You can now run: /usr/local/bin/aws –version
nick:~ $

6. Check your work.

nick:puppet $ aws –version
aws-cli/1.2.13 Python/2.7.5 Darwin/13.0.0
nick:puppet $

Configure AWS CLI

The new AWS CLI toolkit sticks some basic information in a
configuration file called config. You
can’t really come up with a clearer naming policy than that.

The new CLI tools require access keys.
Access keys are only supplied by the AWS Identity and Access Management (IAM) service. If you have not set up IAM, you
can’t use the tools. These are the prerequisite steps.

1. Use the AWS management console.

2. Set up an AWS IAM group and user.

3. Copy your new credentials to a safe location.

AWS provides a script to create the configuration file. You
can use a text editor instead if you like to cause yourself pain.

4. Run the configuration file creator.

nick:~ $ aws configure
AWS Access Key ID [None]: ABCAIXMQMAXVHGTX7RDQ
AWS Secret Access Key [None]: a123ygXa6IUyXuoYsWC3ocqxc7KrOmFiOlIr4BtV
Default region name [None]: eu-west-1
Default output format [None]: text
nick:~ $

5. Check your work.

nick:~ $ cat ~/.aws/config
[default]
region = eu-west-1
output = text
aws_access_key_id = ABCAIXMQMAXVHGTX7RDQ
aws_secret_access_key = a123ygXa6IUyXuoYsWC3ocqxc7KrOmFiOlIr4BtV
nick:~ $

Was that worth the effort?

The old tools are familiar and still get the job
done. If you are happy with the old
tools, there’s no need to switch just yet. But, for the old tools, the end is
coming.

Upcoming installments in this cloud automation series

  • Choose an AWS region
  • Add AWS security groups
  • Work with cloud-init
  • Create the Puppet master
  • Create the Puppet agent