In part one of my cloud automation series on building a simple web service, I focused on details about the technology, the architecture, and the steps involved in the process. Part two covered automating the Puppet Labs automation system.
Now I'm building virtual machines on the Amazon Elastic Compute Cloud (EC2) platform using the new Amazon Web Services (AWS) Command Line Interface (CLI) tools. Commands that used to look like
now look like
aws ec2 describe-instances
OK, that doesn't look like it's worth the effort. Why bother installing the new tools when the old toolkit still works? The new tools are easier to install, cover more AWS services, and are more consistent.
The old toolkit
The old AWS EC2 API toolkit is still maintained and is widely deployed. The install procedure for this old-timer is complicated because it requires Java, it's split into several parts, and it requires a set of environment variables. Here's a summary of the old install procedure.
1. Sign up for AWS.
2. Download the archive of EC2 API tools, ec2-api-tools.zip.
4. Unpack the verified archive.
The old AWS EC2 API toolkit requires a bunch of environment variables, like these.
# environment variables for AWS on my OS X machine export EC2_PRIVATE_KEY=~/.ec2/my-ec2-private-key.pem export EC2_KEYPAIR_EU_WEST_1=$EC2_PRIVATE_KEY export EC2_CERT=~/.ec2/my-cert.pem export EC2_REGION=eu-west-1 export EC2_URL=https://ec2.eu-west-1.amazonaws.com/ export EC2_HOME=~/AWS/tools/ec2-api-tools-220.127.116.11 export JAVA_HOME=`/usr/libexec/java_home` export PATH=$PATH:$EC2_HOME/bin
The problem is not whether the old toolkit works—it's just a bit fiddly. The install is fiddly, the configuration is fiddly, and the commands aren't as clear as they could be.
The new AWS CLI toolkit
In September 2013 Amazon released a new AWS CLI toolkit v1.0. It's easier to install and use than the old suites of tools. The new toolkit covers 28 AWS services—it's vast. The EC2 section alone contains 148 EC2 commands—a complete overhaul of the old AWS EC2 API toolkit.
The install procedure for the new CLI toolkit is much simpler. Amazon provides an MSI installation archive for Windows, another archive for OS X, Linux, and UNIX, and even a cross-platform pip archive for Python users.
Install AWS CLI
I carried out this install procedure on my OS X machine to install the latest version (1.2.13).
1. Sign up for AWS.
2. Open a terminal.
3. Download the archive.
nick:~ $ curl -O https://s3.amazonaws.com/aws-cli/awscli-bundle.zip % Total % Received % Xferd Average Speed Time Time Time Current 1. Dload Upload Total Spent Left Speed 31 5130k 31 1616k 0 0 76697 0 0:01:08 0:00:21 0:00:47 119k …
4. Unpack the archive.
nick:~ $ unzip awscli-bundle.zip Archive: awscli-bundle.zip inflating: awscli-bundle/install … inflating: awscli-bundle/packages/virtualenv1.10.1.tar.gz nick:~ $
5. Run the install script.
nick:~ $ sudo ./awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws Password: Running cmd: /usr/bin/python virtualenv.py —python /usr/bin/python /usr/local/aws Running cmd: /usr/local/aws/bin/pip install —no-index —find-links file:///Users/nick/awscli-bundle/packages awscli-1.2.13.tar.gz You can now run: /usr/local/bin/aws —version nick:~ $
6. Check your work.
nick:puppet $ aws —version aws-cli/1.2.13 Python/2.7.5 Darwin/13.0.0 nick:puppet $
Configure AWS CLI
The new AWS CLI toolkit sticks some basic information in a configuration file called config. You can't really come up with a clearer naming policy than that.
The new CLI tools require access keys. Access keys are only supplied by the AWS Identity and Access Management (IAM) service. If you have not set up IAM, you can't use the tools. These are the prerequisite steps.
1. Use the AWS management console.
2. Set up an AWS IAM group and user.
3. Copy your new credentials to a safe location.
AWS provides a script to create the configuration file. You can use a text editor instead if you like to cause yourself pain.
4. Run the configuration file creator.
nick:~ $ aws configure AWS Access Key ID [None]: ABCAIXMQMAXVHGTX7RDQ AWS Secret Access Key [None]: a123ygXa6IUyXuoYsWC3ocqxc7KrOmFiOlIr4BtV Default region name [None]: eu-west-1 Default output format [None]: text nick:~ $
5. Check your work.
nick:~ $ cat ~/.aws/config [default] region = eu-west-1 output = text aws_access_key_id = ABCAIXMQMAXVHGTX7RDQ aws_secret_access_key = a123ygXa6IUyXuoYsWC3ocqxc7KrOmFiOlIr4BtV nick:~ $
Was that worth the effort?
The old tools are familiar and still get the job done. If you are happy with the old tools, there's no need to switch just yet. But, for the old tools, the end is coming.
Upcoming installments in this cloud automation series
- Choose an AWS region
- Add AWS security groups
- Work with cloud-init
- Create the Puppet master
- Create the Puppet agent
Nick Hardiman builds and maintains the infrastructure required to run Internet services. Nick deals with the lower layers of the Internet - the machines, networks, operating systems, and applications. Nick's job stops there, and he hands over to the designers and developers who build the top layer that customers use.