Software Development

How to use the command line with Amazon's EC2 API tools

Nick Hardiman runs through the steps of installing Java, creating security credentials, and setting up some variables so that you can play with the EC2 API tools from the command line on your local machine.

I downloaded the archive of EC2 API tools, checked them (I installed GPG and performed security checks) and unpacked them.

Now comes the fun bit. I can run them and be amazed at the things that come back from Amazon. I've done plenty of work in the past on my remote EC2 machines using the CLI (Command Line Interface), but I haven't yet used the CLI on my local machine.

I perform these steps on my local machine. My computer here is a Macbook, so some of these steps are specific to OSX (they are similar to - but not quite the same as - the commands for Windows).

Here's the quick summary.

  1. Open a terminal on my personal computer.
  2. Install Java.
  3. Create security credentials (an X.509 certificate and private key).
  4. Set up a few environment variables.
  5. Run my first command.

Open a terminal on my personal computer

On a Mac this is located in Applications | Utilities | Terminal.

If the terminal looks old fashioned in today's graphical world, that's because it is. The terminal's roots stretch back through the BSD OS and across almost the entire history of computing to TTYs (Tele-TYpewriters). It's a bit like realizing sharks have been around for 400 million years.

Install Java

Java on OSX is an optional package: it is not installed by default. The EC2 API Tools won't work without Java.

If I see an error along the lines of JAVA_HOME is not set when running one of the EC2 API tools, it may mean Java is not yet installed.

My-MacBook-Pro:~ nick$ ec2-describe-regions
/Users/nick/Documents/AWS/ec2-api-tools-1.5.3.1/bin/ec2-cmd: line 18: JAVA_HOME: JAVA_HOME is not set
My-MacBook-Pro:~ nick$

  1. Run a simple Java command. /usr/bin/java -version The Software Update application starts, displaying a pop-up window with the message "To open Java, you need to install a Java runtime."
  2. Install. Software Update finds the package in the Internet, downloads, and installs it.

Create security credentials (an X.509 certificate and private key)

I log in using my security credentials. Credentials are things that give me authority to act, such as setting my preferences in an online store. Everyone, including every cave-dwelling hermit and every technophobe allergic to keyboards, has logged into websites using an account name and password as their credentials.

So far I have been using AWS services by logging into the AWS Management Console using my e-mail address and password, and logging into my EC2 machines using my public and private key.

Now I want to use the EC2 API Tools. I have to use a third set of credentials - a certificate and private key. This is how I create them.

  1. Open a web browser.
  2. Go to the AWS Web Site.
  3. Log in using your old security credentials (e-mail address and password).
  4. Navigate to the Security Credentials page. https://aws-portal.amazon.com/gp/aws/securityCredentials
  5. Find the Access Credentials section.
  6. Click the X.509 Certificates tab.
  7. Click the Create a new Certificate link.
  8. Download the private key and X.509 certificate files from Amazon's web site to your local machine. The files have names along the lines of:

    pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem

    cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem

    Move these files to a folder in your home directory called .ec2.
  9. Close the web browser.

Set up a few environment variables

The JAVA_HOME environment variable has to be set for Java to work. For some reason, installing Java on OSX doesn't set this variable. Running an EC2 API tool displays an error along the lines of JAVA_HOME is not set.

  1. View the error. My-MacBook-Pro:~ nick$ ec2-describe-regions /Users/nick/Documents/AWS/ec2-api-tools-1.5.3.1/bin/ec2-cmd: line 18: JAVA_HOME: JAVA_HOME is not set My-MacBook-Pro:~ nick$
  2. Set the value. export JAVA_HOME=`/usr/libexec/java_home`
  3. Check your work.

    My-MacBook-Pro:~ nick$ echo $JAVA_HOME

    /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

    My-MacBook-Pro:~ nick$

Set the locations of the EC2 files, the private key file, and X.509 certificate file.

export EC2_HOME=~/Documents/AWS/ec2-api-tools-1.5.3.1/

export PATH=$PATH:$EC2_HOME/bin

export EC2_PRIVATE_KEY=~/.ec2/pk-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem

export EC2_CERT=~/.ec2/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem

Run my first command

Run a command that is read-only. A simple one is ec2-describe-regions that lists AWS regions. If all goes well this command displays half a dozen entries.

My-MacBook-Pro:~ nick$ ec2-describe-regions
REGION  ap-northeast-1       ec2.ap-northeast-1.amazonaws.com
REGION sa-east-1    ec2.sa-east-1.amazonaws.com
REGION us-east-1    ec2.us-east-1.amazonaws.com
REGION ap-northeast-1      ec2.ap-northeast-1.amazonaws.com
REGION us-west-2    ec2.us-west-2.amazonaws.com
REGION us-west-1    ec2.us-west-1.amazonaws.com
REGION ap-southeast-1      ec2.ap-southeast-1.amazonaws.com
My-MacBook-Pro:~ nick$ 

I can play around with this command safely. I can remove the certificate location, just to check this really is being used for my credentials

My-MacBook-Pro:~ nick$ unset EC2_CERT
My-MacBook-Pro:~ nick$ 
My-MacBook-Pro:~ nick$ ec2-describe-regions
Required option '-C, --cert CERT' missing (-h for usage)
My-MacBook-Pro:~ nick$ 

I can also set the certificate to something invalid, like an ordinary file.

My-MacBook-Pro:~ nick$ export EC2_CERT=/etc/hosts
My-MacBook-Pro:~ nick$ ec2-describe-regions
Unexpected error:
org.codehaus.xfire.fault.XFireFault: Fault: java.lang.NullPointerException
       at org.codehaus.xfire.fault.XFireFault.createFault(XFireFault.java:89)
       at org.codehaus.xfire.client.Invocation.invoke(Invocation.java:83)
..
       at org.codehaus.xfire.client.Invocation.invoke(Invocation.java:79)
       ... 10 more
My-MacBook-Pro:~ nick$ 

There are dozens of lines of this scary-looking text. It is a stack of error messages a Java programmer can use to find an exception.

Repair the damage.

My-MacBook-Pro:~ nick$ export EC2_CERT=~/.ec2/cert-HKZYKTAIG2ECMXYIBH3HXV4ZBEXAMPLE.pem
My-MacBook-Pro:~ nick$ 

About

Nick Hardiman builds and maintains the infrastructure required to run Internet services. Nick deals with the lower layers of the Internet - the machines, networks, operating systems, and applications. Nick's job stops there, and he hands over to the ...

0 comments

Editor's Picks