This article is the first in a seven-part cloud automation series that
will guide you through the build process for a simple web service. This big how-to uses Ubuntu 14.04 LTS, Amazon Web Services (AWS) Command Line Interface (CLI), cloud-init, and Puppet to deliver
an Apache server.

If you’re into DevOps, this tutorial is about the “Ops” bit; building this system is all about infrastructure, OS, and application
configuration. It’s not the “Dev” bit of object-oriented coding, unit tests, and
continuous integration.

The architecture

The architecture is straightforward:

  • one Apache server on one machine, and
  • one Puppet server on another machine.

I didn’t include any High Availability (HA) infrastructure, auto-scaling, or developer tools–I’m not even adding other LAMP components.

The technical stack is simple:

  • AWS provided the foundation I build on. I let AWS
    take care of the hardware, networking, and virtualization layers.
  • The next layer is the operating system. I use a
    free OS from Ubuntu.
  • Applications go on top. Apache and its
    supporting packages (binaries, libraries, and SSL certificate) come from the
    Ubuntu repositories.

The build process

The process is broken down into these steps.

  1. Design the cloud technology stack
  2. Install the new AWS CLI toolkit 
  3. Choose an AWS region 
  4. Add AWS security groups
  5. Work with cloud-init
  6. Create the Puppet master
  7. Create the Puppet agent

One Amazon EC2 machine hosts the Puppet master, and a second machine
hosts Apache. I’m going to use cloud-init to create a Puppet master service.
The Apache build happens automatically–the Puppet master installs it. The
Apache machine is a puppet controlled by the puppet master (how creepy does
that sound?).

Why complicate things
with cloud-init and Puppet?

Why build two machines when one will do? And why add in
configuration management technologies you can do without?

Manually building one machine is easier than getting your
head around cloud-init and a Puppet server. If you have built a few LAMP
servers in your time, you know how streamlined this process has become over the
years. Run through these steps, and you’re done.

  1. Pick the right AMI.
  2. Rent an EC2 machine.
  3. Log in.
  4. Run one command – apt-get install apache.

It’s easy work, but it’s manual work–it’s not making
the most of the cloud. The cloud opens up new possibilities that we did not have
on-premise: the power to automatically build customer services,
automatically configure them, and
scale them to fit demand. These configuration management tools are part of
making that happen.

The benefit of doing the complicated work now is that it reduces how much pain you might feel in the future. All customer services change over time. The more
automation you can add, the more time you can spend sweating the big stuff and
the quicker you can adapt to change.

Also, many enterprises want to move test, dev, and batch
services to AWS. It doesn’t hurt to be the guy with the skills to make that

It might sound
simple, but it requires a lot of work

This way of working isn’t a time saver. I still have to
manually build the Puppet master using esoteric cloud technology so, in fact,
it’s harder work. The advantage comes later, when my Puppet master controls
many puppets, providing many customer services.

I make this system using AWS tools and a bunch of building
blocks: EC2 machines, security groups, a keypair, cloud-config, and a Puppet
manifest. You’re going to see a fair number of labels with the prefix “p-” (an
abbreviation of “Puppet-related kinda thing”) on all these new things.

More AWS how-tos

AWS auto-scaling

  1. Getting to know Amazon’s auto-scaling command line tools
  2. Auto-scaling an EC2 service: Creating a new AMI
  3. Installing and checking Amazon Auto Scaling and CloudWatch tools
  4. Add an auto-scaling group and policy for Amazon EC2 machines
  5. How to add Cloudwatch monitors to auto-scale your Amazon Web Service
  6. AWS auto-scaling: Add notification and test to see what happens

AWS CloudFormation

  1. How to be a SaaS vendor on a shoestring with Amazon Web Services
  2. First steps to SaaS with AWS CloudFormation templates
  3. Use AWS CloudFormation to create a highly available cluster
  4. Anatomy of an AWS CloudFormation template
  5. Modifying AWS CloudFormation templates: Polish your code