Cloud optimize

Modifying AWS CloudFormation templates: Polish your code

Nick Hardiman shows a step-by-step example of how to modify an AWS CloudFormation template -- the last part of building your own SaaS on Amazon's platform.

I have been demonstrating how to take a SaaS solution from business plan to a live application, using the example of a shiny, new trouble ticket app. Here is what we've covered so far:

Picking up from last week's anatomy lesson, where I detailed the various parts of an AWS CloudFormation template, I am now ready to modify it to suit my specific requirements -- automating the build of my support ticket service.

Click to enlarge.

The SaaS build work has now reached the heavily technical stage. I've written the head-meltingly complicated procedure below (thanks to the Drupal development agency Microserve for checking my code).

Experimenting with Amazon CloudFormation costs real money. Only follow these instructions if you are happy to waste a little cash. I assume you have already signed up for AWS.

I don't describe all the SaaS build work required -- there is no need to drag innocent people through the entire development process. These steps stick together a few building blocks to get to a basic working state.

Get a copy of Amazon's CloudFusion template file

I chop and change Amazon's template to get quick results, rather than doing it properly by writing a template from scratch. If you've ever wondered what a "dirty hack" is, you're about to make one.

  1. Open the URL The page AWS CloudFormation Sample Templates - US East (Virginia) region appears.
  2. Find the link called Highly Available Web Server with Multi-AZ Amazon RDS database instance and using S3 for storing file content. Amazon is not using short and snappy names here.
  3. Download the file Drupal_Multi_AZ.template from Amazon's web site to your workstation.
  4. Pick a name for your copy. I called mine aws-cfn-support-template.json. No, my name's not short and snappy either.
  5. If you use a code repository like Github or Gitorious, check in your new version.

You now have your local copy to edit and a remote backup (the original file on AWS).

Change the AWS CloudFormation Templates "Resources" section.

This is where we start hitting the code with a mallet. Check my last post for what a "Resources" section means - that describes the layout of a template file.

These edits change the configuration and sneak in a few Drupal developer tricks -  drush commands, modules and an install profile.

Is it all too much code to copy? Find the full template on github.

Change the ElasticLoadBalancer's configuration

The elastic load balancers are certainly not flexible when it comes to right and wrong. They must receive a simple OK response (an HTTP 200). Drupal may give a range of responses, not just OK.

Check an ordinary file like README.txt instead of asking Drupal.

  1. Edit your copy of the template file. Find these lines:

    "HealthCheck" : {

    "Target" : "HTTP:80/",

    They are about halfway down the file.
  2. Change that Target line:

    "HealthCheck" : {

    "Target" : "HTTP:80/README.txt",

Update Drupal code and add modules

The Resources section contains an embedded bash script. This is described in my last post too. These edits change that script.

Add some extra modules for the support ticket system. I want these files copied to every server. Module files end up in /var/www/html/sites/all/modules. Modules don't go in the S3 bucket (the S3 area is /var/www/html/sites/default/files).

This section contains commands used by Linux administrators, Drupal administrators, and AWS administrators. Do you know what the rm command is? Have you used drush? How about cfn-signal?

1. Find the end of the bash script. It looks like this:

"rm /home/ec2-user/settings.php\n",
"# All is well so signal success\n",
"/opt/aws/bin/cfn-signal -e 0 -r \"Drupal setup complete\" '", { "Ref" : "WaitHandle" }, "'\n"

You are going to add lines of code where that empty line is, below the rm command and above the # comment.

2. Add these lines to update the application files. Cloudformation installs old versions of drupal and drush when the EC2 machines are built.  Every new EC2 virtual machine will get these updated files.

"# Update the code on all machines. \n",
"cd ~ec2-user \n",
"~ec2-user/drush/drush self-update --choice=2 --yes \n",
"cd /var/www/html \n",
"~ec2-user/drush/drush pm-updatecode --yes \n",

3. Update the database to match the files.  I only want one machine to do this so the code checks for the first machine only. If all machines try at the same time, the database will probably end up broken.

"# Update the database from the first machine only. \n",
"if [ `hostname` = $first ]\n",
"    ~ec2-user/drush/drush updatedb --yes \n",

4. Add the module files I need for my work.

"# Add modules. Many more are automatically added to this list. \n",
"# Files are copied to all machines. \n",
"~ec2-user/drush/drush pm-download acl commerce content_access entity rules support --yes \n",

5. Enable the modules by updating the database.

"# Enable modules and rebuild the permissions table. The database holds this. \n",
"if [ `hostname` = $first ]\n",
"    ~ec2-user/drush/drush pm-enable  commerce content_access support --yes \n",
"    ~ec2-user/drush/drush php-eval 'node_access_rebuild();' \n",

6. Make sure Drupal can read its own files.

"# Fix ownership and permissions. \n",
"chown -R root:apache /var/www/html \n",
"chmod 640 /var/www/html/sites/default/settings.php \n",

7. Save your work.

Form your new cloud

Re-run the steps for running up the AWS example of a Drupal cluster. My earlier posts covered the procedure to follow and the values you need.

Build your new site.

    1. Find your new template.
    2. Follow the procedure for launching a cloud, but don't use Amazon's default template.
    3. Upload your new template using the Upload a Template File option.
    4. Fill in the forms.
    5. Create the CloudFormation stack. When I ran this it took 20 minutes to complete.
    6. Refresh the Events list to see progress.

      Click to enlarge.
      Check your new site.

        1. Open the URL of your new support site. Mine is A login page appears.
        2. Log in using your new SiteAdmin and SitePassword. A support page appears.

          Clean up.

            Destroy your new CloudFormation stack when you are done.

              A comprehension test

              Crikey, it's like being back at school.

              A new Amazon VM is always a little out of date. Patches and upgrades are available for a few of its packages. One thing this template doesn't do is update the packages on each virtual machine.

              Where would you put a yum update command?


              Nick Hardiman builds and maintains the infrastructure required to run Internet services. Nick deals with the lower layers of the Internet - the machines, networks, operating systems, and applications. Nick's job stops there, and he hands over to the ...