In part one of my cloud automation series on building a simple web service, I focused on details about the technology, the architecture, and the steps involved in the process. Part two covered automating the Puppet Labs automation system. Part three built virtual machines on the Amazon Elastic Compute Cloud (EC2) platform using the new Amazon Web Services (AWS) Command Line Interface (CLI) tools. Part four handled choosing an AWS region. Now I'll address adding AWS security groups.
I'm building two EC2 machines: one is a Puppet master and the other hosts Apache. Before I can build the machines, I have to think about security. The moment a machine is public, it is a target for cracking -- that's why a lot of work goes into protecting the system, not just providing a customer service.
These procedures set up two security building blocks.
- Create AWS security groups: These groups restrict access to my new services.
- Create a key pair: No one can connect to my new machines without these keys.
Controlling conversations with AWS security groups
These security groups allow puppets to talk to the puppet master. Everyone else is banned from talking to the puppet master.
- p-agent-group: a security group for my puppet agent.
- p-master-group: a security group for my puppet master.
Behind the scenes, an AWS firewall allows requests from one source to one destination. All other requests are unceremoniously dropped. The source is the agent machine, and the destination is an internet socket.
What's an internet socket?
An internet socket is the door that all those naughty crackers will try to get through.
At the OS level, the puppet master is just another TCP server -- it provides a service to many clients and talks the TCP protocol with them. Like a web server, a mail server, and a DNS server, the puppet master listens to an internet socket, waiting for clients to send requests.
An internet socket is an IPv4 address and a port, usually written in the form (IP address):(port), like this.
A server can pick any free port to listen to. The IANA decided this first-come first-served port selection was confusing and came up with a list of 1,000 system ports, including 22 (SSH), 53 (DNS), and 443 (HTTPS).
Your web browser is a client, and it understands internet sockets. Go to the URL bar of your browser and add the port after the site name to see what happens. Here's an example: https://www.google.co.uk:443/.
Create the p-agent-group and p-master-group security groups
Create two security groups: one for the puppet master and one for the puppet agents.
The Puppet master listens for requests on TCP port 8140 -- that's a problem because AWS security, by default, blocks traffic to EC2 machines. We need a puppet master security group that allows incoming requests. The master will receive puppet requests from agents and SSH requests from sysadmins.
Use a few of the new AWS CLI commands to create the p-agent-group security group.
1. Create a security group for the puppets. Use the new aws ec2 create-ecurity-group command. The old equivalent of this command is ec2-create-group p-agent-group -d "my puppet agent security group".
nick:~ $ aws ec2 create-security-group --group-name p-agent-group --description "my puppet agent security group" sg-345608f3 true nick:~ $
2. Allow SSH requests from anywhere to the puppets. If you have stared at an Operation timed out error message, you won't forget this step. Use the new authorize-security-group-ingress command.
nick:~ $ aws ec2 authorize-security-group-ingress --group-name p-agent-group --protocol tcp --cidr 0.0.0.0/0 --port 22 true nick:~ $
3. Allow web requests from anywhere to the puppets.
aws ec2 authorize-security-group-ingress --group-name p-agent-group --protocol tcp --cidr 0.0.0.0/0 --port 80 aws ec2 authorize-security-group-ingress --group-name p-agent-group --protocol tcp --cidr 0.0.0.0/0 --port 443
4. Check your work. This is the equivalent of the old ec2dgrp (ec2-describe-group) command to list groups and permissions. The commands have changed, but the output is still difficult to understand.
nick:~ $ aws ec2 describe-security-groups --group-names p-agent-group SECURITYGROUPS my puppet agent security group sg-345608f3 p-agent-group 123894605340 IPPERMISSIONS 22 tcp 22 IPRANGES 0.0.0.0/0 IPPERMISSIONS 80 tcp 80 IPRANGES 0.0.0.0/0 IPPERMISSIONS 443 tcp 443 IPRANGES 0.0.0.0/0 nick:~ $
Create the p-master-group security group.
5. Create a security group for the puppet master.
nick:~ $ aws ec2 create-security-group --group-name p-master-group --description "my puppet master security group" sg-67860b57 true nick:~ $
6. Tell the AWS firewall to allow requests from agents to the master.
nick:~ $ aws ec2 authorize-security-group-ingress --group-name p-master-group --protocol tcp --port 8140 --source-group p-agent-group true nick:~ $
7. Allow SSH requests from anywhere to the puppet master.
nick:~ $ aws ec2 authorize-security-group-ingress --group-name p-master-group --protocol tcp --cidr 0.0.0.0/0 --port 22 true nick:~ $
8. Check your work.
nick:~ $ aws ec2 describe-security-groups --group-names p-master-group SECURITYGROUPS my puppet master security group sg-67860b57 p-master-group 123894605340 IPPERMISSIONS 8140 tcp 8140 USERIDGROUPPAIRS sg-845508f3 p-agent-group 243894605340 IPPERMISSIONS 22 tcp 22 IPRANGES 0.0.0.0/0 nick:~ $
If you made a mistake, get rid of a security group with the new delete-security-group command (like the old ec2delgrp/ec2-delete-group command).
9. Delete a group.
nick:~ $ aws ec2 delete-security-group --group-name oops-typo true
10. Is it gone?
nick:~ $ aws ec2 describe-security-groups --group-names oops-typo A client error (InvalidGroup.NotFound) occurred when calling the DescribeSecurityGroups operation: The security group oops-typo' does not exist nick:~ $
Don't expose yourself
There is a lot of information condensed in those IPPERMISSIONS lines. These lines have security implications so it's worth understanding what is going on. For instance, that IPRANGES 0.0.0.0/0 line is network administrator shorthand for "all IPv4 addresses." That's the entire internet. This permission is going to guarantee malicious attempts to breach your machine's security from around the world.
Is it safe? Yes. None of these naughty crackers will get in as long as your private key is safe and your software is up-to-date.
What, really? Is it really safe? Um, well, kinda. There are always 10 ways to do it in IT, and all of them will have their flaws. Talk to a security expert about managing risk, AWS identity management, and puppet autosigning.
Building blocks -- an EC2 keypair
- p-keypair: the public and private keys I use to SSH to my new machines.
- p-private.key: the private key from my keypair. I must keep this safe.
The ec2addkey (AKA ec2-create-keypair and ec2-add-keypair) command does some clever mathematics to create a new keypair -- two huge random numbers. One is your new public key and the other is your matching private key. AWS keeps a copy of the public key to hand over to new machines.
1. Create a new keypair. This produces a massive block of text, a bit like this.
nick:~ $ aws ec2 create-key-pair --key-name p-keypair c7:06:54:f5:d5:cc:76:90:1e:f9:db:a5:70:ef:dc:bf:33 -----BEGIN RSA PRIVATE KEY----- MIIEpQIBAAKCAQEA1oYHy7kVxHS7oH2BqXzeD9LqtxH7xHL5MGEOP 80/sm/wgVNhXXLwuJj/arqEJ4UpyoXycQdE0y1tBhtOSFNESYKRU+ sMTutpSfqSmXV0W7m9jeiL+tfsEwaujjkZ2httv4SY6TzgnG5ns5s 5R/kDwF5zlCO5JgMMCoQpSd9f1/tlDfPVNm23DNysWPPXDtNAoGAL ho4tAt281kLDLUcsIf3WTd9UD37dPGCTIRduhJIrJzxcoAxmym4ZA QLLaLcL/zNyfto+2mtgfOQ6lYOgCSxgToKmjMq/AHabQfOJ7OLR45 -----END RSA PRIVATE KEY----- nick:~ $
2. Copy the output -- that massive block of text -- to a new file called p-private.key.
3. Protect the file.
nick:~ $ chmod 600 p-private.key nick:~ $
4. Check your work. List your keypairs.
nick:~ $ aws ec2 describe-key-pairs KEYPAIRS c7:06:54:f5:d5:cc:76:90:1e:f9:db p-keypair KEYPAIRS ef:1e:08:c0:e1:3b:9e:f1:00:4a:97 oops-typo nick:~ $
If you made a mistake, delete the new keypair with the ec2delkey command (AKA ec2-delete-keypair).
5. Delete a keypair.
nick:~ $ aws ec2 delete-key-pair --key-name oops-typo true nick:~ $
6. Check your work.
nick:~ $ aws ec2 describe-key-pairs KEYPAIRS c7:06:54:f5:d5:cc:76:90:1e:f9:db p-keypair nick:~ $
Repetition, repetition, repetition
No one is born with the gift of command line knowledge. Getting the hang of this stuff takes a fair bit of rote learning. Sooner or later, you will see ways of changing these procedures to better fit your work.
- Amazon Web Services leads war on cloud price reductions (TechRepublic)
- Amazon Web Services: What's it worth? (ZDNet)
- AWS plans April price cuts, maintains rapid pace (ZDNet)
Disclaimer: TechRepublic and ZDNet are CBS Interactive properties.
Nick Hardiman builds and maintains the infrastructure required to run Internet services. Nick deals with the lower layers of the Internet - the machines, networks, operating systems, and applications. Nick's job stops there, and he hands over to the designers and developers who build the top layer that customers use.