Cloud

How to get started with Amazon's Simple Storage Service

Nick Hardiman shows you the step-by-step to creating buckets and objects in the Amazon Simple Storage Service (S3).

Among the many cloud services that Amazon offers is the Simple Storage Service, also known as S3. To get an idea of how you can use it, I'll start with a very simple example of a basic use case. Let's say, I want to use Amazon to distribute a file to my colleagues. These are my distribution requirements.

  • It's a big file - 4GB.
  • I want to upload it to a public place
  • I want to store it for a few weeks - I don't care what happens after that.
  • I am not worried about security. This is not a sensitive file.
  • I guess a hundred people will download it.

The easiest way for me to distribute a file is not Amazon's EBS or S3 - it's Dropbox. I already subscribe to the Dropbox free service, and sharing a Dropbox file is easy. Unfortunately size matters. The Dropbox free tier has a 2GB limit, so my 4GB file is way too big. I could recruit friends, get my Dropbox allowance upped, and do some brutal housekeeping, but that sounds like work.

The next item on my easy list is to use the AWS console and store my file where Dropbox stores its files - on Amazon S3 (Simple Storage Service). I need a bucket, an object, and cash in my wallet.

A bucket and an object

When Amazon launched its storage service in 2006 and cloud computing got real, they used the concept of each customer putting objects in buckets. That's easy, nursery-rhyme easy.

Each object is a file (with extra behind-the-scenes bits). The size limit for an object is a ridiculously huge 5TB, and there is no limit to the total number of objects. Amazon now stores about a trillion objects.

Each bucket is a container for objects. The name of each bucket has to be completely unique in the world of AWS.

Cash in my wallet

Creating a bucket and uploading the file cost me nothing. The storage and download requests cost pretty much nothing. The download data transfer is what hurts.

This is what the Amazon S3 pricing page tells me to expect for my file.

  • Upload - free
  • Storage - $0.50 ($0.125 x 4GB)
  • Requests - $0.01 ($0.01 per 10,000 requests)
  • Download - $47.88 (4GB x 100 people = 400GB data transfer, the first GB is free and the next 399GB x $0.120)

A month's enterprise usage is going to be far heavier. A terabyte of transfer costs over a hundred dollars. Use the Amazon calculator to make sure there are no unpleasant surprises.

Pick a name for your new bucket.

There are limits to the kind of label you can choose for your bucket. Some characters are good and others are bad. The label nicks-1st-bucket is OK. NicksFirstBucket might be OK. Nick's first bucket is not OK.

If you are going to get serious with S3, create a bucket naming policy. Here are your basic rules.

  • Good characters: a-z, numbers 0-9 and the special characters period (.), underscore (_ ), and dash (-).
  • Bad characters: everything else. No spaces, apostrophe's, ampersands or umlauts (sorry Günter).
  • Risky characters: The letters A-Z are OK in the US region but aren't acceptable in Ireland.

The S3 management console does check what you type, so you don't have to worry about mistakes. The console will punish you for trailing spaces (the ones you can't see at the end of a string), such as "nicks-1st-bucket ".

The bucket label appears in the download URL so rude words and sneaky insults are a bad idea.

Create a new bucket.

Bucket creation is free. Create dozens of buckets and Amazon won't charge you.

  1. Open the AWS console. URL: https://console.aws.amazon.com/console/home. The AWS home page opens.
  2. Navigate to the S3 control panel. The link is on this home page. Amazon Web Services > Storage & Content Delivery > S3 Scalable Storage in the Cloud. The S3 Management Console opens.
  3. Create a new bucket. Buckets > Create Bucket. A modal window opens.
  4. Type in your new bucket name. I chose nicks-1st-bucket.
  5. Choose a region. People usually choose the region closest to the downloading clients. I chose Ireland (this will show up in the URL as eu-west-1).
  6. Click the Create button. The modal window closes and your bucket name appears in the left pane.

Add an object.

At this point, the bucket name is highlighted in the left pane and the right pane says, The bucket ‘nicks-1st-bucket' is empty.

Be prepared for slow file upload. Uploading a 4GB file in a café with a 30KB per second upload speed means you will be nowhere near finished at closing time.

  1. Click the Upload button. Another modal window called Upload - Select Files opens.
  2. Click Add files and a file browser opens.
  3. Choose a file and click the Start Upload button. A Transfers pane appears with Upload task in it. When the upload is finished, the file name appears in the Objects and Folders pane.
  4. Play with the console: Add another file to the Transfers list.
  5. Hit the cancel and retry buttons.
  6. Play with other buttons. The Properties button reveals extra features such as permissions, logging and notifications. The Transfers button opens and closes the Transfers pane.

Make the object public.

No-one else can see your uploaded object. This procedure creates a URL that you can share, like this.

https://s3-eu-west-1.amazonaws.com/nicks-1st-bucket/my-massive-image-file.img

  1. Find the Actions menu. Click the Actions button, or right click (or Ctrl-click on a Mac) the file name.
  2. Make the object publicly available. Select Make Public. A popup Are you sure window opens.
  3. Click OK. A Make Public task appears in the Transfers pane.
  4. Check your work.
  5. Find the object's public URL. Properties pane | Details tab | Link: value.
  6. Click the URL. A new browser tab opens and downloads the object.

Clean up.

Don't leave old public objects for others to download.

  1. Delete the object. Objects and Folders pane | Actions menu | Delete | OK. A Delete task appears in the Transfers pane.
  2. Delete the bucket. Buckets pane | Actions menu | Delete | OK.
  3. Close the console.

Get creative

Public distribution of a file using S3 is simple (the clue is in the name), but you have to keep an eye on the download cost.

S3 has been around longer than other cloud services, and it is very popular. You can store an entire static website on S3, add security features, use BitTorrent downloads, hook up S3 to a business application and so on. When you are ready to get creative in your use of S3, search for the experience of others. An idea similar to yours has probably been worked on and documented by someone else, so save time by learning from their experience.

You can also view the gallery of this process with more screenshots by following this link: "Creating buckets and objects in Amazon Simple Storage Service."

About

Nick Hardiman builds and maintains the infrastructure required to run Internet services. Nick deals with the lower layers of the Internet - the machines, networks, operating systems, and applications. Nick's job stops there, and he hands over to the ...

1 comments
tpacheco
tpacheco

I get that this is an easy way to get started with AWS (it is and thanks for showing just how easy it is) but was wondering if you could have just done the same thing with dropbox Pro and paid just $9.99 for the whole month and saved yourself ~$30?