Configuring S3 storage cloud ACLs across buckets with CloudBerry Explorer

Amazon Web Services S3 storage is easily administered with the CloudBerry Explorer tool. In this blog post, IT pro Rick Vanover shares a few notes about intra-bucket transfers and the inherited ACL behavior with these transfers.

The Amazon Web Services (AWS) Simple Storage Service (S3) cloud is one of the best cloud storage solutions in the market today. So much so, that an additional region has been added – APAC Tokyo. The APAC Tokyo region offers a number of the AWS catalog and is competitively priced lower than the US N. California region, but slightly more expensive than the US Standard, EU Ireland, and APAC Singapore regions. As a side note, the S3 region in Tokyo stayed online during the recent earthquake in Japan.

For accessing S3 storage, I’ve long used the CloudBerry Explorer tool. Further, I’ve started to organize a few of my S3 buckets to suit my level of cloud technology usage. For my lab and a few other interests, I maintain three S3 buckets. A bucket in the S3 storage cloud is loosely analogous to a file share. Each bucket has its own home (a region in the S3 storage cloud) and its entry-level access control lists (ACLs) as well as inherited ACLs.

An S3 bucket can be created with the AWS Management Console or via the CloudBerry Explorer tool. I find myself using CloudBerry Explorer more frequently; primarily because of the more integrated view of the local file system. The same view of the both CloudBerry Explorer and the AWS Management Console are shown in Figure A below: Figure A

The AWS Management Console and CloudBerry Explorer tool compared

The AWS Management Console and CloudBerry Explorer tool compared (click to enlarge)

One of the ways I use the S3 storage cloud is to have some public material as well as some private material, for things such as encrypted backups. Occasionally, I want to transfer something from one of the private S3 zones to a public zone. Rather than keeping a list of ‘one-off’ permissions for these roles, I’ve created a separate bucket. One of the cool things about S3 pricing is that if you perform a transfer within a region, there is no fee for the transfer. The storage consumption at the destination may still apply, however. In this situation, I’d go ahead and create a separate bucket for the public distribution content.

This is where ACLs come into play. Each bucket has a default ACL configuration when it is created that gives the user who created the bucket read, write, read ACP, write ACP and the full control permission for the parent object. The ACP permissions allow users to read or write the ACL for logging purposes. The default permissions further do not allow any other user class access to the files within the bucket. The issue becomes that with any transfer operations, the existing ACLs are removed and the ACL of the parent within the destination are assigned to the new storage object. Figure B below shows a file that is configured with public (HTTP access) for anyone on the Internet, which it received as the configuration of the parent bucket: Figure B

The generic ACL panel, showing public read access

The generic ACL panel, showing public read access

This isn’t a problem, but simply the way S3 behaves for transfer between buckets. This applies whether or not the transfer is in the same region. CloudBerry Explorer has the ability to re-apply the ACL settings from the parent to the contents, should there be any inconsistencies for a bucket that is to have the same configuration for all contents. The CloudBerry Explorer PRO tool allows a bucket policy to be crafted than can get very granular, with over countless variables and scripting capabilities for bucket ACL policies.

Do you dabble much with the S3 storage cloud and ACLs? If so, share your comments below.