Set up Cross-Region Replication for S3 Buckets

When it comes to storage services on the AWS Cloud, we cannot mention all of them without talking about S3. Amazon S3 is a cloud-based object storage service with industry-leading scalability, data availability, security, and performance. Data lakes, websites, cloud-native applications, backups, archives, machine learning, and analytics are some of the amounts of data that can be stored and protected by S3. In this guide, we will be discussing how to set up Cross-Region Replication for S3 Buckets. Before proceeding, I will like to point to other posts that can help you get familiar with the AWS S3 and its bucket. Review the following posts: How to Delete AWS S3 Bucket and its Objects using  AWS CLI in Linux, and how to sync S3 Bucket with an EC2 instance.

Amazon S3

Amazon S3 is a simple storage service, is one of the most popular object storage services offered by Amazon Web Services, and it is engineered for 99.999999999% (11 9’s) of durability, and stores data for millions of customers all around the world. It provides robust support for seamless and real-time data replication, a feature that most organizations strive for. Amazon Web Services (AWS) is a cloud provider that offers users and businesses robust end-to-end cloud-based solutions and APIs.

It enables users to access, retrieve, and replicate their data on demand and seamlessly across multiple regions. This will discuss S3 cross-region replication step-by-step instructions to help you set up replication within S3 and duplicate your data seamlessly across buckets located in different regions. Upon completing this content, you will have a thorough understanding of S3 data replication, and you will be able to set up Cross Region Replication easily. Here are some related guides: How to Serve Private S3 Bucket Contents Via CloudFront, how to enable Amazon S3 default bucket encryption using S3 Console, and how to grant public access to S3 Bucket using Policy.

Understanding S3 Replication

Is the automatic, asynchronous copying of objects between buckets in the same or different AWS Regions. Replication copies updated objects and newly created objects from a source bucket to a destination bucket or buckets.

Replication configuration involves adding replication rules to the source bucket. The destination bucket or buckets where the replicated objects are stored are determined by replication rules, together with the source bucket objects to replicate. The items in a bucket or a subset of objects with a particular key name prefix, one or more object tags, or both, can be replicated using a rule. The source bucket’s destination bucket may be in the same AWS account or a separate account.

When an object version ID is specified as one to be deleted, Amazon S3 removes that object version from the source bucket. But the deletion is not duplicated in the target bucket. It does not remove the identical object version from the destination bucket, in other words. This guards against unauthorized data removals.

Note: Replication rules are automatically enabled when added to buckets, so they talk place as soon as you save the rule.

Why Do You Need to Setup Cross-Replication for your S3 Bucket?

One of the reasons why setting up cross-region replication is important is to allow you the flexibility of managing your data across regions to prevent data loss. Replication can be really helpful when it comes to protecting your critical data against failure as it allows you to fall back on the data you have replicated in another region.

Prerequisites for setting up Cross-Region Replication for S3 Buckets

To follow along in this post, you will need to have the following:

1. Active AWS Subscription
2. An IAM user with policies defined to grant access to manage S3 Bucket. See this post to learn how to create policies.

Setting up S3 Bucket Cross-Region Replication

You can now perform Cross-Region Replication for your data in S3 after you’ve created your S3 buckets and configured their policies. To do so, follow the steps below:

Step 1 – Login to the AWS Management Console at AWS Console

Step 2 – Search for S3 or click on S3 from the recently visited resources

Step 3 – Create the Source Bucket. As shown in the screenshot below, the name of our source bucket is mysourcebucket023 and it’s created in the us-east-1 region. Feel free to name your source bucket with your desired name and select your preferred region.

After creating the source bucket, upload objects into it. Here, we have uploaded two objects to the bucket.

Step 4 – Create the Destination Bucket.

Setting up S3 Replication Rules

Step 1 – To do this, choose the source bucket. In this case, our source bucket is in the us-east-1 region. If you have more than two buckets in your account, in the Buckets list, choose the name of the bucket that you want to create the replication rule.

Step 2 – In the selected Bucket’s console, choose Management, scroll down to Replication rules, and then choose Create replication rule.

Step 3 – Enter a name for your rule under Rule name to help you identify it later. The name must be unique within the bucket and must be required. You will be required to enable versioning for the source bucket. Click to enable it. Under the status tab, choose “Enabled” to enable the rule when created.

Step 4 -Choose apply to all objects in the bucket or limit the scope of this rule using one or more filters if you want to replicate all objects that have the same prefix. For the destination, select choose a bucket from the account and browse to select the destination bucket. Also remember to enable versioning for the destination bucket.

Note: To replicate to a bucket or buckets in a different AWS account, select Choose a bucket in another account, and enter the destination bucket account ID and name.

Step 5 – Set up an AWS Identity and Access Management (IAM) role that Amazon S3 can assume to replicate objects on your behalf.

To create an IAM role, do one of the following on the Replication rule configuration section, under the IAM role:

1. Choose Create new role under Choose from existing IAM roles to have Amazon S3 create a  new IAM role for you. When you save the rule, a new policy for the IAM role is created that matches the source and destination buckets you specify. You have the option of using an existing IAM role. If you do, you must select a role that grants Amazon S3 the replication permissions it requires. If this role does not grant Amazon S3 sufficient permissions to follow your replication

- To select an existing role, use the dropdown menu or enter the IAM role ARN.

Step 6 – Finally, click Save to create the rule. After you save your rule, you can edit, enable, disable, or delete it by selecting it and selecting Edit rule.

From the below screenshots, we now have the same objects in both source and destination buckets.

Setting up S3 Bucket Cross-Region Replication Using AWS CLI

When the source and destination buckets are owned by the same AWS account, you use the AWS CLI to set up replication as follows:

1. Create buckets for the source and destination.
2. Turn on versioning for the buckets.
3. Create an IAM role that allows Amazon S3 to replicate objects.
4. To the source bucket, add the replication configuration.

When source and destination buckets are held by the same AWS account, replication can be set up.

To do so, run the command below to create the source bucket, and enable versioning.

$aws s3 mb s3://my-bucket
$aws s3api put-bucket-versioning --bucket my_bucket --versioning-configuration Status=Enabled

In this guide, you have learned how to set up Cross-Region Replication for S3 Buckets. I hope you found this blog post helpful. If you have any questions, please let me know in the comment session.