AWS - S3 Replication

-


In this post will see what is s3 replication and how to use it.

Replication enables automatic, asynchronous copying of objects across Amazon S3 buckets. 

Buckets that are configured for object replication can be owned by the same AWS account or by different accounts.

You can replicate objects to a single destination bucket or to multiple destination buckets. The destination buckets can be in different AWS Regions or within the same Region as the source bucket.

To automatically replicate new objects as they are written to the bucket, use live replication, such as Cross-Region Replication (CRR). 

To replicate existing objects to a different bucket on demand, use S3 Batch Replication.

To enable CRR, you add a replication configuration to your source bucket. The minimum configuration must provide the following:

The destination bucket or buckets where you want Amazon S3 to replicate objects.

An AWS Identity and Access Management (IAM) role that Amazon S3 can assume to replicate objects on your behalf.

I am going to create 2 buckets:

us-east-1: Source bucket.

us-east-2: Destination bucket.



We need to enable versioning on both the source and destination buckets for replication to work.

Let's proceed with creating the replication rule. From the source bucket.





We have selected the source and destination buckets.

During the replication setup, we can make changes like what storage class to be used at the destination bucket.

 


Other parameters:



We are done with the setup. So, when the objects are uploaded to the source bucket they will be replicated to the destination bucket within 15 mins (RTC - Replication Time Control).

I am uploading a file to the source bucket.


Within minutes you should see them replication on the destination bucket.



Now, What happens when the destination file is deleted? Will the replication automatically copy it again? NOOOOOOO

Replication what we are seeing is ONE DIRECTIONAL. So, any changes to the source object will get replicated to the destination.

NOTE: There is BI-DIRECTIONAL replication as well.


Comments

Post a Comment

Popular posts from this blog

SRE/DevOps Syllabus

-
Image
  DEVOPS/SRE Principles. Git What is Git? Architecture of Git. Working principle of Git. Create and cloning a repo. Version control branching. Version control commit. Version control managing workflows. Git hooks. Git Reflog. Git Stash. Git Cherry Picking. Undoing changes in different states of Git. Git based terraform template management. AWS CI/CD SDLC Automation. Code Commit. Code Build. Code Deployment. Code Pipeline. Elastic Beankstalk. Code Artifact. CodeGuru. Terraform Terraform Basics. Terraform State. Working with Terraform. Terraform with AWS. Remote State. Terraform Provisioners. Terraform Import, Tainting and Debugging. Terraform Modules. Terraform Functions and Conditional Expressions. Automation using Python - AWS AWS lambda. Automating EC2 with Lambda. Automating S3 with Lambda. Automating VPC with Lambda. Cost optimization with Lambda. SNS,SQS and SES with Python. Managing and Automating AWS Security with Python. Kubernetes - CKAD Application Developer. Kubernetes A...
Read more

AWS Code Commit - CI/CD Series Part 1

-
Image
  This is a multi-part series to show how to implement a CI/CD pipeline in AWS. In this post, We will see how to use AWS Code Commit. AWS CodeCommit is a fully managed source control service that allows you to store and manage your code securely in the cloud.  It provides a private Git repository for your code and supports all Git commands and workflows. CodeCommit also integrates with other AWS services like AWS CodePipeline and AWS CodeBuild, allowing you to automate your entire software release process. This can help you streamline your development workflows and improve the overall efficiency of your team. I created a repo from AWS CodeCommit -> Create Repository. And I begin with cloning the repo locally. So that I can make changes and push it. To clone the repo click on "HTTPS" under Clone URL. I make changes to index.html file. Now push it to AWS CodeCommit. Let’s verify the changes in AWS Code commit. Changes are reflected in AWS CodeCommit. We are done with Part ...
Read more

Docker - Preventing IP overlapping

-
Image
  What is IP address overlapping? When the container IP address overlaps with another IP external, then the application running on the container is not accessible from outside due to the IP address overlapping.  NOTE: This IP address overlapping can happen on the bridge interface IP or docker container IP. Let's see how to address this IP overlapping issue on the bridge interface. I already have a running container.  root@demo:~# docker run -d --name web --network mynet1 nginx:alpine 268647e11011d513d2357ce1c81e0f48787038a83d01d028d31a5b29dbdb885b root@demo:~# docker container ls CONTAINER ID   IMAGE          COMMAND                  CREATED         STATUS         PORTS     NAMES 268647e11011   nginx:alpine   "/docker-entrypoint.…"   4 seconds ago   Up 3 seconds   80/tcp  ...
Read more