AWS upload file from EC2 to S3

Uploading a file from an EC2 command line to an S3 bucket is simple but requires some preparation. Finding the places to configure and the appropriate values took me some hours (I am not too experienced with AWS stuff). This document summarizes how I did it, finally.

Overview

Uploading a file is just (don’t forget the trailing slash /)

aws mv any.txt s3://ilapark-data/dev/

To make this work, we have to

• install AWS CLI
• allow EC2 user to access the bucket and folder
• give EC2 user required S3 capabilities

Prerequisites:

• The environment uses amazon Linux distribution

Install AWS CLI

yum update -y
install yum packages
yum -y install awscli

Give EC2 user required S3 capabilities

The EC2 instance is associated with an IAM role. This role you can find in the EC2 instance details

Following the link, you find the Permission policies of the role. Here you have to create and attach a new policy (“Add permissions” → “Create inline policy”). The policy has to contain the action needed (s3:PutObject). In case you want the user to do other actions on the bucket, add the respective Action items to the Permissions list.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PutToS3Bucket",
            "Effect": "Allow",
            "Action": [
                "s3:Put*",
            ],
            "Resource": "*"
        }
    ]
}

Allow EC2 user to access bucket

We have to assign permissions to the bucket at root level. These permissions describe the resource to be accessed, the user who accesses and the action to be done. Open the S3 bucket dashboard (https://s3.console.aws.amazon.com/s3/bucket) and open the destination bucket configuration. The “Permissions” tab contains the section “Bucket policy”. Press “Edit” here to modify the JSON handling the access. Enter your policy here like

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowPutDataForDEV",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::273275915349:role/sy-ilapark_dev_app_ec2-role"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::ilapark-data/dev/*"
       }
    ]
}

Some notes about the values of the JSON above:

• Sid: name or the permission you can define as you want
• Principal: This is the ARN of the role associated with the EC2 instance as described in the previous section. Open the role configuration page and find it in the upper part center of the sceen.

• Action: The S3 action you want to allow.
• Resource: The bucket with path you want to allow access to. This can be the whole bucket or a path inside the bucket. Don’t forget the trailing slash with wildcard /*

At the end, the main thing to do is configuring the EC2 role and specifying the S3 access. Hope this text helps you implementing it fast.

Thanks for your feedback.