API Gateway with AWS S3

 

In this post, we will see how to read objects from S3 using AWS API GW.

I already have a bucket with a JSON file.

Here is the lambda function for reading the object from the bucket. In the below code, I have hardcoded the bucket name and the object name.

import json

import boto3

client = boto3.client('s3')

def lambda_handler(event, context):

    response = client.get_object(

    Bucket='demo-001-first',-------------> Bucket Name

    Key='bucket1.json',------------------> Object Name

)

    # convert from streaming to bytes

    data_bytes = response['Body'].read()

    #bytes to string

    data_strings = data_bytes.decode("UTF-8")

# convert from strings to dict

    data_dict = json.loads(data_strings)

    return {

        'statusCode': 200,

        'body': data_dict

    }

Finally, I have my API GW (REST) as below.

This is a simple approach.

when I access the API GW endpoint URL. I get the below response.

Lets modify the diagram.

In this version, depending the on the user request which has the bucket name in the user QUERY, we want the lambda function to fetch data from the respective buckets.

If the Query Parameters point to bucket1, then fetch data from bucket1.

Lets start tweaking our lambda function to work based on the input value (Bucket and Key) instead of hardcoded values.

This is the input to the lambda function. As we know the lambda main function has "event" and "context". Event has information on the input parameters.

Lets modify the code.

import json

import boto3

client = boto3.client('s3')

def lambda_handler(event, context):

    bucket_name = event['bucket'] ---> Getting bucket name from input

    key_name = event['key'] ---> Getting key name from input

    response = client.get_object(

    Bucket=bucket_name,

    Key=key_name,

)

    # convert from streaming to bytes

    data_bytes = response['Body'].read()

    #bytes to string

    data_strings = data_bytes.decode("UTF-8")

# convert from strings to dict

    data_dict = json.loads(data_strings)

    return {

        'statusCode': 200,

        'body': data_dict

    }

We are done with the lambda part.

Let's go back to the API GW and configure URL Query String Parameters under the METHOD REQUEST.

We configure the user query with 2 parameters "bucket" and "key".

Now, under "Integration Request", we need to create a "Mapping Template" which typically extracts the data from URL Query String Parameters.

This extracts the bucket and key from the URL query and pass it to the invocation lambda as input variables.

Lets test it.

It works.