Upload from AWS Lambda function to S3 [Serverless Framework Edition]

I’ll introduce how to upload to S3 from a function deployed to AWS Lambda with Serverless Framework.

Prerequisites

Assume setup based on the article Deploy to AWS Lambda with Serverless Framework | CodeNote on this site.

InvalidAccessKeyId error with default settings

With the default Serverless Framework settings, trying to use S3 from a Lambda function will result in the error InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.

{
  InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
    at Request.extractError (/var/task/node_modules/aws-sdk/lib/services/s3.js:585:35)
    at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request. (/var/task/node_modules/aws-sdk/lib/request.js:38:9)
    at Request. (/var/task/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
  message: 'The AWS Access Key Id you provided does not exist in our records.',
  code: 'InvalidAccessKeyId',
  region: null,
  time: 2018-12-22T15:40:48.422Z,
  requestId: 'A7BA76D679AA0908',
  extendedRequestId: 'BgpkYpdh88xEFW1IOdb6E1hKS484X3ZVcsymi0TL2S/x5nTF0CNsZQ18MkIE5yYn/6WjJz2KaSc=',
  cfId: undefined,
  statusCode: 403,
  retryable: false,
  retryDelay: 41.81325093666113
}

Steps to use S3 from AWS Lambda

I’ll introduce the steps following Set up AWS S3 / Crawling the target site and save as HTML to S3 · Pull Request #4 that I’m developing.

npm install aws-sdk

npm install --save aws-sdk

Define S3 bucket as environment variable

By defining environment: under provider: in serverless.yml as follows, you can use environment variables from serverless.yml like ${self:provider.environment.S3_BUCKET}.

provider:
# you can define service wide environment variables here
  environment:
    S3_BUCKET: your-bucket-name-${opt:stage, self:provider.stage}

Create S3 bucket

While you could create an S3 bucket from the S3 Management Console, since you’re using Serverless Framework, I recommend adding the following configuration to serverless.yml to create the S3 bucket via CloudFormation.

# you can add CloudFormation resource templates here
resources:
  Resources:
    S3BucketResource:
      Type: AWS::S3::Bucket
      Properties:
        BucketName: ${self:provider.environment.S3_BUCKET}
  Outputs:
     S3BucketOutput:
       Description: "Description for the output"
       Value: S3BucketResource

Grant IAM permissions to S3 bucket

Up to this point, you’ve only created an S3 bucket, and the Lambda function has no permissions to the created S3 bucket.

Modifying provider: iamRoleStatements: in serverless.yml as follows allows the Lambda function to access S3.

provider:
# you can add statements to the Lambda function's IAM Role here
  iamRoleStatements:
    - Effect: "Allow"
      Action:
        - "s3:ListBucket"
        - "s3:GetObject"
        - "s3:PutObject"
        - "s3:DeleteObject"
      Resource:
        Fn::Join:
          - ""
          - - "arn:aws:s3:::"
            - ${self:provider.environment.S3_BUCKET}
            - "/*"

Upload from Lambda to S3

Finally, here’s sample code to upload an HTML file from a Lambda function to the S3 bucket created and granted IAM permissions above.

'use strict';
const AWS = require('aws-sdk');

module.exports.putS3 = async (event, context) => {
  try {
    const html = 'Hello world';

    AWS.config.update({
      accessKeyId: process.env.AWS_ACCESS_KEY_ID,
      secretAccessKey: process.env.AWS_SEECRET_ACCESS_KEY,
      region: process.env.AWS_REGION
    });
    const s3 = new AWS.S3();
    const response = await s3.putObject({
      Bucket: process.env.S3_BUCKET,
      Key: `${Date.now()}.html`,
      ContentType: 'text/html',
      Body: html
    }).promise();

    console.log({
      result: 'OK',
      s3response: response
    });
  } catch (err) {
    console.error(err);
    console.log({
      result: 'NG'
    });
  }
};

That’s all from the Gemba where we want to manipulate S3 from AWS Lambda functions deployed with Serverless Framework.