2023-03-13

Deploying AWS Lambda with AWS CLI

Introduction

In this article, I will introduce the code for deploying AWS Lambda using AWS CLI. The code we introduce will allow you to achieve the following:

Building Lambda using Docker images
Integrating with Amazon API Gateway
Setting Provisioned Concurrency on Lambda
Continuously updating Lambda

Directory Structure

We will write the code according to the following directory structure.

├── Dockerfile
├── Makefile
├── deploy
│   ├── config
│   │   ├── policy.json
│   │   └── role.json
│   ├── push-ecr.sh
│   ├── provision-api-gateway.sh
│   ├── provision-ecr.sh
│   ├── provision-lambda.sh
│   └── update-lambda.sh
└── src/ # source code

Setting Environment Variables

Set the following environment variables.

AWS_ACCOUNT_ID
AWS_REGION
IMAGE_NAME
LAMBDA_ALIAS
PROVISIONED_CONCURRENCY

# AWS account ID
$ export AWS_ACCOUNT_ID=123456789123

# AWS region
$ export AWS_REGION=ap-northeast-1

# ECR image and lambda name
$ export IMAGE_NAME=my-program

# Lambda alias
$ export LAMBDA_ALIAS=prod

# Number of Lambda Provisioned Concurrency
$ export PROVISIONED_CONCURRENCY=1

Create a shell script deploy/provision-ecr.sh to create an ECR repository.

deploy/provision-ecr.sh

#!/bin/sh

# Create ECR repository
aws ecr create-repository --repository-name ${IMAGE_NAME} --region ${AWS_REGION}

Then, create a shell script deploy/push-ecr.sh to execute the following:

Log in to ECR
Build Docker image from Dockerfile
Push the built Docker image to ECR.

deploy/push-ecr

#!/bin/bash

# ECR repository URI
ECR_URI=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$IMAGE_NAME:latest

# ECR login
aws ecr get-login-password | docker login --username AWS --password-stdin https://$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com

# Build docker image
docker build -t $ECR_URI .

# Push docker image to ECR
docker push $ECR_URI

Create JSON files for IAM Role and Policy to grant permissions to Lambda.

deploy/config/policy.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "*"
    }
  ]
}

deploy/config/role.json

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Then, create a shell script provision-lambda.sh to execute the following:

Create IAM Policy for Lambda from deploy/config/policy.json
Create IAM Role for Lambda from deploy/config/role.json
Attach the Policy to the Role
Create Lambda (using the image from ECR as the source)
Create Lambda version and alias (to use with Provisioned Concurrency)
Set Provisioned Concurrency on Lambda.

deploy/provision-lambda.sh

#!/bin/bash

# Create IAM policy for lambda
aws iam create-policy --policy-name AWSLambdaBasicExecutionRole-${IMAGE_NAME} --policy-document file://deploy/config/policy.json

# Create IAM role for lambda
aws iam create-role --role-name ${IMAGE_NAME}-lambda-role --assume-role-policy-document file://deploy/config/role.json

# Sleep for waiting provision of policy and role creation
sleep 10

# Attach policy to role
aws iam attach-role-policy \
--role-name ${IMAGE_NAME}-lambda-role \
--policy-arn arn:aws:iam::${AWS_ACCOUNT_ID}:policy/AWSLambdaBasicExecutionRole-${IMAGE_NAME}

# Sleep for waiting attachment
sleep 10

# Create lambda
aws lambda create-function  \
  --function-name ${IMAGE_NAME} \
  --package-type Image \
  --code ImageUri=${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${IMAGE_NAME}:latest \
  --timeout 600 \
  --memory-size 10240 \
  --ephemeral-storage Size=2048 \
  --role arn:aws:iam::${AWS_ACCOUNT_ID}:role/${IMAGE_NAME}-lambda-role


# Wait for Lambda function to be created
while true; do
  LAMBDA_STATUS=$(aws lambda get-function --function-name ${IMAGE_NAME} --query "Configuration.State" --output text)

  if [[ ${LAMBDA_STATUS} == "Active" ]]; then
    break
  elif [[ ${LAMBDA_STATUS} == "Failed" ]]; then
    echo "Lambda creation failed" >&2
    exit 1
  else
    echo "Waiting for Lambda function to be created..."
    sleep 10
  fi
done

echo "Lambda function created."

# Create lambda version and alias
LAMBDA_VERSION=$(aws lambda publish-version --function-name ${IMAGE_NAME} --region ${AWS_REGION} --query 'Version' --output text)

aws lambda create-alias \
  --function-name ${IMAGE_NAME} \
  --name ${LAMBDA_ALIAS} \
  --function-version ${LAMBDA_VERSION} \
  --region ${AWS_REGION}

# Add provisioned concurrency to lambda
aws lambda put-provisioned-concurrency-config \
  --function-name ${IMAGE_NAME} \
  --qualifier ${LAMBDA_ALIAS} \
  --provisioned-concurrent-executions "$PROVISIONED_CONCURRENCY" \
  --region ${AWS_REGION}

# Wait for provisioned concurrency to be configured
while true; do
  CONCURRENCY_STATUS=$(aws lambda get-provisioned-concurrency-config \
    --function-name ${IMAGE_NAME} \
    --qualifier ${LAMBDA_ALIAS} \
    --query "Status" --output text)

  if [[ ${CONCURRENCY_STATUS} == "READY" ]]; then
    break
  elif [[ ${CONCURRENCY_STATUS} == "FAILED" ]]; then
    echo "Provisioned concurrency addition failed" >&2
    exit 1
  else
    echo "Waiting for provisioned concurrency to be configured..."
    sleep 10
  fi
done

echo "Provisioned concurrency added to the Lambda function."

Create a shell script update-lambda.sh to continuously update Lambda. The following will be executed in update-lambda.sh:

Update Lambda from the specified ECR image
Update Lambda alias (updating the alias also updates the Provisioned Concurrency)

deploy/update-lambda.sh

# !/bin/bash

# Update lambda
aws lambda update-function-code --function-name ${IMAGE_NAME} --image ${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_REGION}.amazonaws.com/${IMAGE_NAME}:latest

while true; do
  LAST_UPDATE_STATUS=$(aws lambda get-function --function-name ${IMAGE_NAME} --region ${AWS_REGION} --query 'Configuration.LastUpdateStatus' --output text)

  if [ ${LAST_UPDATE_STATUS} != "InProgress" ]; then
    break
  fi

  echo "Waiting for Lambda function update to complete..."
  sleep 10
done


NEW_LAMBDA_VERSION=$(aws lambda publish-version --function-name $IMAGE_NAME --region $AWS_REGION --query 'Version' --output text)

# Update lambda alias
aws lambda update-alias \
  --function-name $IMAGE_NAME \
  --name $LAMBDA_ALIAS \
  --function-version $NEW_LAMBDA_VERSION \
  --region $AWS_REGION

# Wait for provisioned concurrency to be configured
while true; do
  CONCURRENCY_STATUS=$(aws lambda get-provisioned-concurrency-config \
    --function-name ${IMAGE_NAME} \
    --qualifier ${LAMBDA_ALIAS} \
    --query "Status" --output text)

  if [[ ${CONCURRENCY_STATUS} == "READY" ]]; then
    break
  elif [[ ${CONCURRENCY_STATUS} == "FAILED" ]]; then
    echo "Provisioned concurrency addition failed" >&2
    exit 1
  else
    echo "Waiting for provisioned concurrency to be configured..."
    sleep 10
  fi
done

echo "Provisioned concurrency added to the Lambda function."

Create a shell script provision-api-gateway.sh to set up API Gateway. provision-api-gateway.sh will execute the following:

Create REST API
Create POST method
Integrate with Lambda
Set permissions for Lambda
Deploy API Gateway

deploy/provision-api-gateway.sh

#!/bin/bash

LAMBDA_FUNCTION_ARN=arn:aws:lambda:${AWS_REGION}:${AWS_ACCOUNT_ID}:function:${IMAGE_NAME}

# Create REST API
REST_API=$(aws apigateway create-rest-api --name $IMAGE_NAME)
REST_API_ID=$(echo $REST_API | python -c "import sys, json; print(json.load(sys.stdin)['id'])")

# Get root resource ID
RESOURCE=$(aws apigateway get-resources --rest-api-id $REST_API_ID)
RESOURCE_ID=$(echo $RESOURCE | python -c "import sys, json; print(json.load(sys.stdin)['items'][0]['id'])")

# Create POST method for the root resource
aws apigateway put-method \
    --rest-api-id $REST_API_ID \
    --resource-id $RESOURCE_ID \
    --http-method POST \
    --no-api-key-required \
    --authorization-type NONE

# Create integration for Lambda function
aws apigateway put-integration \
    --rest-api-id $REST_API_ID \
    --resource-id $RESOURCE_ID \
    --http-method POST \
    --type AWS \
    --integration-http-method POST \
    --uri "arn:aws:apigateway:${AWS_REGION}:lambda:path/2015-03-31/functions/${LAMBDA_FUNCTION_ARN}:${LAMBDA_ALIAS}/invocations"

aws apigateway put-method-response \
    --rest-api-id $REST_API_ID \
    --resource-id $RESOURCE_ID \
    --http-method POST \
    --status-code 200 \
    --response-models '{"application/json": "Empty"}'

aws apigateway put-integration-response \
    --rest-api-id $REST_API_ID \
    --resource-id $RESOURCE_ID \
    --http-method POST \
    --status-code 200 \
    --response-templates '{"application/json": ""}'

# Add Lambda permission
aws lambda add-permission \
    --function-name $IMAGE_NAME:${LAMBDA_ALIAS} \
    --statement-id apigateway-access \
    --action lambda:InvokeFunction \
    --principal apigateway.amazonaws.com \
    --source-arn "arn:aws:execute-api:${AWS_REGION}:${AWS_ACCOUNT_ID}:${REST_API_ID}/*/POST/"

# Deploy API Gateway
aws apigateway create-deployment \
    --rest-api-id $REST_API_ID \
    --stage-name prod

# Output API Gateway endpoint
echo "https://$REST_API_ID.execute-api.$AWS_REGION.amazonaws.com/prod"

Makefile

Create a Makefile like the following:

Makefile

.PHONY: deploy-init deploy

deploy-init:
	sh deploy/provision-ecr.sh
	sh deploy/push-ecr.sh
	sh deploy/provision-lambda.sh
	sh deploy/provision-api-gateway.sh

deploy:
	sh deploy/push-ecr.sh
	sh deploy/update-lambda.sh