<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=2877026&amp;fmt=gif">

Continuous Deployment of Lambda Functions

By Prakshal Jain - October 15, 2020

Commit, Build, Deploy

Lambda is all fun and games until you need to continuously deploy functions stored in a git repository. To continuously deploy Lambda functions managed by Github, we’ll utilize AWS Pipeline and AWS CloudFormation. Each branch of the code (dev, qa, prod) will create a Lambda function with the following naming convention: <function-name>-<branch>.

Pre-Requisites

Before getting started with AWS, let’s set up the Github repo.

  1. Create a repo. Add branches for dev and qa

  2. ❗️In the root folder, create a file called buildspec.yaml and add the code HERE

  3. Create two directories in the root folder: function and scripts

  4. ❗️Save the code HERE as deploy.sh under the scripts directory.

  5. Add the lambda function under the function directory

Great! Now let’s set up an S3 bucket to store the artifacts.

S3 Bucket

  1. Navigate to S3 in AWS Management Console

  2. Create an S3 bucket to store the build artifacts containing zipped versions of the lambda functions. Name it <app_name>-lambda-code-pipeline

  3. Create three folders: dev, qa, and prod

Next, let’s set up the IAM role that’ll be used by CodeDeploy to deploy Lambda functions through CloudFormation.

IAM

  1. Navigate to IAM in AWS Management Console

  2. Click Roles on the left menu

  3. Click “Create Role”

  4. Select “CloudFormation” from the AWS Service List

  5. Click “Next”

  6. ❗️Click “Create Policy” in the top left. A new tab should open to create a custom policy. Add the code HERE and update the variable values as defined at the top of the file.

  7. Click “Review Policy”

  8. Ensure everything is correct and save the policy as cf-lambda-policy

  9. Once the policy is created, navigate back to the Roles page and click the refresh icon

  10. Search for the policy created above (e.g. cf-lambda-policy) and add it to the role

  11. Click “Next:Tags” to add tags. Click “Next” to review

  12. Ensure everything is correct and provide a name for the role, such as cf-deploy-role

  13. Click “Create Role”

Now, onto creating the Code Pipeline!

CodePipeline

CodePipeline allows us to create a continuous deployment process using CodeCommit, CodeBuild, and CodeDeploy.

The process below shows how to create the pipeline for the dev branch.

🔁 Repeat the process for qa and prod.

  1. Navigate to CodePipeline

  2. Click “Create Pipeline”

  3. Name the pipeline <app_name>-<env>. Set it to <app_name>-dev for starting off

  4. Select “New Service Role” under the Service Role menu

  5. Keep the default service name that is generated

  6. Click “Next” to set up the source details

Source

  1. Under Source Provider, choose “Github”

  2. Click “Connect to Github.” Follow the prompts to connect with your account

  3. Select the Github repo you created earlier and dev from the branch

  4. Select “Github Webhooks” for the Change Detection option

  5. Click “Next”

CodeBuild

  1. Under Build Provider, select “AWS CodeBuild”

  2. Select the region of the CodeBuild project

  3. Click “Create Project”

  4. Enter a project name, such as, <app-name>-<env>-build

  5. Under Environment Image, select “Managed Image”

  6. For Operating System, select “Amazon Linux 2”

  7. Under Runtime, select “Standard”

  8. Under Image, select “aws/codebuild/amazonlinux2-x86_64-standard:3.0.”

  9. Set Environment Type to “Linux”

  10. Under the Service role, select “Create a New Role”. Enter a name for the role, such as <app_name>-<env>-build-role

  11. Under Additional Configuration, click “Add Environment Variables”

  12. 1️⃣ Set key to S3_BUCKET; set value to the name of the S3 bucket where the artifacts will be stored

  13. 2️⃣ Set key to LAMBDA_FUNCTION_NAME; set value to the name of the Lambda function to deploy

  14. 3️⃣ Set key to BRANCH; set value to #{SourceVariables.BranchName}

  15. 4️⃣ Set key to LAMBDA_DEPLOYMENT_PREFERENCE; set value to a type of deployment (e.g. Canary10Percent30Minutes, Linear10PercentEvery1Minute, AllAtOnce)

  16. Under Build specifications, select “Use a Buildspec file”

  17. Optional: Select S3 Logs under the Logs section if you’d like logs to also be written to an S3 bucket

  18. Click “Continue to Code Pipeline.”

  19. Ensure that the build project you just created has been selected in the Project name dropdown

  20. Leave Build type as “Single Build”

  21. Click “Next”

IAM Update

🔀 Detour: let’s update the IAM role created above for CodeBuild to provide access to S3 since the build process copies the code to S3.

  1. Navigate to IAM

  2. Click on Roles in the left menu

  3. Search for the role name created above

  4. Click the role name to open it

  5. Click “Attach Policies”

  6. Click “Create Policy”

  7. Click on the JSON tab

  8. Copy the code HERE into the editor. It will give access to the S3 bucket and the Dynamodb table. ⚠️ Update the variables listed at the top of the code and delete those lines once the variables have been updated⚠️

  9. Click “Review Policy”

  10. Enter a name for the policy, such as code-build-s3-lambda-policy

  11. Click “Create Policy”

  12. Click the Refresh icon on the Policy page

  13. Search the name of the policy just created and add it to the role

  14. Click “Attach Policy”

Great! Now, the CodeBuild role has S3 and Lambda access. Time to move on!

CodeDeploy

  1. For the deploy provider, select “AWS CloudFormation”

  2. For action mode, select “Create or Update a stack”

  3. Enter a name for the stack to be created, such as, <app-name>-cf-stack

  4. Under Template, select “BuildArtifact” from the Artifact name dropdown

  5. Enter packaged.yaml for the file name

  6. Select CAPABILITY_NAMED_IAM and CAPABILITY_AUTO_EXPAND under capabilities. Learn more about these permissions HERE.\

  7. Under Role Name, select the CloudFormation role created earlier (cf-deploy-role)

  8. Click “Next”

  9. Ensure everything is accurate. Click “Create Pipeline”

  10. ⚠️ Once the pipeline is created, it’ll automatically be triggered for the first run

Tracking Progress

CodePipeline

  1. Navigate to CodePipeline

  2. Use the UI to track the progress of each stage of the pipeline

  3. Click View Details under each stage to get more information and debug in case of failures

CloudFormation

  1. Navigate to CloudFormation

  2. Click on the stack created earlier

  3. Click on the Events tab in the stack details

  4. Click the refresh button in the top right corner to refresh the events list and track the creation/updates of the stack

  5. Note if any errors occur and work towards resolving them

CodeDeploy

  1. Navigate to CodeDeploy

  2. Click Deployments on the left menu under CodeDeploy

  3. Click on the Deployment ID of the deployment just created and track the progress through the UI. It might be instant or take 10 mins based on the deployment preference you specified earlier

CloudFormation triggered CodeDeploy DeploymentCloudFormation triggered CodeDeploy Deployment

S3

  1. Navigate to S3

  2. Open the bucket created earlier to store the artifacts

  3. It should now contain the versioned lambda code in zip folders

Lambda Function

  1. Navigate to Lambda in AWS Management Console

  2. Click Functions

  3. Look for the function that you specified earlier as an environment variable (LAMBDA_FUNCTION_NAME)

  4. Open the function by clicking on the function name

  5. Click on the Qualifiers dropdown in the top-right and you should see the Lambda versions and the alias default

  6. That’s it! You’ve covered all the bases and ensured that all components are functioning! Now, just push changes to Github and it should automatically deploy the code to Lambda 🍻

Common Challenges

1. CloudFormation Stack Status: UPDATE_ROLLBACK_COMPLETE

  • The first run of the pipeline fails at the deployment stage when creating a stack. You attempt to retry the deploy stage but get the following error: Failed to update stack. Current status: UPDATE_ROLLBACK_COMPLETE.

  • Once a stack has been in the ROLLBACK state, it cannot be updated. You MUST delete the stack before proceeding.

  1. Navigate to CloudFormation

  2. Click on Stacks to get the list of stacks.

  3. Find the stack that was created by the CodePipeline.

  4. Click the radio button next to it and click Delete.

  5. Make changes to the pipeline and try again. 🔁

Author
Prakshal Jain

Tags: Cloud Services

Fill in your Details