This tutorial covers:
- Creating a new AWS CDK application
- Adding a Lambda authorizer and defining CDK constructs
- Automating and testing the deployment of the CDK stack
This is the second tutorial in a two-part series. You can also learn how to automate AWS Lambda function deployments to AWS CDK.
AWS Cloud Development Kit (AWS CDK) is an open-source framework that allows you to use the programming language of your choice to define and deploy your cloud resources. AWS CDK is an infrastructure as code (IaC) solution, similar to Terraform, that lets you use the expressive power of object-oriented programming languages to define your cloud resources. The AWS CDK framework provides libraries in most of the popular programming languages for all the major AWS services. You can use these libraries to easily define a cloud application stack for your entire system. It eliminates context switching and helps accelerate the development process. Developers do not need to learn a new programming language or a new tool to benefit from AWS CDK.
In this tutorial, I will guide you through using AWS CDK to deploy REST APIs with AWS Lambda-based authorizers. You will learn how API Gateway constructs can be used to customize the behavior of the API by adding authorizers, usage plans, throttling, rate limiting, and more.
Prerequisites
For this tutorial, you will need to set up NodeJS on your machine since you will be using it to define the AWS CDK application and the AWS Lambda handlers. You will also need to install AWS CLI and AWS CDK CLI on your system so that you can configure AWS credentials and manually build your CDK application. You will need an AWS account for deploying the application and a CircleCI account for automating the deployments. Here is a list of everything you need in place to follow along with this tutorial:
Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.
Creating a new AWS CDK project
Create a new directory for the CDK project and navigate into it. Run these commands:
mkdir aws-cdk-api-auth-lambda-circle-ci
cd aws-cdk-api-auth-lambda-circle-ci
Using the CDK CLI, run the cdk init command to create a new CDK project in TypeScript:
cdk init app --language typescript
This command creates a new CDK project with a single stack and a single application.
Note: AWS CDK supports all major programming languages, including TypeScript, Python, Java, and C#. If you choose a different programming language, you can still follow the steps in this tutorial, but the syntax will change based on the programming language that you chose.
Adding a NodeJS Lambda function
In this section, you will define an AWS Lambda function using NodeJS that can be used for proxy integration with AWS API Gateway. API Gateway’s AWS Lambda proxy integration provides a simple and powerful mechanism to build the business logic of an API. The proxy integration allows the clients to call a single AWS Lambda function in the backend whenever a REST API is called through API Gateway.
This example uses an AWS Lambda function very similar to the one defined in the first tutorial in this series, Automate AWS Lambda function deployments to AWS CDK. Just change the request and response objects for the AWS Lambda proxy integration.
First, create a lambda directory at the root of the CDK project. Inside the lambda folder, create another folder named processJob. Create a package.json file in the processJob directory for defining the dependencies. Open the package.json file, and add the following content:
{
"name": "circle-ci-upload-csv-lambda-function",
"version": "0.1.0",
"dependencies": {
"csv-stringify": "^6.0.5",
"fs": "0.0.1-security",
"uuid": "^8.3.2"
}
}
This script defines the name of the project and adds a few dependencies that will be used by the Lambda handler.
Now, navigate to the processJob folder from the terminal to install the NPM packages.
cd lambda/processJob
npm install
Next, create an index.js file in the processJob directory and add the following code snippet to it. We have taken the entire code snippet from the linked tutorial and have just modified the request and response objects.
"use strict";
const AWS = require('aws-sdk');
const { v4: uuidv4 } = require('uuid');
var fs = require('fs');
const { stringify } = require('csv-stringify/sync');
AWS.config.update({ region: 'us-west-2' });
var ddb = new AWS.DynamoDB();
const s3 = new AWS.S3();
const TABLE_NAME = process.env.TABLE_NAME;
const BUCKET_NAME = process.env.BUCKET_NAME;
exports.handler = async function (event) {
try {
const uploadedObjectKey = generateDataAndUploadToS3();
const eventBody = JSON.parse(event["body"]);
const jobId = eventBody["jobId"];
console.log("event", jobId);
var params = {
TableName: TABLE_NAME,
Item: {
jobId: { S: jobId },
reportFileName: { S: uploadedObjectKey },
},
};
// Call DynamoDB to add the item to the table
await ddb.putItem(params).promise();
return {
statusCode: 200,
headers: { "Content-Type": "application/json" },
body: JSON.stringify({
status: "success",
jobId: jobId,
objectKey: uploadedObjectKey,
}),
};
} catch (error) {
throw Error(`Error in backend: ${error}`);
}
};
const generateDataAndUploadToS3 = () => {
var filePath = "/tmp/test_user_data.csv";
const objectKey = `${uuidv4()}.csv`;
writeCsvToFileAndUpload(filePath, objectKey);
return objectKey;
};
const uploadFile = (fileName, objectKey) => {
// Read content from the file
const fileContent = fs.readFileSync(fileName);
// Setting up S3 upload parameters
const params = {
Bucket: BUCKET_NAME,
Key: objectKey,
Body: fileContent,
};
// Uploading files to the bucket
s3.upload(params, function (err, data) {
if (err) {
throw err;
}
console.log(`File uploaded successfully. ${data.Location}`);
});
return objectKey;
};
function writeCsvToFileAndUpload(filePath, objectKey) {
var data = getCsvData();
var output = stringify(data);
fs.writeFileSync(filePath, output);
// we will add the uploadFile method later
uploadFile(filePath, objectKey);
}
function getCsvData() {
return [
['1', '2', '3', '4'],
['a', 'b', 'c', 'd']
];
}
There is a reason that the request and response object needs to be modified. When the Lambda function is called through API Gateway, the request object consists of a JSON that includes the request body, HTTP method type, REST API resource path, query parameters, headers, and request context. Here is a snippet that shows this:
{
"body": "{\"jobId\": \"1\"}",
"path": "/path/to/resource",
"httpMethod": "POST",
"isBase64Encoded": false,
"queryStringParameters": {
...
},
"headers": {
...
},
"requestContext": {
...
}
}
The JSON request payload is stringified and set under the body parameter. To extract the payload in the Lambda you will have to modify the code as shown below.
const eventBody = JSON.parse(event["body"]);
const jobId = eventBody["jobId"];
Also, since the Lambda response would be used as it is by the API Gateway, you need to format the response to a JSON REST API response that includes the status code, status, headers, and response body. Therefore, you stringify and add the actual response under the body parameter of the JSON object as shown below.
{
"statusCode": 200,
'headers': {'Content-Type': 'application/json'},
"body": JSON.stringify({
"status": "success",
"jobId": jobId,
"objectKey": uploadedObjectKey
})
}
Adding a Lambda authorizer
Next, define another AWS Lambda function that will serve as a custom authorizer. Whenever a client calls the REST API, API Gateway will invoke this Lambda function passing the Authorization header value to it. The Lambda handler will validate the token sent in the Authorization header and will return an IAM policy statement if the token has adequate permissions. If the token is not authorized to call the REST API, the Lambda handler will return an error response.
First, create a lambda/authorizer directory at the root of the CDK project. Inside the authorizer directory add a package.json file for defining the dependencies. In the package.json define the name of the project and add a few dependencies that will be used by the Lambda handler.
{
"name": "circle-ci-auth-lambda-function",
"version": "0.1.0",
"dependencies": {}
}
Next, create an index.js file in the authorizer directory for the authorizer Lambda handler and add an empty Lambda handler to it.
exports.handler = function(event, context, callback) {
};
Before you implement the Lambda handler, define a method that generates the IAM policy statement granting the execute-api:Invoke permission to the REST API that invoked the authorization Lambda.
var generatePolicy = function(principalId, effect, resource) {
var authResponse = {};
authResponse.principalId = principalId;
if (effect && resource) {
var policyDocument = {};
policyDocument.Version = '2012-10-17';
policyDocument.Statement = [];
var statementOne = {};
statementOne.Action = 'execute-api:Invoke';
statementOne.Effect = effect;
statementOne.Resource = resource;
policyDocument.Statement[0] = statementOne;
authResponse.policyDocument = policyDocument;
}
return authResponse;
}
Now that you have the generatePolicy function defined, implement the Lambda handler. The Lambda handler will extract the authorization token from the event parameter and then validate the token. For a valid token, it will call the generatePolicy method to return the appropriate IAM policy.
exports.handler = function (event, context, callback) {
var token = event.authorizationToken;
switch (token) {
case "allow":
callback(null, generatePolicy("user", "Allow", event.methodArn));
break;
case "deny":
callback(null, generatePolicy("user", "Deny", event.methodArn));
break;
case "unauthorized":
callback("Unauthorized"); // Return a 401 Unauthorized response
break;
default:
callback("Error: Invalid token"); // Return a 500 Invalid token response
}
};
Now that you have defined the Lambda for the processing job as well as the Lambda for authorization, you can define CDK Constructs for the application.
Defining CDK constructs for the application
AWS CDK constructs encapsulate the configuration detail and gluing logic for multiple AWS services. CDK provides libraries in most of the major programming languages.
Replace the contents of lib/aws-cdk-api-auth-lambda-circle-ci-stack.ts file with the following code snippet that defines the constructs for AWS S3 bucket, AWS Lambda function, and AWS DynamoDB.
import {
Stack,
StackProps,
aws_s3 as s3,
aws_dynamodb as dynamodb,
aws_lambda as lambda,
Duration
} from 'aws-cdk-lib';
import { Construct } from 'constructs';
export class AwsCdkApiAuthLambdaCircleCiStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
// we will add all the constructs here
// replace bucket name with a unique name
const circleCiGwpBucket = new s3.Bucket(this, "CircleCIGwpAuthExampleBucket", {
bucketName: "<YOUR_BUCKET_NAME>",
});
const circleCiGwpTable = new dynamodb.Table(this, "CircleCIGwpAuthExampleTable", {
tableName: "CircleCIGwpAuthExampleTable",
partitionKey: { name: "jobId", type: dynamodb.AttributeType.STRING },
});
const circleCiGwpLambda = new lambda.Function(
this,
"CircleCiGwpProcessJobLambda",
{
runtime: lambda.Runtime.NODEJS_14_X,
handler: "index.handler",
timeout: Duration.seconds(30),
code: lambda.Code.fromAsset("lambda/processJob/"),
environment: {
TABLE_NAME: circleCiGwpTable.tableName,
BUCKET_NAME: circleCiGwpBucket.bucketName
},
}
);
circleCiGwpBucket.grantPut(circleCiGwpLambda);
circleCiGwpTable.grantReadWriteData(circleCiGwpLambda);
}
}
These constructs match the ones defined in the first tutorial of this series, Automate AWS Lambda function deployments to AWS CDK, so I won’t go into detail about creating these constructs. These are basic CDK constructs that create a new S3 bucket, a new DynamoDB table, and a Lambda function using the Lambda handler defined in the lambda/processJob directory. After defining the constructs, grant the appropriate IAM permissions to the Lambda function.
AWS S3 bucket names are unique across all AWS accounts so you will need to provide a unique name for your bucket.
TABLE_NAMEandBUCKET_NAMEare passed asenvironmentvariables and they would be available for use in the AWS Lambda handler.
Before defining any more constructs, you need to define in the stack:
- CDK Construct to define the authorization Lambda.
- API Gateway
TokenAuthorizerconstruct that uses the authorization Lambda as its handler. - API Gateway
RestApiconstruct for the service. - API Gateway
LambdaIntegrationconstruct that uses the process jobLambdaas its handler.
Use the Lambda integration construct to add an authorization handler method to the API token authorizer resource setting.
Defining an authorization Lambda
Next, add a CDK construct to create an AWS Lambda function for custom authorization. The authorization Lambda will use the NodeJS runtime and the code that you defined in the lambda/authorizer directory.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const circleCiAuthLambda = new lambda.Function(
this,
"CircleCiAuthLambda",
{
runtime: lambda.Runtime.NODEJS_14_X,
handler: "index.handler",
timeout: Duration.seconds(30),
code: lambda.Code.fromAsset("lambda/authorizer/"),
}
);
}
Defining a token authorizer
To define an API Gateway token authorizer, add a CDK Construct for the TokenAuthorizer. The token authorizer uses the authorization Lambda function that you defined earlier.
import {
Stack,
StackProps,
aws_s3 as s3,
aws_dynamodb as dynamodb,
aws_lambda as lambda,
//update existing import to add aws_apigateway
aws_apigateway as apigateway,
Duration
} from 'aws-cdk-lib';
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const circleCiAuthorizer = new apigateway.TokenAuthorizer(this, 'CircleCIGWPAuthorizer', {
handler: circleCiAuthLambda
});
}
Defining the REST API service
Next, define an API Gateway REST API service providing a name and description to it. You will use this RestApi service to add resources to it.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const circleCiGwpApi = new apigateway.RestApi(this, "CircleCIGWPAPI", {
restApiName: "Circle CI GWP API",
description: "Sample API for Circle CI GWP"
});
}
You can now add resources to the circleCiGwpApi service. Resources are the actual endpoints that you are creating, excluding the base URL.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const jobResource = circleCiGwpApi.root.addResource("jobs");
}
Adding a Lambda integration
A Lambda integration integrates an AWS Lambda function to an API Gateway method. You will use the process job Lambda function that you defined earlier as the handler for the Lambda integration.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const processJobIntegration = new apigateway.LambdaIntegration(
circleCiGwpLambda
);
}
Adding an API Gateway method with Lambda authorizer
Finally, add a POST method to the jobResource and use the authorization Lambda as the auth handler.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
jobResource.addMethod("POST", processJobIntegration, {
authorizer: circleCiAuthorizer,
authorizationType: apigateway.AuthorizationType.CUSTOM,
});
}
Customizing the API usage plan
In this section, you will learn how to customize the behavior and experience of the REST APIs by defining usage plans, throttling settings, and rate-limiting.
The plan uses API keys to identify API clients and who can access the associated API stages for each key.
- Usage plans: A usage plan specifies who can access the deployed APIs. The usage plan can optionally be set at the method level. API keys are associated with a usage plan and are used to identify the API client who can access the API for each key.
- API keys: API keys are string values that can be used to grant access to your API.
- Throttling limits: Throttling limits determine the threshold at which the request throttling should begin and it can be set at the API or method level.
Now you can define a usage plan for the API.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const circleCiUsagePlan = circleCiGwpApi.addUsagePlan('CircleCiUsagePlan', {
name: 'CircleCiEasyPlan',
throttle: {
rateLimit: 100,
burstLimit: 2
}
});
}
Note, that you are also defining the throttling limits along with the usage plan. The rateLimit refers to the API’s average requests per second over an extended period. The burstLimit refers to the maximum API request rate limit over a time ranging from one to a few seconds. Setting throttling limits is optional and you could choose not to enforce any such limits for your API.
Because usage plans require an API key to be associated with it to identify the clients, add an API key to it.
constructor(scope: Construct, id: string, props?: StackProps) {
// add this snippet below the existing code
const circleCiApiKey = circleCiGwpApi.addApiKey('CircleCiApiKey');
circleCiUsagePlan.addApiKey(circleCiApiKey);
}
CircleCiApiKey doesn’t have any association with the CircleCI dashboard. You can use any other name for the API key as per your requirements.
Deploying the CDK stack
Now that you have defined the CDK constructs in the stack, you can go ahead and deploy the application to an AWS account. First, deploy the application manually before automating the deployment using CircleCI. Make sure that you have the AWS CDK CLI installed on your system. In addition to it, you will need to install the AWS CLI and configure access credentials. You can follow the links in the prerequisite section to install the CLI and configure the credentials.
Run the following commands to bootstrap the application and deploy it.
cdk bootstrap
cdk deploy
When you execute the cdk deploy command, it will prompt you to confirm the IAM role/policy changes that would be applied to your account. Notice, that the terminal will display the base URL of the REST API service that you deployed. Grab this URL and keep it handy as you will use it to test the API in the next section.
Testing the deployed API
Now that the application has been deployed to the AWS account, test the API by calling the API endpoint using curl. Replace the base URL that you obtained in the previous section in the curl request.
curl -X POST \
'<base_URL>/jobs' \
--header 'Accept: */*' \
--header 'Authorization: allow' \
--header 'Content-Type: application/json' \
--data-raw '{
"jobId": "1"
}'
Notice that you have set the Authorization: allow header which acts as the token that will be validated by the authorization Lambda.
On executing the curl request, you should receive a success response similar to the one shown below.
Instead of passing the allow value in the Authorization header, try passing deny or some other value to make sure that the API returns a success response only when it receives a valid token.
curl -X POST \
'<baseURL>/jobs' \
--header 'Accept: */*' \
--header 'Authorization: deny' \
--header 'Content-Type: application/json' \
--data-raw '{
"jobId": "1"
}'
As expected, the API returns an authorized response since the token is not valid.
Automating application deployment using CircleCI
Now that you were able to deploy the CDK application manually using the command line, automate the workflow so that the infrastructure changes can be packaged and deployed automatically every time you push code to the main branch. To automate the deployments you will need to:
- Update .gitignore
- Update NPM scripts
- Add the configuration script
- Create a CircleCI project
- Set up environment variables
Update .gitignore
The code generated by the cdk init command contains a .gitignore file that ignores all .js files by default. Make sure to replace the contents of .gitignore with the following code snippet.
!jest.config.js
*.d.ts
node_modules
# CDK asset staging directory
.cdk.staging
cdk.out
Update NPM scripts
Our CircleCI deployment configuration uses NPM scripts for executing the deploy and diff commands. Add the following scripts to the root level package.json file.
// update the aws-cdk-api-auth-lambda-circle-ci/package.json file with the following scripts
{
...
"scripts": {
...
// add the ci_diff and ci_deploy scripts
"ci_diff": "cdk diff -c env=${ENV:-stg} 2>&1 | sed -r 's/\\x1B\\[([0-9]{1,2}(;[0-9]{1,2})?)?[mGK]//g' || true",
"ci_deploy": "cdk deploy -c env=${ENV:-stg} --require-approval never"
},
...
}
Add the configuration script
First, add a .circleci/config.yml script in the root of the project containing the configuration file for the CI pipeline. Add the following code snippet to the config.yml.
version: 2.1
orbs:
aws-cli: circleci/aws-cli@2.0.6
executors:
default:
docker:
- image: "cimg/node:14.18.2"
environment:
AWS_REGION: "us-west-2"
jobs:
build:
executor: "default"
steps:
- aws-cli/setup:
aws-access-key-id: AWS_ACCESS_KEY
aws-secret-access-key: AWS_ACCESS_SECRET
aws-region: AWS_REGION_NAME
- checkout
- run:
name: 'Install Authorizer lambda packages'
command: |
cd lambda/processJob && npm install
- run:
name: 'Install Process Job lambda packages'
command: |
cd lambda/processJob && npm install
- run:
name: "build"
command: |
npm install
npm run build
- run:
name: "cdk_diff"
command: |
if [ -n "$CIRCLE_PULL_REQUEST" ]; then
export ENV=stg
if [ "${CIRCLE_BRANCH}" == "develop" ]; then
export ENV=prd
fi
pr_number=${CIRCLE_PULL_REQUEST##*/}
block='```'
diff=$(echo -e "cdk diff (env=${ENV})\n${block}\n$(npm run --silent ci_diff)\n${block}")
data=$(jq -n --arg body "$diff" '{ body: $body }') # escape
curl -X POST -H 'Content-Type:application/json' \
-H 'Accept: application/vnd.github.v3+json' \
-H "Authorization: token ${GITHUB_TOKEN}" \
-d "$data" \
"https://api.github.com/repos/${CIRCLE_PROJECT_USERNAME}/${CIRCLE_PROJECT_REPONAME}/issues/${pr_number}/comments"
fi
- run:
name: "cdk_deploy"
command: |
if [ "${CIRCLE_BRANCH}" == "main" ]; then
ENV=prd npm run ci_deploy
elif [ "${CIRCLE_BRANCH}" == "develop" ]; then
ENV=stg npm run ci_deploy
fi
The CI script uses the aws-cli orb for setting AWS configuration, such as the access key and secret.
The cdk_deploy command checks the branch and accordingly deploys on the prd or stg environment. Note that the cdk_deploy command executes the ci_deploy script defined in the package.json file.
Our pipeline configuration will take care of building, packaging, and deploying the CDK stack to the specified AWS account. Commit the changes and push them to the GitHub repository.
Create a CircleCI project for the application
Next, set up the repository as a CircleCI project using the CircleCI console. On the Circle CI console, click the Projects tab and search for the GitHub repo name. Click the Set Up Project button for your project.
A dialog prompt will appear asking you want to manually add a new configuration file or use an existing one. Since you have already pushed the required configuration file to the codebase, select the Fastest option and enter the name of the branch hosting your configuration file. Click Set Up Project to continue.
Completing the setup will automatically trigger the pipeline. The pipeline would fail in its first run since you haven’t define the environment variables.
Set up environment variables
On the project page, click Project settings and go to the Environment variables tab. On the screen that appears, click Add environment variable button and add the following environment variables.
AWS_ACCESS_KEYobtained from the IAM role page in the AWS consoleAWS_ACCESS_SECRETobtained from the IAM role page in the AWS consoleAWS_REGION_NAMEto a region where you want to deploy your application
Once you add the environment variables, it should show the key values on the dashboard.
Now that the environment variables are configured, trigger the pipeline again. This time the build should succeed.
Conclusion
In this tutorial, you saw how to use AWS CDK constructs to easily deploy an application and expose its functionality using REST APIs. CDK can be used to easily plug in a Lambda-based custom authorizer and further customize the application experience by defining usage plans, API keys, and throttling limits. AWS CDK allows you to use familiar tools and programming language for your IaC code. It also lets you write testable code and integrate the infrastructure code with your existing code review workflows.
You can check out the full source code used in this tutorial on GitHub. The GitHub project can also be used as a template for you if you are trying to define a similar kind of stack.
Vivek Kumar Maskara is a Software Engineer at JP Morgan. He loves writing code, developing apps, creating websites, and writing technical blogs about his experiences. His profile and contact information can be found at maskaravivek.com.
