Use CircleCI orbs to build and push Docker images to Amazon ECR

This article is an introduction to using CircleCI orbs to build and push production-grade Docker images to the Amazon Elastic Container Registry (ECR) for storage. Familiarity with the tools and technologies used in this article is preferred but not necessary. Here is a brief summary of the common terms and technologies:

  • Docker: A software platform for building applications based on containers.
  • Container: A method of virtualization that packages an application’s code, configurations, and dependencies into building blocks for consistency, efficiency, productivity, and version control.
  • Container image: A self-contained piece of software that has everything in it needed to run an application: code, tools, and resources.
  • Container image repository: A collection of named and related container images, usually providing different versions of the same application or service identified by their tags.
  • Docker image registry: is a service that stores container images and is hosted either by a third-party or as a public/private registry such as Docker Hub, AWS (ECR), GCP (GCR), Quay, etc. They simplify your development to production workflow.
  • Container orchestration: Container orchestration is all about managing the lifecycles of containers, especially in large, dynamic environments.

Let’s dive into Amazon ECR, what it is, and how CircleCI can help you get the most out of this service.

What is Amazon ECR?

Amazon ECR is a fully-managed, private Docker container registry that makes it easy for developers to store, manage, and deploy Docker container images. Amazon ECR integrates seamlessly with Amazon Elastic Container Service (Amazon ECSe) and Amazon Elastic Kubernetes Service (Amazon EKS). Amazon ECR can also be used with other cloud vendors.

Why would you want to use a private Docker registry like Amazon ECR?

  • In many circumstances, it is desirable to segregate the development, test, and production registries.
  • It’s fully managed which eliminates the need to operate your own container repositories or worry about scaling the underlying infrastructure.
  • It’s secure. A private container registry with scanning capabilities and role-based access control offers more security, governance (IAM), and efficient management. It transfers your container images over HTTPS and automatically encrypts your images at rest.
  • Running registries close to the systems running the containers cuts deployment latency and reduces exposure to network outages.

Developers need to use a registry to store images created during the application development process. Continuous integration pipelines connect building container images and pushing these artifacts into Amazon ECR. Imagine a pipeline where you push a commit that triggers a build on CircleCI that pushes a new image into Amazon ECR. Your registry can then fire off a webhook and trigger a deployment. All without a human having to manually do anything. Registries make a fully automated pipeline like this much easier. Container images placed in the registry can then be used in various phases of the development. In a few steps, using CircleCI orbs, we are going to containerize a simple Node.js application, build the image, and push it to Amazon ECR.

Setting up Amazon ECR

From the AWS management console, select IAM. This service lets you manage access to AWS resources.

Create a role/user. In this example, I called mine ci-cd-ecr, but any arbitrary name will work.

Next, we will need to programmatically access our Amazon ECR service using an AWS_ACCESS_KEY and an AWS_SECRET_KEY. Click on the newly created user and go to the Your Security Credentials page. Here you will be able to create a new access key. Store the key securely as we will be using it to access the Amazon ECR service from CircleCI. The key will be in a format similar to this:

AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

Next, create a policy from the Policies section and attach the ci-cd-ecr role created earlier. The policy gives full access to Amazon ECR. The CircleCI orb, using our newly created ci-cd-ecr role, will have full access to our Amazon ECR service, including creating image repositories if they don’t exist.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecr:*",
                "cloudtrail:LookupEvents"
            ],
            "Resource": "*"
        }
    ]
}

Generate an access key and secret key for the user and store them in a safe place. We will need them when setting up our CircleCI pipeline example.

Setting up a CircleCI pipeline using the aws-ecr orb

The code for this demo is available on GitHub here. CircleCI config file can be found in the .circleci/ folder. Using orbs, as you’ll see, we accomplish building an image and pushing it to Amazon ECR using only 18 lines of config! To use orbs, we need to use CircleCI version 2.1. It is the version that has support for orbs. The aws-ecr orb comes prepackaged with commands to:

  • Build an image
  • Tag the image (using the Git commit hash of the HEAD == CIRCLE_SHA1 )
  • Login to Amazon ECR
  • Create an Amazon ECR repo, if one doesn’t exist
  • Push an image to Amazon ECR

Here is the full config for our pipeline:

version: 2.1

orbs:
  aws-ecr: circleci/aws-ecr@6.7.0

workflows:
  build_and_push_image:
    jobs:
      - aws-ecr/build-and-push-image:
          account-url: AWS_ECR_ACCOUNT_URL
          aws-access-key-id: AWS_ACCESS_KEY_ID
          aws-secret-access-key: AWS_SECRET_ACCESS_KEY
          create-repo: true
          dockerfile: Dockerfile
          path: .
          region: AWS_REGION
          repo: circleci-ecr-orb-demo
          tag: "$CIRCLE_SHA1"

Setting up environment variables for Amazon ECR

There are several environment variables you can see in the config that will need to be set:

  • AWS_ECR_ACCOUNT_URL - The environment variable storing your Amazon ECR account URL that maps to an AWS account, e.g. {awsAccountNum}.dkr.ecr.us-west-2.amazonaws.com
  • AWS_ACCESS_KEY_ID - The AWS access key id for the ci-cd-ecr IAM role we had created earlier.
  • AWS_SECRET_ACCESS_KEY - AWS secret key for ci-cd-ecr IAM role that we had created earlier. Set this to the name of the environment variable you will set to hold this value, i.e. AWS_SECRET_ACCESS_KEY
  • AWS_REGION - AWS region where your ECR resources will be located.

Note: You don’t have to set the $CIRCLE_SHA1 because it is the default variable available in all CircleCI projects. It’s the SHA1 hash of the last commit of the current build. Using the Git commit hash gives you the ability to trace what’s in the containers. Ideally, it allows us to trace a container back to its Docker image, and then back to the Dockerfile and code contained in the image. In an automated execution environment, this will take us back to the commit that caused the Docker image to be built.

The config example above uses the $CIRCLE_SHA1 to set the tag:. Alternatively, you can use the CircleCI build number ($CIRCLE_BUILD_NUM) as a tag in your builds.

The dockerfile: command above specifies the path to your Dockerfile. Our demo repo uses the following Dockerfile:

# Set the base image to use for subsequent instructions
FROM node:alpine

# Add metadata to an image 
LABEL app="simple-node-application"
# Directive to set environmental variables key to value pair
ENV NPM_CONFIG_LOGLEVEL warn

# Set the working directory for any subsequent ADD, COPY, CMD, ENTRYPOINT, 
# or RUN instructions that follow it in the Dockerfile
WORKDIR /usr/src/app

# Copy files or folders from source to the dest path in the image's filesystem.
COPY package.json /usr/src/app/
COPY . /usr/src/app/

# Execute any commands on top of the current image as a new layer and commit the results.
RUN npm install --production

# Define the network ports that this container will listen on at runtime.
EXPOSE 3000

# Configure the container to be run as an executable.
ENTRYPOINT ["npm", "start"]

With both CircleCI and Amazon ECR well configured, you can start building images and pushing them to the repository.

Pushing Docker images to Amazon ECR in just 18 lines of config

In just 18 lines of config, we were able to build and push our Docker image to Amazon ECR. CircleCI orbs save time by importing pre-built commands, jobs, and executors into CircleCI configuration files and provide easy integration with cloud services and other tools.

Dominic Motuka is a DevOps Engineer at Andela with 4+ years of hands-on experience supporting, automating, and optimizing production-ready deployments in AWS and GCP, leveraging configuration management, CI/CD, and DevOps processes.

Read more posts by Dominic Motuka