Using Docker authenticated pulls

Cloud

CircleCI has partnered with Docker to ensure that our users can continue to access Docker Hub without rate limits. As of November 1st 2020, with few exceptions, you should not be impacted by any rate limits when pulling images from Docker Hub through CircleCI. However, these rate limits may be implemented for CircleCI users in the future. This is why we are encouraging you and your team to add Docker Hub authentication to your CircleCI configuration and consider upgrading your Docker Hub plan, as appropriate, to prevent any impact from rate limits in the future.

Docker Hub

Docker executor

For the Docker executor, specify a username and password in the auth field of your config.yml file. To protect the password, place it in a context, or use a per-project Environment Variable.

Contexts are the more flexible option. CircleCI supports multiple contexts, which is an effective way to modularize secrets, ensuring jobs can only access what they need.

This example grants the "build" job access to Docker credentials context, docker-hub-creds, without bloating the existing build-env-vars context:

workflows:
  my-workflow:
    jobs:
      - build:
          context:
            - build-env-vars
            - docker-hub-creds

jobs:
  build:
    docker:
      - image: acme-private/private-image:321
        auth:
          username: mydockerhub-user  # can specify string literal values
          password: $DOCKERHUB_PASSWORD  # or project environment variable reference

If you have two-factor authentication set up on Docker Hub, you can simply use your personal access token for the password key instead. For example:

- image: acme-private/private-image:321
  auth:
    username: mydockerhub-user
    password: $DOCKERHUB_ACCESS_TOKEN

You can also use images from a private repository like gcr.io or quay.io. Ensure you supply the full registry/image URL for the image key, and use the appropriate username/password for the auth key. For example:

- image: quay.io/project/image:tag
  auth:
    username: $QUAY_USERNAME
    password: $QUAY_PASSWORD

Machine executor (with Docker orb)

Alternatively, you can utilize the machine executor to achieve the same result using the Docker orb:

version: 2.1
orbs:
  docker: circleci/docker@1.4.0

workflows:
  my-workflow:
    jobs:
      - machine-job:
          context:
            - build-env-vars
            - docker-hub-creds

jobs:
  machine-job:
    machine:
      image: ubuntu-2004:current
    resource_class: large
    steps:
      - docker/check:
          docker-username: DOCKERHUB_LOGIN  # DOCKER_LOGIN is the default value, if it exists, it automatically would be used.
          docker-password: DOCKERHUB_PASSWORD  # DOCKER_PASSWORD is the default value
      - docker/pull:
          images: 'circleci/node:latest'

Machine executor (with Docker CLI)

Or with the CLI:

version: 2.1
jobs:
  build:
    machine:
      image: ubuntu-2004:current
    resource_class: large
    working_directory: ~/my_app
    steps:
      # Docker is preinstalled, along with docker compose
      - checkout

      # start proprietary DB using private Docker image
      - run: |
          docker login -u $DOCKER_USER -p $DOCKER_PASS
          docker run -d --name db company/proprietary-db:1.2.3

AWS ECR

CircleCI supports pulling private images from AWS ECR.

You can pull your private images from ECR repositories in any regions. However, for the best experience, we strongly recommend you make a copy of your image in us-east-1 region, and specify that us-east-1 image for the Docker executor. CircleCI job execution infrastructure is in the us-east-1 region, so using us-east-1 images accelerates the process of spinning up your environment.

You can pull images from ECR for your jobs using a few different methods.

The best practice approach is to pull from ECR using OpenID Connect (OIDC). See the Pull an image from AWS ECR with OIDC how-to guide for full details.

Some advantages to using OIDC over using AWS credentials to pull images from ECR:

Improved security: By using OIDC authentication, storing AWS credentials directly in your CircleCI configuration or environment variables is avoided, reducing the risk of exposure.
Simplified credential management: OIDC allows CircleCI to automatically manage the authentication process, eliminating the need to manually manage and rotate AWS credentials.
Fine-grained access control: By associating an IAM role with the OIDC authentication, exact permissions granted to CircleCI for pulling ECR images can be controlled, ensuring a least-privilege approach.

Alternatively, you can use one of the following methods:

Set your AWS credentials using standard CircleCI private environment variables.
Specify your AWS credentials in .circleci/config.yml using aws_auth:

version: 2.1
jobs:
  build:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: AKIAQWERVA  # can specify string literal values
          aws_secret_access_key: $ECR_AWS_SECRET_ACCESS_KEY  # or project UI envar reference

Both options are virtually the same. However, the second option enables you to specify the variable name you want for the credentials. This can be useful where you have different AWS credentials for different infrastructures. For example, your SaaS app runs the speedier tests and deploys to staging infrastructure on every commit, while for git tag pushes, we run the complete test suite before deploying to production:

version: 2.1
jobs:
  build:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: $AWS_ACCESS_KEY_ID_STAGING
          aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_STAGING
    steps:
      - run:
          name: "Every Day Tests"
          command: "testing...."
      - run:
          name: "Deploy to Staging Infrastructure"
          command: "something something darkside.... cli"
  deploy:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: $AWS_ACCESS_KEY_ID_PRODUCTION
          aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_PRODUCTION
    steps:
      - run:
          name: "Full Test Suite"
          command: "testing...."
      - run:
          name: "Deploy to Production Infrastructure"
          command: "something something darkside.... cli"

workflows:
  main:
    jobs:
      - build:
          filters:
            tags:
              only: /^\d{4}\.\d+$/
      - deploy:
          requires:
            - build
          filters:
            branches:
              ignore: /.*/
            tags:
              only: /^\d{4}\.\d+$/

The minimum permissions required for your AWS account are as follows (you will need to substitute YOUR_ECR_REPO_ARN):

{
  "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer"
            ],
            "Effect": "Allow",
            "Resource": "<your-ECR-repo-ARN>"
        },
        {
            "Action": [
                "ecr:GetAuthorizationToken"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Using Docker authenticated pulls

On This Page

Docker Hub

Docker executor

Machine executor (with Docker orb)

Machine executor (with Docker CLI)

AWS ECR

See also

Suggest an edit to this page

Still need help?