Using Docker authenticated pulls
This document describes how to authenticate with your Docker registry provider to pull images. Authenticated pulls allow access to private Docker images. Depending on your registry provider, they may also grant higher rate limits.
CircleCI has partnered with Docker so that our users can continue to access Docker Hub without rate limits. Since November 1st 2020, Docker Hub rate limits do not affect your image pulls through CircleCI, except in rare cases. Docker may apply these rate limits to CircleCI users in the future. We encourage you to add Docker Hub authentication to your CircleCI configuration. Consider upgrading your Docker Hub plan to prevent any impact from rate limits.
Docker Hub
Docker executor
For the Docker executor, specify a username and password in the auth field of your config.yml file. To protect the password, place it in a context, or use a per-project Environment Variable.
| Contexts are the more flexible option. CircleCI supports multiple contexts. Use them to split up secrets so that jobs access only what they need. |
This example grants the "build" job access to Docker credentials context, docker-hub-creds, without bloating the existing build-env-vars context:
workflows:
my-workflow:
jobs:
- build:
context:
- build-env-vars
- docker-hub-creds
jobs:
build:
docker:
- image: acme-private/private-image:321
auth:
username: mydockerhub-user # can specify string literal values
password: $DOCKERHUB_PASSWORD # or project environment variable reference
If you have two-factor authentication set up on Docker Hub, you can use your personal access token for the password key instead.
For example:
- image: acme-private/private-image:321
auth:
username: mydockerhub-user
password: $DOCKERHUB_ACCESS_TOKEN
You can also use images from a private repository like gcr.io or quay.io. Supply the full registry/image URL for the image key. Use the appropriate
username/password for the auth key. For example:
- image: quay.io/project/image:tag
auth:
username: $QUAY_USERNAME
password: $QUAY_PASSWORD
Machine executor (with Docker orb)
You can also use the machine executor with the Docker orb to achieve the same result:
version: 2.1
orbs:
docker: circleci/docker@1.4.0
workflows:
my-workflow:
jobs:
- machine-job:
context:
- build-env-vars
- docker-hub-creds
jobs:
machine-job:
machine:
image: ubuntu-2404:current
resource_class: large
steps:
- docker/check:
docker-username: DOCKERHUB_LOGIN # DOCKER_LOGIN is the default value, if it exists, it automatically would be used.
docker-password: DOCKERHUB_PASSWORD # DOCKER_PASSWORD is the default value
- docker/pull:
images: 'circleci/node:latest'
Machine executor (with Docker CLI)
Or with the CLI:
version: 2.1
jobs:
build:
machine:
image: ubuntu-2404:current
resource_class: large
working_directory: ~/my_app
steps:
# Docker is preinstalled, along with docker compose
- checkout
# start proprietary DB using private Docker image
- run: |
docker login -u $DOCKER_USER -p $DOCKER_PASS
docker run -d --name db company/proprietary-db:1.2.3
AWS ECR
CircleCI supports pulling private images from AWS ECR.
You can pull private images from ECR repositories in any AWS region. CircleCI runs jobs in more than one region. Pulls are fastest when the image is in the region where your job runs. If you replicate your images across regions, set prefer_same_region to true. CircleCI then attempts to pull from the nearest copy.
|
Set prefer_same_region to true on an image to have CircleCI attempt to pull it from the same region as the compute that runs your job. The same-region pull is best effort. If it fails, CircleCI falls back to pulling from the image’s original region.
CircleCI does not replicate your images. To make a same-region copy available, replicate your images to other regions yourself. For example, use Amazon ECR private image replication.
jobs:
build:
docker:
- image: <account>.dkr.ecr.us-east-1.amazonaws.com/<repo>:<tag>
aws_auth:
oidc_role_arn: arn:aws:iam::<account>:role/<role>
prefer_same_region: true
You can pull images from ECR for your jobs using more than one method.
The best practice approach is to pull from ECR using OpenID Connect (OIDC). See the Pull an Image From AWS ECR With OIDC how-to guide for full details.
OIDC has advantages over AWS credentials for pulling images from ECR:
-
Improved security: OIDC authentication removes the need to store AWS credentials in your CircleCI configuration or environment variables, which reduces the risk of exposure.
-
Simplified credential management: CircleCI manages the authentication process for you. You do not need to manage and rotate AWS credentials.
-
Fine-grained access control: Associate an IAM role with the OIDC authentication to control the exact permissions CircleCI has for pulling ECR images. This ensures a least-privilege approach.
You can also use one of the following methods:
-
Set your AWS credentials using standard CircleCI private environment variables.
-
Specify your AWS credentials in
.circleci/config.ymlusingaws_auth:
version: 2.1
jobs:
build:
docker:
- image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
aws_auth:
aws_access_key_id: AKIAQWERVA # can specify string literal values
aws_secret_access_key: $ECR_AWS_SECRET_ACCESS_KEY # or project UI envar reference
Both options are virtually the same. But the second option lets you choose the variable name for the credentials. This helps when you have different AWS credentials for different infrastructures.
For example, your SaaS app runs the speedier tests and deploys to staging infrastructure on every commit. For git tag pushes, we run the complete test suite before deploying to production:
version: 2.1
jobs:
build:
docker:
- image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
aws_auth:
aws_access_key_id: $AWS_ACCESS_KEY_ID_STAGING
aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_STAGING
steps:
- run:
name: "Every Day Tests"
command: "testing...."
- run:
name: "Deploy to Staging Infrastructure"
command: "something something darkside.... cli"
deploy:
docker:
- image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
aws_auth:
aws_access_key_id: $AWS_ACCESS_KEY_ID_PRODUCTION
aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_PRODUCTION
steps:
- run:
name: "Full Test Suite"
command: "testing...."
- run:
name: "Deploy to Production Infrastructure"
command: "something something darkside.... cli"
workflows:
main:
jobs:
- build:
filters: pipeline.git.branch or (pipeline.git.tag matches /^\d{4}\.\d+$/) # run the build job on every branch, and on tags matching the pattern
- deploy:
requires:
- build
filters: pipeline.git.tag matches /^\d{4}\.\d+$/ # only run the deploy job on tags matching the pattern
Your AWS account needs at least the following permissions. Substitute your ECR repository ARN for <your-ECR-repo-ARN>:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ecr:BatchGetImage",
"ecr:GetDownloadUrlForLayer"
],
"Effect": "Allow",
"Resource": "<your-ECR-repo-ARN>"
},
{
"Action": [
"ecr:GetAuthorizationToken"
],
"Effect": "Allow",
"Resource": "*"
}
]
}
GCP Artifact Registry
CircleCI supports pulling private images from Google Cloud Artifact Registry (GAR) and Google Container Registry (GCR).
The best practice approach is to pull from GAR/GCR using OpenID Connect (OIDC). See the Pull an Image From GCP Artifact Registry With OIDC how-to guide for full details.
OIDC has advantages over GCP service account keys:
-
Improved security: OIDC authentication removes the need to store GCP service account keys in your CircleCI configuration or environment variables, reducing the risk of exposure.
-
Simplified credential management: CircleCI manages the authentication process for you. You do not need to manage and rotate GCP service account keys.
-
Fine-grained access control: Associate a service account with the OIDC authentication to control the exact permissions CircleCI has for pulling images. This ensures a least-privilege approach.
You can also authenticate using a GCP service account key stored as a CircleCI environment variable. Use it to configure Docker credentials before pulling the image in your job steps.
| You do not need to configure authentication for public images hosted on GAR or GCR. CircleCI pulls them without credentials. |