Documentation structure for LLMs (llms.txt)

Using Docker authenticated pulls

Cloud Server

This document describes how to authenticate with your Docker registry provider to pull images. Authenticated pulls allow access to private Docker images. Depending on your registry provider, they may also grant higher rate limits.

CircleCI has partnered with Docker so that our users can continue to access Docker Hub without rate limits. Since November 1st 2020, Docker Hub rate limits do not affect your image pulls through CircleCI, except in rare cases. Docker may apply these rate limits to CircleCI users in the future. We encourage you to add Docker Hub authentication to your CircleCI configuration. Consider upgrading your Docker Hub plan to prevent any impact from rate limits.

Docker Hub

Docker executor

For the Docker executor, specify a username and password in the auth field of your config.yml file. To protect the password, place it in a context, or use a per-project Environment Variable.

Contexts are the more flexible option. CircleCI supports multiple contexts. Use them to split up secrets so that jobs access only what they need.

This example grants the "build" job access to Docker credentials context, docker-hub-creds, without bloating the existing build-env-vars context:

workflows:
  my-workflow:
    jobs:
      - build:
          context:
            - build-env-vars
            - docker-hub-creds

jobs:
  build:
    docker:
      - image: acme-private/private-image:321
        auth:
          username: mydockerhub-user  # can specify string literal values
          password: $DOCKERHUB_PASSWORD  # or project environment variable reference

If you have two-factor authentication set up on Docker Hub, you can use your personal access token for the password key instead. For example:

- image: acme-private/private-image:321
  auth:
    username: mydockerhub-user
    password: $DOCKERHUB_ACCESS_TOKEN

You can also use images from a private repository like gcr.io or quay.io. Supply the full registry/image URL for the image key. Use the appropriate username/password for the auth key. For example:

- image: quay.io/project/image:tag
  auth:
    username: $QUAY_USERNAME
    password: $QUAY_PASSWORD

Machine executor (with Docker orb)

You can also use the machine executor with the Docker orb to achieve the same result:

version: 2.1
orbs:
  docker: circleci/docker@1.4.0

workflows:
  my-workflow:
    jobs:
      - machine-job:
          context:
            - build-env-vars
            - docker-hub-creds

jobs:
  machine-job:
    machine:
      image: ubuntu-2404:current
    resource_class: large
    steps:
      - docker/check:
          docker-username: DOCKERHUB_LOGIN  # DOCKER_LOGIN is the default value, if it exists, it automatically would be used.
          docker-password: DOCKERHUB_PASSWORD  # DOCKER_PASSWORD is the default value
      - docker/pull:
          images: 'circleci/node:latest'

Machine executor (with Docker CLI)

Or with the CLI:

version: 2.1
jobs:
  build:
    machine:
      image: ubuntu-2404:current
    resource_class: large
    working_directory: ~/my_app
    steps:
      # Docker is preinstalled, along with docker compose
      - checkout

      # start proprietary DB using private Docker image
      - run: |
          docker login -u $DOCKER_USER -p $DOCKER_PASS
          docker run -d --name db company/proprietary-db:1.2.3

AWS ECR

CircleCI supports pulling private images from AWS ECR.

You can pull private images from ECR repositories in any AWS region. CircleCI runs jobs in more than one region. Pulls are fastest when the image is in the region where your job runs. If you replicate your images across regions, set prefer_same_region to true. CircleCI then attempts to pull from the nearest copy.

Set prefer_same_region to true on an image to have CircleCI attempt to pull it from the same region as the compute that runs your job. The same-region pull is best effort. If it fails, CircleCI falls back to pulling from the image’s original region.

CircleCI does not replicate your images. To make a same-region copy available, replicate your images to other regions yourself. For example, use Amazon ECR private image replication.

jobs:
  build:
    docker:
      - image: <account>.dkr.ecr.us-east-1.amazonaws.com/<repo>:<tag>
        aws_auth:
          oidc_role_arn: arn:aws:iam::<account>:role/<role>
        prefer_same_region: true

You can pull images from ECR for your jobs using more than one method.

The best practice approach is to pull from ECR using OpenID Connect (OIDC). See the Pull an Image From AWS ECR With OIDC how-to guide for full details.

OIDC has advantages over AWS credentials for pulling images from ECR:

  • Improved security: OIDC authentication removes the need to store AWS credentials in your CircleCI configuration or environment variables, which reduces the risk of exposure.

  • Simplified credential management: CircleCI manages the authentication process for you. You do not need to manage and rotate AWS credentials.

  • Fine-grained access control: Associate an IAM role with the OIDC authentication to control the exact permissions CircleCI has for pulling ECR images. This ensures a least-privilege approach.

You can also use one of the following methods:

  1. Set your AWS credentials using standard CircleCI private environment variables.

  2. Specify your AWS credentials in .circleci/config.yml using aws_auth:

version: 2.1
jobs:
  build:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: AKIAQWERVA  # can specify string literal values
          aws_secret_access_key: $ECR_AWS_SECRET_ACCESS_KEY  # or project UI envar reference

Both options are virtually the same. But the second option lets you choose the variable name for the credentials. This helps when you have different AWS credentials for different infrastructures.

For example, your SaaS app runs the speedier tests and deploys to staging infrastructure on every commit. For git tag pushes, we run the complete test suite before deploying to production:

version: 2.1
jobs:
  build:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: $AWS_ACCESS_KEY_ID_STAGING
          aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_STAGING
    steps:
      - run:
          name: "Every Day Tests"
          command: "testing...."
      - run:
          name: "Deploy to Staging Infrastructure"
          command: "something something darkside.... cli"
  deploy:
    docker:
      - image: account-id.dkr.ecr.us-east-1.amazonaws.com/org/repo:0.1
        aws_auth:
          aws_access_key_id: $AWS_ACCESS_KEY_ID_PRODUCTION
          aws_secret_access_key: $AWS_SECRET_ACCESS_KEY_PRODUCTION
    steps:
      - run:
          name: "Full Test Suite"
          command: "testing...."
      - run:
          name: "Deploy to Production Infrastructure"
          command: "something something darkside.... cli"

workflows:
  main:
    jobs:
      - build:
          filters: pipeline.git.branch or (pipeline.git.tag matches /^\d{4}\.\d+$/) # run the build job on every branch, and on tags matching the pattern
      - deploy:
          requires:
            - build
          filters: pipeline.git.tag matches /^\d{4}\.\d+$/ # only run the deploy job on tags matching the pattern

Your AWS account needs at least the following permissions. Substitute your ECR repository ARN for <your-ECR-repo-ARN>:

{
  "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ecr:BatchGetImage",
                "ecr:GetDownloadUrlForLayer"
            ],
            "Effect": "Allow",
            "Resource": "<your-ECR-repo-ARN>"
        },
        {
            "Action": [
                "ecr:GetAuthorizationToken"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

GCP Artifact Registry

CircleCI supports pulling private images from Google Cloud Artifact Registry (GAR) and Google Container Registry (GCR).

The best practice approach is to pull from GAR/GCR using OpenID Connect (OIDC). See the Pull an Image From GCP Artifact Registry With OIDC how-to guide for full details.

OIDC has advantages over GCP service account keys:

  • Improved security: OIDC authentication removes the need to store GCP service account keys in your CircleCI configuration or environment variables, reducing the risk of exposure.

  • Simplified credential management: CircleCI manages the authentication process for you. You do not need to manage and rotate GCP service account keys.

  • Fine-grained access control: Associate a service account with the OIDC authentication to control the exact permissions CircleCI has for pulling images. This ensures a least-privilege approach.

You can also authenticate using a GCP service account key stored as a CircleCI environment variable. Use it to configure Docker credentials before pulling the image in your job steps.

You do not need to configure authentication for public images hosted on GAR or GCR. CircleCI pulls them without credentials.