Setting Up Docker Hub Pull Through Mirror

5 days ago • 5 min read

Server v2.x Server Admin

ここでは、Docker Hub のレジストリのミラーを設定する方法について説明します。

概要
Set Up a Pull Through Cache Registry
Configure Nomad Clients to use the Pull Through Cache Registry (run for each Nomad client)
Configure VM Service to let Machine/Remote Docker VMs use the Pull Through Cache Registry
Testing your Setup
- Use Private Images without Explicit Authentication
- Check Logs on the Cache Registry
設定を元に戻すには
- Disarm Nomad Clients
- Disarm VM Service
Resources

概要

2020年11月1日から、Docker Hub で匿名ユーザーに対する発信元 IP に基づいたプル回数制限が導入されます。運用中のサービスでの混乱を避けるために、Docker Hub から認証付きでプル操作を行うようにすることをお勧めします。認証は設定ファイルで行えます(Docker の認証付きプルの使用を参照)。

あるいは、Docker Hub アカウントの資格情報をあらかじめ構成した Docker Hub のレジストリのミラーをセットアップするという方法もあります。レジストリのミラーを使用すると、場合によっては多数の設定ファイルを変更するよりも簡単に対応できます。また、Docker Hub への通信が減るため、パフォーマンスがさらに向上する可能性もあります。

Nomad クライアントがレジストリのミラーを使用するように設定する (Nomad クライアント「ごと」に実行)

The Services machine does not need to be configured to use the mirror. The Services machine does not pull Docker images regularly (it pulls images only upon initial setup and upgrade processes), and most images are pulled from Replicated. Hence it has few chances to hit the rate limit.

Set Up a Pull Through Cache Registry

This section covers setting up a pull through cache registry, which works as a mirror and reverse proxy for Docker Hub. Two types of pull through cache registry are presented: The elementary and easier-to-setup version using HTTP, and the more secure option using HTTPS. Either option is functional for most cases, but you might need the more secure option in certain cases.

Prerequisites: Set Up Docker Hub Account

Docker Hub アカウントを用意 します。これは、レジストリのミラーから Docker Hub にアクセスする際に使用します。

We recommend that you set up a dedicated account. Resources that the account has access to will also become accessible by all end users of your CircleCI server installation. This is especially important if the account has private images registered on Docker Hub, as these private images will become accessible by all users of your CircleCI server installation.

有料アカウントを使用すると、回数制限をさらに緩めることができます。 https://www.docker.com/pricing を参照してください。

レジストリのミラー(Docker Hub のプロキシ) をセットアップする

独立した Linux サーバーを用意 し、Docker をインストールします。

We set up this registry as an independent server (i.e. outside of CircleCI boxes) to avoid load on this cache registry affecting other CircleCI server services.

例として、このサーバーの IP アドレスを 192.0.2.1 とします。 This URL will be needed later to arm Nomad clients and the VM Service.

以下のコマンドを実行して、キャッシュサーバーを起動 します。

DOCKER_HUB_USERNAME は Docker Hub アカウントのユーザー名に、DOCKER_HUB_ACCESS_TOKEN は https://hub.docker.com/settings/security で取得したアクセストークンに置き換えてください。

sudo docker run -d -p 80:5000 --restart=always --name=through-cache -e REGISTRY_PROXY_REMOTEURL="https://registry-1.docker.io" -e REGISTRY_PROXY_USERNAME=DOCKER_HUB_USERNAME -e REGISTRY_PROXY_PASSWORD=DOCKER_HUB_ACCESS_TOKEN registry

TCP ポート 80 (HTTP) が開いていてアクセス可能であることを確認します。 HTTP) is open and accessible. For better security, we recommend that you only open the port to Nomad clients and VMs for machine executors and remote docker engines.

たとえば AWS では、セキュリティグループと OS レベルのファイアウォールの両方が HTTP 経由で Nomad クライアントからの接続を受け付けるように正しく設定されていることを確認する必要があります。

Set Up a Secure Pull Through Cache Registry (HTTPS Proxy for Docker Hub)

Under certain circumstances (especially when you are enabling BuildKit on machine executors and remote Docker engines), your pull through cache registry needs to accept connections over HTTPS. These instructions cover setting up a pull through cache that listens for HTTPS connections.

sudo systemctl restart docker.service を実行して、変更を適用します。

We set up this registry as an independent server (i.e. outside of CircleCI boxes) to avoid load on this cache registry affecting other CircleCI server services.

The server needs to be accessible by its hostname. Namely, you must configure DNS accordingly so that the hostname (assume your-mirror.example.com) resolves to the IP address of the server correctly. In this guide we assume that the URL for the registry we are setting up is https://your-mirror.example.com (be aware that the URL scheme is https, not http). In this guide we assume that the URL for the registry we are setting up is https://your-mirror.example.com (be aware that the URL scheme is http*s*, not http).

Obtain a TLS certificate for your-mirror.example.com and put the certificate and the private key into /root/tls of the server.

This guide assumes you are obtaining a certificate issued by a well-known CA, not a self-signed certificate. Use of self-signed certificates has NOT been tested.

以下のコマンドを実行して、キャッシュサーバーを起動 します。

https://docs.docker.com/registry/configuration/ (公式 Docker Registry の構成方法、英語)

Replace fullchain.pem and privkey.pem with the filenames of your certificate and private key, respectively. The certificate specified there must be correctly chained enough for tracing the certification path from the leaf to the root.

sudo bash -c 'cat <<< $(jq ".¥"registry-mirrors¥" = [¥"http://192.0.2.1¥"]" /etc/docker/daemon.json) > /etc/docker/daemon.json'

Finally, make sure that the TCP port 443 (i.e. HTTPS) is open and accessible. For better security, we recommend you to open the port only to Nomad clients and VMs for machine executors and remote docker engines.

On AWS, for example, you will need to make sure that both Security Groups and the firewall at the OS level are configured correctly to accept connections from Nomad clients and VMs for machine/setup_remote_docker jobs over HTTPS.

Plan for Renewal of TLS Certificates

You will need to renew TLS certificates periodically. These are the steps required to renew certificates.

Update the certificate and the private key in /root/tls.
Run docker restart through-cache-secure.

Technically, this can be automated. For example, if you are using Let’s Encrypt for your certificates, you can setup a cron task that executes certbot renew and the steps above.

Configure Nomad Clients to use the Pull Through Cache Registry (run for each Nomad client)

以下のコマンドを実行して、Docker デーモンの registry-mirrors オプションを指定 します。

Replace http://192.0.2.1.or.https.your-mirror.example.com with the URL of your pull through cache registry accordingly.
```
https://docs.docker.com/registry/recipes/mirror/ (レジストリのミラー
の構成方法、英語)
```
Docker デーモンをリロード して、設定を適用します。

sudo systemctl restart docker.service

Configure VM Service to let Machine/Remote Docker VMs use the Pull Through Cache Registry

Nomad クライアント「ごと」に以下の手順を実行します。

Run the command below to create a directory for your customization files.

sudo mkdir -p /etc/circleconfig/vm-service

Populate a customization script to be loaded by vm-service. Add the script below to /etc/circleconfig/vm-service/customizations.

Replace http://192.0.2.1.or.https.your-mirror.example.com in DOCKER_MIRROR_HOSTNAME variable with the URL of your pull through cache registry accordingly.

This customization is only available in 2.19.0 version and later.

export JAVA_OPTS='-cp /resources:/service/app.jar'
export DOCKER_MIRROR_HOSTNAME="http://192.0.2.1.or.https.your-mirror.example.com"

mkdir -p /resources/ec2
cat > /resources/ec2/linux_cloud_init.yaml << EOD
#cloud-config
system_info:
  default_user:
    name: "%1\$s"
ssh_authorized_keys:
  - "%2\$s"
runcmd:
  - bash -c 'if [ ! -f /etc/docker/daemon.json ]; then mkdir -p /etc/docker; echo "{}" > /etc/docker/daemon.json; fi'
  - bash -c 'cat <<< \$(jq ".\"registry-mirrors\" = [\"$DOCKER_MIRROR_HOSTNAME\"]" /etc/docker/daemon.json) > /etc/docker/daemon.json'
  - systemctl restart docker.service
EOD

Restart VM Service to apply the customization.

sudo docker restart vm-service

Testing your Setup

Use Private Images without Explicit Authentication

If the Docker ID for the cache registry has a private image, the private image should be accessible without explicit end-user authentication.

Below is a sample config to test the access (assume that the cache registry uses Docker ID yourmachineaccount, and there is a private image yourmachineaccount/private-image-with-docker-client):

version: 2

jobs:
  remote-docker:
    docker:
      - image: yourmachineaccount/private-image-with-docker-client # A copy of library/docker
    steps:
      - setup_remote_docker
      - run: docker pull yourmachineaccount/private-image-with-docker-client

  machine:
    machine: true
    steps:
      - run: docker pull yourmachineaccount/private-image-with-docker-client

workflows:
  version: 2

  run:
    jobs:
      - remote-docker
      - machine

Check Logs on the Cache Registry

By running sudo docker logs through-cache (or sudo docker logs through-cache-secure if you have set up a secure registry) you can see log outputs from your cache registry. If it is operational, there should be messages that the registry is responding to the requests for manifests and blobs with HTTP status code 200.

設定を元に戻すには

Disarm Nomad Clients

Follow the steps below on each Nomad client.

/etc/docker/daemon.json の registry-mirrors オプションを削除 します。

sudo bash -c 'cat <<< $(jq "del(.\"registry-mirrors\")" /etc/docker/daemon.json) > /etc/docker/daemon.json'

Run sudo systemctl restart docker.service to apply the change.

Disarm VM Service

Nomad クライアント「ごと」に以下の手順を実行します。

Void the JAVA_OPTS environment variable by running the command below.

echo 'unset JAVA_OPTS' | sudo tee -a /etc/circleconfig/vm-service/customizations
Run sudo docker restart vm-service to apply the change.

Resources

ドキュメントの改善にご協力ください

このガイドは、CircleCI の他のドキュメントと同様にオープンソースで、GitHub で使用できます。ご協力いただき、ありがとうございます。

このページの編集をご提案ください (最初に「コントリビューションガイド」をご覧ください)。
ドキュメントの問題点を報告する、またはフィードバックやコメントを送信するには、GitHub で issue を作成してください。
CircleCI は、ユーザーの皆様の弊社プラットフォームにおけるエクスペリエンスを向上させる方法を常に模索しています。 If you would like to share feedback, please join our research community.

CircleCI Documentation by CircleCI is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.