How to build a CI/CD pipeline with Docker

The goal of every software team is to minimize the gap between conception, implementation, and deployment to maximize competitive advantage. Yet speed should not come at the expense of the application integrity or customer (user) experience. To help with this, many teams integrate a continuous integration/continuous deployment (CI/CD) pipeline into their development process. With CI/CD, you get speedy deployments protected by your test suite — nothing reaches customers without first passing tests.

In this article, you will learn how to build a CI/CD pipeline using Docker, a tool widely used to create, deploy, and run applications with containers. Docker simplifies the CI/CD process by allowing developers to package applications with all of their dependencies into a standardized unit for software development. This uniformity means that you can be assured that your application runs the same way in every environment, from development to production.

To learn the basics of Docker and CI/CD, you will Dockerize a basic Flask application, deploy it to the Azure Container Registry (ACR), and set up a CircleCI pipeline to automate the build, test, and deployment process.

Prerequisites

• Familiarity with Docker and Docker Desktop installed on your local machine. You can follow Docker’s tutorial for Windows or macOS.
• A GitHub account
• A CircleCI account
• An Azure account account with an active subscription
• Python (>3.10) installed on your workstation
• The Azure CLI installed on your workstation

Setting up the development environment

To help demonstrate the concepts in this article, we created a simple Flask application that you can clone. The sample application is an exchange rate API that gives naive rates for supported currencies. Use git to clone the sample application to your development environment.

git clone https://github.com/CIRCLECI-GWP/python-circleci-docker.git

Change into the cloned directory

cd python-circleci-docker

Create a virtual environment for your project using the following command:

python3 -m venv .venv

To activate the virtual environment, run the following command:

source .venv/bin/activate

To learn more about virtual environments, you can read the official Flask documentation.

Once your virtual environment is activated, install the project requirements using the following command:

pip3 install -r requirements.txt

Testing the code

Tests are an essential component of CI/CD. As much as you want to deploy changes as quickly as possible via automation, you also want to protect the integrity of the application. New updates to a codebase should not have negative effects on the application.

The first step in a good CI/CD process is often to run the application test suite. If all tests pass, then the updates can be built and deployed with the greater confidence that the system will work as expected. If the tests fail, the deployment process is terminated and the failing code reviewed before trying again.

The sample application uses the pytest framework for testing. You can learn more about pytest in Pytest: Getting started with automated testing for Python.

The tests are spread across two files: test_app.py and test_exchange_rate_helper.py. These tests ensure that the behavior of the application is consistent with the posited scenarios in each test. Let’s take a look at test_exchange_rate_helper.py to clarify:

import unittest
import exchange_rate_helper

class TestExchangeRateHelper(unittest.TestCase):

  def test_get_all_rates(self):
	  rates = exchange_rate_helper.getAllExchangeRates()
	  expectedNumberOfRates = len(exchange_rate_helper.getSupportedCurrencies())
	  actualNumberOfRates = len(rates)
	  self.assertEqual(expectedNumberOfRates, actualNumberOfRates)

  def test_get_unsupported_currency_throws_exception(self):
	  with self.assertRaises(ValueError):
		  rates = exchange_rate_helper.getExchangeRatesForCurrency("YUAN")

  def test_rates_for_currency_returns_expected_number_of_items(self):
	  rates = exchange_rate_helper.getExchangeRatesForCurrency("NGN")
	  expectedNumberOfRates = len(exchange_rate_helper.getSupportedCurrencies()) - 1
	  actualNumberOfRates = len(rates)
	  self.assertEqual(expectedNumberOfRates, actualNumberOfRates)

The TestExchangeRateHelper is instantiated from the base class unittest.Test, which is the smallest unit of testing. It checks for a specific response to a particular set of inputs. The unittest framework provides a base class, TestCase, that you will use to create new test cases.

The test_get_all_rates functions tests that the exchange_rate_helper module returns rates for all the supported currencies.

The test_get_unsupported_currency_throws_exception function tests the outcome when an unsupported currency is passed to the exchange_rate_helper module. When this happens, the application should throw an exception. The test verifies this via the assertRaises() function call.

Finally, the test_rates_for_currency_returns_expected_number_of_items function tests ensures that the exchange_rate_helper module returns the expected number of rates when provided with a supported currency.

To be sure everything is in order, you can run the tests written for the project using the following command.

pytest -v

Note: When you run this command for the first time, it may prompt you to install the python3-pytest package. If it does, type y and press enter to install the package.

Your test results should match the output shown below:

Next, run the development server using the following command.

flask run --port=8000

This runs the application on port 8000 on your application. Open http://localhost:8000/ in your browser to review the response from the index endpoint.

Additionally, the application has these routes:

Route	Description
/rates	Get all the rates for supported currencies
/currencies	Get all the supported currencies by the API
/rate/	Get the rates for a specified supported currency. If the provided currency is not supported, an error response is returned.

Once you’re sure everything works, stop the application by pressing ctrl + C to stop the server.

Next, deactivate virtual environment by typing deactivate. If required, you can always reactivate the virtual environment as shown here.

Dockerizing the application

For this tutorial, you will be hosting your container image on Azure Container Registry, but you can use the same concepts to deploy to Docker Hub or any other registry you prefer.

To build a container image, you’ll need to first create a Dockerfile.

We’ve already done this for the sample application, but if you’re Dockerizing a different app, create a new file named Dockerfile and ensure that its content matches this:

FROM python:alpine

WORKDIR /app

COPY requirements.txt /app

RUN pip install -r requirements.txt

COPY . /app

CMD ["python3", "app.py"]

This file uses the alpine version of the Python Docker image. Next the working directory is set to /app after which the requirements.txt file is copied into the working directory. Pip is used to install the project requirements after which the project files are copied to the working directory. The last line lets Docker know the command to run when the container starts.

Setting up Azure Container Registry (ACR)

In this article, I will show you how to set up the container registry via the CLI. If you’re interested in doing it via your Azure dashboard, you can follow the steps in this article.

Authenticating with Azure

To run commands via Azure CLI, you will need to be authenticated. Do that with the following command.

az login

This opens a new window in your browser. Provide your email address and password to complete the authentication process. In your terminal, once the authentication process is completed the subscription details would be printed out. Make a special note of the id key as it will be used when creating a service principal.

The output should look like this:

Retrieving tenants and subscriptions for the selection...

[Tenant and subscription selection]

No     Subscription name    Subscription ID                       Tenant
-----  -------------------  ------------------------------------  -----------------
[1] *  your-subscription    alphanumeric-code.                  

The default is marked with an *; the default tenant is 'Default Directory' and subscription is 'your-subscription' (alphanumeric-code).

Select a subscription and tenant (Type a number or Enter for no changes): 1

Tenant: Default Directory
Subscription: your-subsciption (alphanumeric-code)

If you haven’t already done so, create a new service principal using the following command.

az ad sp create-for-rbac --name <service_principal_name> --scopes /subscriptions/<subscription_id> --role owner

In the command, service_principal_name can be any name you choose, subscription_id is the value of the id key in the terminal output of the successful login. On successful completion, the service principal information will be printed on the terminal. The information displayed include the appId and password which will be used to log in to the container registry.

The output of the above command should look like this:

{
  "appId": "34erer1f6-t74356-475w-b5aa-bf08df45b0c0",
  "displayName": "svc-principle-name",
  "password": "qkt8Q~kbEWHIUHUnnvXNfjCeVAdxrwer4g45hAbg5",
  "tenant": "9e2f3560-1f82-41e2-80b3-0b8fbff9b070"
}

Setting up the container registry using the Azure CLI

Create a new resource group using the following command.

az group create --name FlaskRG --location eastus

Next, create a new container registry using the following command.

az acr create --resource-group FlaskRG --name flaskdockerdemo --sku Basic

NOTE: The registry name has to be unique. If you get a message such as The registry DNS name flaskdockerdemo.azurecr.io is already in use., you will need to choose a different, unique name. You can also use the Registries API to check for available names.

With a registry successfully set up, you can build the docker image, and push it to the registry.

Building the Docker image

The application contains a Dockerfile which contains instructions with which Docker can build and serve the API. When building the application, specify the registry DNS name

docker build -t flaskdockerdemo.azurecr.io/flask-rates-api:latest .

Note: Replace flaskdockerdemo with the name you chose for the registry URL and login server details.

You can run the Docker image locally using the following command.

docker run -p 8000:5000 flaskdockerdemo.azurecr.io/flask-rates-api

The application will run on port 8000.

Pushing the Docker image to ACR

Once the build process is complete, you can push the image to the container registry. Before pushing to the registry, you must be authenticated on the container registry. Do this using the following command

echo $AZURE_SP_PASSWORD | docker login flaskdockerdemo.azurecr.io -u $AZURE_SP_APP_ID --password-stdin

Replace $AZURE_SP_PASSWORD and $AZURE_SP_APP_ID in the command above with the service principal password and appId respectively.

Push to the container registry using the following command.

docker push flaskdockerdemo.azurecr.io/flask-rates-api

To confirm if the image has been deployed, you can run the following command.

az acr repository list --name flaskdockerdemo --output table

You will see this output:

Result
-----------------
flask-rates-api

Setting up the CircleCI configuration

Now that you’ve successfully set up a container registry, you have everything required to set up a CI/CD pipeline. The first step is preparing a config.yml file, which gives CircleCI step-by-step instructions.

At the root of your project, create a new folder named .circleci and in it a file named config.yml. Add the following code to the file.

version: 2.1

orbs:
  python: circleci/python@2.1.1
  azure-acr: circleci/azure-acr@0.2.1

jobs:
  pull-and-test:
    description: "Setup Django application and run tests"
    executor: python/default
    steps:
      - checkout
      - python/install-packages:
          pkg-manager: pip
      - run:
          name: "Run tests"
          command: pytest -v

workflows:
  deploy-docker-image:
    jobs:
      - pull-and-test
      - azure-acr/build-and-push-image:
          login-server-name: flaskdockerdemojeff.azurecr.io
          registry-name: flaskdockerdemojeff
          repo: flask-rates-api
          requires:
            - pull-and-test

This workflow makes use of two orbs provided by CircleCI. Orbs are shareable packages of CircleCI configuration you can use to simplify your builds. For this pipeline, the Python and Azure ACR orbs are used.

The pull-and-test job is used to pull the latest code and run the test suites. This job uses the Python orb as an executor.

Finally, the deploy-docker-image workflow is used to determine the sequence of job execution. Once the pull-and-test job is completed, the build-and-push-image job provided by the azure-acr orb is executed. This job checks out the latest code from the associated repository, builds an updated image, and pushes it to the container registry. This job requires some additional parameters namely the login server, registry name, and repository name. By default, the built images will be tagged latest.

You should swap the values of login-server-name and registry-name for your project’s values.

To run this project on your CircleCI account, you will need to push your code to a repository on your Github account. Create a new repository and then run the following commands. Remember to replace REPOSITORY_URL with the URL of the newly created repository.

git remote set-url origin <REPOSITORY_URL>
git add .
git commit -m "CI/CD Pipeline configuration"
git push origin main

Connecting the project to CircleCI

Next, connect the GitHub repository to your CircleCI account. Go to your CircleCI dashboard and select the Organization Home tab on the left panel. Click the Create Project button corresponding to the GitHub repository that contains the code.

Give your project a meaningful name and select Create Project.

CircleCI will automatically detect the configuration file, but it won’t execute the first pipeline run just yet. First, you need to set up environment variables.

To configure the environment variables, select Project Settings, the Environment Variables option from the left panel of the Project Settings page. Select Add Environment Variable. Next, type the environment variable and the value you want it to be assigned to.

Here are the environment variable you’ll need:

• AZURE_SP is the appId for your Azure Service Principal
• AZURE_SP_PASSWORD is the password for your Azure Service Principal
• AZURE_SP_TENANT is the tenant ID for your Azure Service Principal

With these variables in place, you’re ready to run the pipeline. To trigger a pipeline run, push a small, meaningless commit.

Finally, you can see that the workflow has completed successfully.

Conclusion

Setting up a CI/CD pipeline with Docker offers many benefits. By encapsulating your application and its dependencies within Docker containers, you ensure consistency across all deployment environments, improving reliability and eliminating the “it works on my machine” problem.

A robust CI/CD pipeline can safeguard your application with a reliable test suite. If any tests fail, the deployment process automatically halts, and you are promptly notified — enabling quick issue resolution without causing downtime. This automated approach minimizes the risk of errors during manual updates and allows your team to concentrate more effectively on new features and optimizations.

With Docker and CI/CD, your deployments become more manageable and reproducible, making your entire development cycle more efficient and less prone to errors. You can find all the code used in this tutorial on GitHub.