Configuring a Python Application on CircleCI

This document describes how to configure CircleCI using a sample application written in Python.

Overview
Configuration walkthrough
Full configuration file
See also

Overview

This guide uses a sample Django application to describe configuration best practices for Python applications building on CircleCI. The application is hosted on GitHub and is building on CircleCI.

Consider forking the repository and rewriting the configuration file as you follow this guide.

Configuration walkthrough

Every CircleCI project requires a configuration file called .circleci/config.yml. Follow the steps below to create a complete config.yml file.

Specify a version

Every config.yml starts with the version key. This key is used to issue warnings about breaking changes.

version: 2

Create a build job

A run is comprised of one or more jobs. Because this run does not use workflows, it must have a build job.

Use the working_directory key to specify where a job’s steps run. By default, the value of working_directory is ~/project, where project is a literal string.

version: 2
jobs:
  build:  # required for runs that don't use workflows
    working_directory: ~/circleci-demo-python-django

Choose an executor type

The steps of a job occur in a virtual environment called an executor.

In this example, the docker executor is used to specify a custom Docker image. The first image listed becomes the job’s primary container. All commands for a job execute in this container.

version: 2
jobs:
  build:
    working_directory: ~/circleci-demo-python-django
    docker:
      - image: circleci/python:3.6.4  # primary container for the build job
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD  # context / project UI env-var reference

Note: circleci/python:3.6.4 is a convenience image provided by CircleCI. These images are extensions of official Docker images and include tools useful for CI/CD environments.

Add other services and set environment variables

Specify additional containers for services like databases. Use the environment key to set environment variables for all commands in a container.

version: 2
jobs:
  build:
    working_directory: ~/circleci-demo-python-django
    docker:
      - image: circleci/python:3.6.4 # every job must define an image for the docker executor and subsequent jobs may define a different image.
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD  # context / project UI env-var reference
        environment:
          PIPENV_VENV_IN_PROJECT: true
          DATABASE_URL: postgresql://root@localhost/circle_test?sslmode=disable
      - image: circleci/postgres:9.6.2 # an example of how to specify a service container
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD  # context / project UI env-var reference
        environment:
          POSTGRES_USER: root
          POSTGRES_DB: circle_test

Install dependencies

After choosing containers for a job, create steps to run specific commands.

Use the checkout step to check out source code. By default, source code is checked out to the path specified by working_directory.

Use the run step to execute bash commands. In this example, Pipenv is used to create a virtual environment and install Python packages.

version: 2
jobs:
  build:
    # ...
    steps:
      - checkout  # checkout source code to working directory
      - run:
          command: |  # use pipenv to install dependencies
            sudo pip install pipenv
            pipenv install

Cache dependencies

To save time between runs, consider caching dependencies or source code.

Use the save_cache step to cache certain files or directories. In this example, the virtual environment and installed packages are cached.

Use the restore_cache step to restore cached files or directories.

version: 2
jobs:
  build:
    # ...
    steps:
      - checkout
      - run: sudo chown -R circleci:circleci /usr/local/bin
      - run: sudo chown -R circleci:circleci /usr/local/lib/python3.6/site-packages
      - restore_cache:  # ensure this step occurs *before* installing dependencies
          key: deps9-{{ .Branch }}-{{ checksum "Pipfile.lock" }}
      - run:
          command: |
            sudo pip install pipenv
            pipenv install
      - save_cache:
          key: deps9-{{ .Branch }}-{{ checksum "Pipfile.lock" }}
          paths:
            - ".venv"
            - "/usr/local/bin"
            - "/usr/local/lib/python3.6/site-packages"

Note: Use the chown command to grant CircleCI access to dependency locations.

Run tests

Use the run step to run your test suite.

version: 2
jobs:
  build:
    # ...
    steps:
      # ...
      - run:
        command: |
          pipenv run python manage.py test

Upload and store test results

Use the store_test_results step to upload test results to CircleCI. These results will display in the Test Summary section of the CircleCI application.

Use the store_artifacts step to save test results as artifacts.

version: 2
jobs:
  build:
    # ...
    steps:
      # ...
      - store_test_results:
          path: test-results
      - store_artifacts:
          path: test-results
          destination: tr1

Deploy application

This Django application is not deployed anywhere. See the Flask Project Walkthrough or the Deploy document for deploy examples.

Full configuration file

version: 2 # use CircleCI 2.0
jobs: # A basic unit of work in a run
  build: # runs not using Workflows must have a `build` job as entry point
    # directory where steps are run
    working_directory: ~/circleci-demo-python-django
    docker: # run the steps with Docker
      # CircleCI Python images available at: https://hub.docker.com/r/circleci/python/
      - image: circleci/python:3.6.4
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD  # context / project UI env-var reference
        environment: # environment variables for primary container
          PIPENV_VENV_IN_PROJECT: true
          DATABASE_URL: postgresql://root@localhost/circle_test?sslmode=disable
      # CircleCI PostgreSQL images available at: https://hub.docker.com/r/circleci/postgres/
      - image: circleci/postgres:9.6.2
        auth:
          username: mydockerhub-user
          password: $DOCKERHUB_PASSWORD  # context / project UI env-var reference
        environment: # environment variables for the Postgres container.
          POSTGRES_USER: root
          POSTGRES_DB: circle_test
    steps: # steps that comprise the `build` job
      - checkout # check out source code to working directory
      - run: sudo chown -R circleci:circleci /usr/local/bin
      - run: sudo chown -R circleci:circleci /usr/local/lib/python3.6/site-packages
      - restore_cache:
      # Read about caching dependencies: https://circleci.com/docs/2.0/caching/
          key: deps9-{{ .Branch }}-{{ checksum "Pipfile.lock" }}
      - run:
          command: |
            sudo pip install pipenv
            pipenv install
      - save_cache: # cache Python dependencies using checksum of Pipfile as the cache-key
          key: deps9-{{ .Branch }}-{{ checksum "Pipfile.lock" }}
          paths:
            - "venv"
      - run:
          command: |
            pipenv run python manage.py test
      - store_test_results: # Upload test results for display in Test Summary: https://circleci.com/docs/2.0/collect-test-data/
          path: test-results
      - store_artifacts: # Upload test summary for display in Artifacts: https://circleci.com/docs/2.0/artifacts/
          path: test-results
          destination: tr1