Setting up continuous integration with GitHub and CircleCI

Continuous integration (CI) involves the test automation of feature branches before they are merged to the main Git branch in a project. This ensures that a codebase does not get updated with changes that could break something. Continuous delivery (CD), on the other hand, builds upon CI by automating releases of these branches or the main branch. This allows small incremental updates that reach your users faster, in line with Agile software development methodology.

In this article, I will take you through the process of setting up a CI pipeline with GitHub and CircleCI. We will use a basic Python application to demonstrate the process, but you can follow along using your own personal project as well.

Here are the steps:

1. Create a simple Python application (with Flask)
2. Create tests for this app
3. Add the config.yml file
4. Push to GitHub
5. Configure CircleCI
6. Update your README with a badge
7. Create a PR and see CircleCI in action

Prerequisites

To follow along with the tutorial, a few things are required:

1. Python installed on your local system
2. A CircleCI account
3. A GitHub account

Creating and building a simple Python app

For simplicity, you will create a Flask application. Flask is a micro-framework for Python. Don’t worry if you’ve never used Flask before — we’ll be building the basic hello world application found in the Flask documentation.

First, create a project directory (folder) and cd into it. Type this into your terminal:

mkdir python_app && cd $_/

Next, open your favorite editor and create a hello.py file. Then, copy the following lines into that file:

from flask import Flask

app = Flask(__name__)

@app.route("/")
def hello_world():
    return "<p>Hello, World!</p>"

Virtual environments in Python

When working in Python, it is strongly advised that you use a virtual environment. This allows you to install Python packages in an abstracted environment that is not your entire local machine.

Some common ways to do this are to use virtualenv or, even better, virtualenvwrapper. We will use the module venv that comes as a part of Python3. Here is how:

• Create the virtual environment:

python -m venv .venv

You can use other names for your virtual environment, like in the next example.

python -m venv Env

• Activate this environment:

source .venv/bin/activate

If you used another name for the environment, replace .venv with that name. You will notice the environment name just before the shell prompt, which tells you that the virtual environment is active. Any Python package that is installed will be installed within this environment. To deactivate this environment simply run:

deactivate

Running the app

Now it’s time to create a requirements.txt file in your editor. Add the word Flask to the file and save it.

Flask

Then, within the virtual environment, install the package by running:

pip install -r requirements.txt

Note: If you deactivated your virtual environment prior to this step, you will first need to run source .venv/bin/activate.

The final command to run this application is:

flask --app hello run

You can see the application running on your browser at http://localhost:5000/.

Creating tests for your application

Software testing is an important skill for any developer, ensuring that your application works as intended and helping to catch bugs early. Writing tests can save you time and headaches down the line.

In your editor, create a tests.py file and paste these lines into it:

from hello import app

with app.test_client() as c:
    response = c.get("/")
    assert response.data == b"<p>Hello, World!</p>"
    assert response.status_code == 200

This test verifies that your Flask application’s home route returns the correct HTML content and status code.

Learn more about testing Flask with Pytest.

Now you can run your test. Open Terminal and run:

python tests.py

Nothing should be returned to your terminal. Why? This is because you structured your Flask app so that it does not output to the terminal on a pass. No terminal output means that your tests are passing. If you want to see more detailed output, you can add print statements or use a testing framework like unittest for more comprehensive test reporting.

Creating a CircleCI config file

To automate your testing process, you’ll need to set up a CircleCI configuration file. This configuration will define how your tests are run and how your application is built and tested.

Create a .circleci folder and inside of that create a config.yml file. Then, copy these lines into it:

version: 2.1
orbs:
  python: circleci/python@2.1.1
jobs:
  build_and_test: # this can be any name you choose
    executor: python/default # use the default executor defined within the orb
    steps:
      - checkout # checkout source code
      - python/install-packages:
          pkg-manager: pip
      - run:
          name: Run tests
          command: python -m pytest
workflows:
  build_test_deploy:
    jobs:
      - build_and_test

This file configures CircleCI to use the Python orb, which simplifies setting up the Python environment and running your tests. The build_and_test job checks out your source code, installs the necessary packages, and runs your tests using pytest. The workflows section defines a workflow called build_test_deploy that runs the build_and_test job.

Learn more about this config file.

Pushing to GitHub

Following the DevOps you would typically initialize Git at the beginning of your project and make frequent, small commits. However, since this tutorial focuses on integrating CircleCI with GitHub, I intentionally delayed this step until now.

This is the current code structure:

.
├── .circleci
│   └── config.yml
├── hello.py
├── requirements.txt
└── tests.py

Open your editor and create a .gitignore file in the working directory. You will use this file to state the files and folders that you do not want to commit to Git. Copy these lines into that file:

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]

# Environments
.env
.venv
env/
venv/
ENV/

Commit your code by running:

git add .

git commit -m "initial commit"

If you do not already have a GitHub account, go to GitHub’s homepage and create one. If you are new to GitHub, you may want to read our tutorial on pushing a project to GitHub.

Next, create a repository by navigating to github.com/new.

Use the name python_app. Leave the rest of the default settings as-is and click Create repository.

After creating your new repository, you will be prompted about your repo.

Click …push an existing repository, then run:

git remote add origin git@github.com:NdagiStanley/python_app.git
git branch -M main
git push -u origin main

Configuring CircleCI

Now that the repo is on GitHub, you can finalize the CI pipeline by configuring CircleCI. Head on over to the CircleCI signup page. Sign up with your email address and connect your GitHub (or Bitbucket or GitLab) in the process.

Indicate whether you are joining a team on CircleCI or setting things up for the first time.

Enter your email and password. Click Continue to proceed.

Fill in a few onboarding questions by selecting options. I’ve gone with generic ones for now.

Click Let’s Go to continue.

The next screen allows you to integrate with your version control system (VCS).

Select GitHub. Name your organization, then click Create Organization.

Note: If you are part of an existing CircleCI organization, you’d click the link at the bottom instead.

For GitHub, you’ll install the CircleCI GitHub app. Select the GitHub organization you want to install it to.

You can specify exactly which repositories CircleCI can access, or opt to give CircleCI access to all repositories in your GitHub organization.

Click Install and Authorize.

In this last step, name your project. Notice the text underneath the field confirming presence of the .circleci/config.yml file.

Click Create Project

In the case where your project doesn’t have the .circleci/config.yml file or it’s not being recognized, review the next section of this tutorial.

With that, your project on CircleCI is ready.

CircleCI setup without config.yml

In the case where your project doesn’t have the .circleci/config.yml file or it’s not being recognized, you’ll get to a screen where CircleCI will analyze the project with the intention of generating a config file you can download and commit to your project.

You have the option to:

• Download Config File to save the generated config file locally
• Use Existing Config in case you forgot to commit the .circleci/config.yml and push it
• Commit and Run so that CircleCI will add the file via your version control system (GitHub in this case)

Select Use Existing Config, then click Start Building to trigger a CircleCI pipeline.

With that, your project on CircleCI is ready.

Initializing the CircleCI pipeline

Before you proceed to the cycle of having the automatic CI pipeline being triggered when code is updated on GitHub, remember to verify your email.

Once you are logged in, make sure that your personal GitHub account is active. Just click the drop down menu (top left) and select your GitHub username. Then, click Add Projects. The most recent project, python_app, is listed there.

Click Set up Project at the right side of the row that includes your project.

On the redirected page, you will notice three options (Fastest, Faster and Fast), with Fastest selected as the default. We’ll use this default option. Take note of the descriptions of the other two.

Enter main in the input field for the GitHub branch (Notice the text underneath the field confirming presence of the .circleci/config.yml file) and click Set Up Project.

Within no time, the build passes. Success!

Now that your project is set up, it’s worth getting familiar with the optional settings available to you. Let’s look at a relevant example.

In the top right corner, click Project Settings (button with a cog icon). Then click Advanced on the left.

Find the Only build pull requests card (scroll down if you have to). This option is turned off by default. This means that every push to GitHub will run on CircleCI, including PRs. Feel free to toggle this if necessary. Sometimes, in a large team setting, this setting can help moderate use of build minutes.

Adding a CI status badge to GitHub

A status badge is a great way to show the build status of your project right on your GitHub repository. It provides a visual indicator of whether your latest build passed or failed, which is helpful for both you and your collaborators.

On your local machine, check out to another Git branch by running:

git checkout -b add_readme

Open your editor and create a README.md file. Copy and paste the following lines into this file: (Replace the username: NdagiStanley accordingly)

# PYTHON APPLICATION

This Python application repo was created to showcase the integration between GitHub and CircleCI.
(https://circleci.com/gh/NdagiStanley/python_app.svg?style=svg)](https://circleci.com/gh/NdagiStanley/python_app)

This adds a title, a description and a status badge.

Now, run the following commands:

git add .
git commit -m "Add README"
git push -u origin add_readme

Your https://github.com/<< username >>/python_app repo has a new branch: add_readme. Click Compare and pull request.

Triggering a build via a pull request

After adding the status badge, the next step is to create a pull request (PR) to merge your changes into the main branch. This will allow us to trigger a build and ensure everything is set up correctly.

Here is how I set up my PR. The PR’s title is autogenerated as the commit message. This is optional.

#### What does this PR do?

Adds README.md file and a CircleCI badge in it.

Click Create pull request and in no time, you have a successful build!

Note that I clicked on Show all checks to reveal that the successful check is from CircleCI. Even the browser’s tab favicon shows a green tick for the successful run.

If you click Details, below Hide all checks, this will redirect you to the build on CircleCI:

Note that the favicon here is green, showing that the build is successful. At the top, click python_app. You will be redirected to the builds for this project.

Conclusion

There you have it! You have integrated GitHub with CircleCI.

In summary, you set up a Python application and created tests for it. You then created a CircleCI config file and pushed the codebase to GitHub. Finally, you connected the GitHub repository you created to CircleCI.

By adding continuous integration to your workflow, you benefit from automated testing, ensuring that your code is always in a deployable state. This helps catch bugs early, improves code quality, and allows for faster, more reliable updates.

If you followed through successfully, you can now set up your own project in GitHub and configure CI builds on CircleCI. As a next step, consider adding automated deployments for your Flask application to complete your CI/CD pipeline and fully automate your development workflow.