Writing tests in any programming language can be difficult, but it does not have to be. Using Flask with Pytest makes it easy to write and run custom tests on Python applications. Flask is a lightweight micro-web framework written in Python. It comes pre-installed with the core features only so that you can pick customizations depending on the requirements of your project. Pytest is a Python testing framework designed to help developers write better systems and release them with confidence.

In this tutorial, I will show you how you can easily write and run tests using Flask and Pytest. As a bonus, I will show you how ot integrate a CI/CD pipeline to run tests on a Flask app using CircleCI.

Prerequisites

To get the most from this tutorial, you will need:

  • Basic understanding of the Python programming language
  • Understanding of Flask framework
  • Basic understanding of testing

You will also need to have these installations and setups:

  1. Python - version >= 3.5 installed in your machine
  2. A GitHub account; you can create one here
  3. A CircleCI account; you can create one here
  4. The project repository from GitHub; clone it here

Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.

Once the project is cloned, you also need to install the dependencies using the command pip install -r requirements.txt from the root folder of the project.

Why use Flask?

Many development teams use Flask because it does not make decisions about which database to use or what the default parser is. All that can be changed with modifications, which makes it extensible and flexible for emerging development needs.

In this project tutorial, you will use a simple book-retrieval API built with Flask. I will show you how to will use the API to show how to write tests in a Flask application. To keep this tutorial simple, focus on testing the API rather than building it. The API application can be found in the api.py file in the project root directory of the cloned repository.

What is Pytest and how do I use it?

Small and scalable tests are simple to write with Pytest. This code snippet shows the basic layout of a Pytest test:

from api import app # Flask instance of the API

def test_index_route():
    response = app.test_client().get('/')

    assert response.status_code == 200
    assert response.data.decode('utf-8') == 'Testing, Flask!'

Your first step is to import a Flask instance – app – from the api (created in the application), as seen in the previous snippet. The imported instance then exposes a test_client() method from Flask that contains the features needed to make HTTP requests to the application under test. In this case, it is to the default (/) API endpoint. The code then asserts that the responses received by test_client() are what is expected after decoding the byte object. In the next steps, you will use this format to write your tests.

Note: By default, Pytest encodes API responses using the utf-8 codec. Use the decode() method to convert byte objects received from the test client to readable string responses, as seen in the previous code snippet.

Setting up Pytest on Flask

Installing Pytest is simple. If you cloned the repository, it is already installed, and you can skip this step. If you have not cloned the repository, run this command on the terminal:

pip install pytest

Note: If you are using a virtual environment, activate it before starting the installation. It is best practice to use a virtual environment for isolating different applications and their associated Python packages.

Importing the Pytest module

After the installation is complete, import the Pytest module. This snippet can be placed on any test file:

import pytest

Naming conventions for tests

For this tutorial, keep your tests in a tests directory in the root folder of the application. Pytest recommends that test file names begin with the format test_*.py or end with format **_test.py, Using these formats enables Pytest to auto-discover the test files easily and also minimizes confusion when running tests. Precede test methods with the word test when creating tests: test_index_route():.

Writing tests with Pytest

Now that you know how to set up Pytest on a Flask app, you can start writing tests for the book-retrieval API. Our first test will be for the /bookapi/books route:

# get books data

books = [
    {
        "id": 1,
        "title": "CS50",
        "description": "Intro to CS and art of programming!",
        "author": "Havard",
        "borrowed": False
    },
    {
        "id": 2,
        "title": "Python 101",
        "description": "little python code book.",
        "author": "Will",
        "borrowed": False
    }
]

@app.route("/bookapi/books")
def get_books():
    """ function to get all books """
    return jsonify({"Books": books})

For this route, all you are doing is returning a list of books that have been hardcoded in the books variable. You will test this endpoint in the next step using the same format to define the test function and assert that the response received by the test_client() is what you expect.

import json
from api import app

def test_get_all_books():
    response = app.test_client().get('/bookapi/books')
    res = json.loads(response.data.decode('utf-8')).get("Books")
    assert type(res[0]) is dict
    assert type(res[1]) is dict
    assert res[0]['author'] == 'Havard'
    assert res[1]['author'] == 'Will'
    assert response.status_code == 200
    assert type(res) is list
    ....

This test snippet verifies that you receive a list of books. It does this by asserting that the response received by the test_client() is indeed the expected list of books. It also asserts that the response contains a list of dictionaries, which are the individual books in the defined book object. Finally, it asserts that the first book in the list has the author Havard and that the second book has the author Will. That is a test to fetch all books from the /bookapi/books endpoint.

Running tests with Pytest

To run Pytest tests, you can either use py.test or pytest in the terminal. You can also run a single file by explicitly specifying the filename after the Pytest command: pytest test_api.py. When these commands are executed, Pytest automatically finds all the tests in either the root directory, or in the specified single file.

When no test files have been explicitly defined, Pytest executes any tests in the root directory that follow the standard naming pattern. You don’t need to specify filenames or directories.

Executing the first test

You have verification that the test was executed successfully! Pytest marks a passed test with a green dot .; it marks a failed test with a red F. Count the number of dots or Fs to figure out how many tests passed and failed, and in what execution order.

Note: If you are debugging the tests to the console and you need to print out a response, you can use $ pytest -s test_*.py to log to stdout. When the Pytest s option is defined, you can console messages inside the test and debug the output during the test execution.

Now that you have successfully executed the first test, you can integrate CircleCI to run tests automatically whenever you push to the main branch.

Setting up CircleCI

Create a .circleci directory in your root directory and then add a config.yml file there. The config file contains the CircleCI configuration for every project. Use the CircleCI Python orb to execute tests as part of this configuration:

version: 2.1
orbs:
  python: circleci/python@2.0.3
jobs:
  build-and-test:
    docker:
      - image: cimg/python:3.10.4
    steps:
      - checkout
      - python/install-packages:
          pkg-manager: pip
      - run:
          name: Run tests
          command: pytest
workflows:
  sample:
    jobs:
      - build-and-test

CircleCI orbs are reusable packages of YAML code that condense multiple lines of code into a single line. In this example, that line of code is: python: circleci/python@2.0.3.

Note: You might need to enable organization settings to allow the use of third-party orbs in the CircleCI dashboard, or request permission from your organization’s CircleCI admin.

After setting up the configuration, commit your changes and then push the changes to GitHub.

Note: GitLab users can also follow this tutorial by pushing the sample project to GitLab and setting up a CI/CD pipeline for their GitLab repo.

Log into CircleCI and go to the Projects dashboard. The GitHub repositories associated with your GitHub username or organization and the specific repository that you want to set up in CircleCI. In this case it is testing-flask-with-pytest.

Select project

Next, click Set Up Project to start building the project on CircleCI. This will display a modal pop up with a default option to use the configuration file within your project’s repository. Enter the name of the branch where the configuration file is housed:

Select configuration

Click Set Up Project to complete the process.

Voila! From the CircleCI dashboard, click the build to review details. You can verify that running the first Pytest test and integrating it into CircleCI was a success.

Pipeline Setup Success

Now that you have successfully set up continuous integration, the next step is grouping tests and running batches of grouped tests.

Grouping and running batches of tests

As application features are added, you need to increase the number of tests to make sure that everything works. It is easy to get overwhelmed by the large number of test scripts available. You do not need to worry though, Pytest already has that figured out. Pytest allows you to run multiple tests from a single file by grouping tests.

Pytest provides markers that you use to set attributes and features for testing functions.

Using Pytest test markers

You can use markers to give tests different behaviors, like skipping tests, running a subset of tests, or even failing them. Some of the default markers bundled with Pytest include xfail, skip, and parameterize. For this project, you will create a custom marker that will group all the tests that perform GET requests to the /bookapi/books and /bookapi/book/:id endpoints.

Here is an example of the structure you will use to create a custom marker:

@pytest.mark.<markername>
def test_method():
  # test code

To use the custom marker in Pytest, define it as an argument in the pytest command:

$ pytest -m <markername>

-m <markername> is the custom marker name you will use for the tests.

It is important to note that you need to import pytest in your test file to use Pytest markers.

Markers also need to be registered so that Pytest can suppress warnings about them. Add this to your pytest.ini file:

[pytest]
markers =
    <markername>: Marker description

Using markers to group tests

For the tutorial, you want to group the tests that make the GET request to the /bookapi endpoints.

Create a customer marker called get_request and add it to the tests like this.

import pytest
...
# Other imports here

@pytest.mark.get_request
def test_get_book_by_id():
    response = app.test_client().get('/bookapi/books/1')
    res = json.loads(response.data.decode('utf-8')).get("Book")
    print(res)
    assert res['id'] == 1
    assert res['author'] == 'Havard'
    assert res['title'] == 'CS50'
    assert response.status_code == 200

Running the test with the pytest -m get_request argument executes all the tests marked with the @pytest.mark.get_request decorator in the test file. You can also verify this by running it in the terminal.

Now, commit and push your changes to GitHub and verify that the pipeline executes successfully.

Passing Tests

Excellent! All the tests were executed successfully.

Conclusion

In this tutorial, you have set up tests with Pytest, executed them, learned to group tests using Pytest markers. You learned how to use Pytest command line arguments to execute tests and how to use the test_client() method to make HTTP requests, and you know how to use the received responses in your tests. I hope you enjoyed this tutorial. If you did, share what you learned with your team. Nothing solidifies learning like teaching someone else. Until the next one, enjoy your coding projects!

Waweru Mwaura is a software engineer and a life-long learner who specializes in quality engineering. He is an author at Packt and enjoys reading about engineering, finance, and technology. You can read more about him on his web profile.

Read more posts by Waweru Mwaura