This tutorial covers:
- Setting up Pytest on a Flask app
- Writing Pytest tests
- Grouping tests
Writing tests in any programming language can be difficult, but it does not have to be. In this tutorial, I will show you how you can easily write and run tests using Flask and Pytest.
As a bonus, we will also integrate a CI/CD pipeline to run tests on a Flask app using CircleCI.
Be sure to check out our other Flask tutorials to learn about application logging, authentication decorators, and automating Flask deployments.
Prerequisites
To get the most from this tutorial, you will need:
- Basic understanding of the Python programming language
- Understanding of Flask framework
- Basic understanding of testing
You will also need to have these installations and setups:
- Python - version >= 3.5 installed in your machine
- A GitHub account; you can create one here
- A CircleCI account; you can create one here
- The project repository from GitHub; clone it here
Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.
Once the project is cloned, go to the root folder of the project. Install the dependencies using the command pip install -r requirements.txt.
Why use Flask?
Flask is a lightweight micro-web framework written in Python. Flask comes pre-installed with just the core features so that you can pick customizations depending on the requirements of your project. Flask does not make decisions about which database to use or what the default parser is. All that can be changed with modifications, which makes it extensible and flexible for the emerging needs of your development team.
In this project tutorial, we will use a simple book-retrieval API built with Flask. We will use the API to show how we write tests in a Flask application. To keep this tutorial simple, we will focus on testing the API rather than building it. The API application can be found in the api.py file in the project root directory of the cloned repository.
Now that you know what Flask is and how to use it, you can build on that knowledge by learning about Pytest and how to set it up on a Flask app.
What is Pytest and how do I use it?
Pytest is a Python testing framework designed to assist developers with writing better systems and releasing them with confidence. Small and scalable tests are simple to write with Pytest. This code snippet shows the basic layout of a Pytest test:
from api import app # Flask instance of the API
def test_index_route():
response = app.test_client().get('/')
assert response.status_code == 200
assert response.data.decode('utf-8') == 'Testing, Flask!'
Testing Flask requires that we first import a Flask instance app from our api (created in our application), as seen in the previous snippet. The imported instance then exposes a test_client() method from Flask that contains the features needed to make HTTP requests to the application under test. In this case, it is to our default (/) API endpoint. We then assert that the responses received by test_client() are to be expected after decoding the byte object. In the next steps below, you will use this format to write your tests.
Note: By default, Pytest encodes API responses using the utf-8 codec. You need to use the decode() method to convert the byte objects received from the test client to readable string responses, as seen in the previous code snippet.
Setting up Pytest on Flask
Installing Pytest is simple. If you cloned the repository, it is already installed, and you can skip this step. If you have not cloned the repository, run this command on the terminal:
pip install pytest
Note: If you are using a virtual environment, activate it before doing the installation. It is best practice to use a virtual environment for isolating different applications and their associated Python packages.
Importing the Pytest module
After the installation is complete, import the Pytest module. This snippet can be placed on any test file:
import pytest
Naming conventions for tests
For this tutorial, keep your tests in a tests directory in the root folder of the application. Pytest recommends that test file names begin with the format test_*.py or end with format **_test.py, Using these formats enables Pytest to auto-discover the test files easily and also minimizes confusion when running tests. Precede test methods with the word test when creating tests: test_index_route():.
Writing tests with Pytest
Now that you know how to set up Pytest on a Flask app, you can start writing tests for the book-retrieval API. Your first test will be for the /bookapi/books route.
# get books data
books = [
{
"id": 1,
"title": "CS50",
"description": "Intro to CS and art of programming!",
"author": "Havard",
"borrowed": False
},
{
"id": 2,
"title": "Python 101",
"description": "little python code book.",
"author": "Will",
"borrowed": False
}
]
@app.route("/bookapi/books")
def get_books():
""" function to get all books """
return jsonify({"Books": books})
All this route does is return a list of books that are hardcoded into the books variable. You will test this endpoint in the next step. Using the same format, define your test function and assert that the response received by the test_client() is what you expect.
import json
from api import app
def test_get_all_books():
response = app.test_client().get('/bookapi/books')
res = json.loads(response.data.decode('utf-8')).get("Books")
assert type(res[0]) is dict
assert type(res[1]) is dict
assert res[0]['author'] == 'Havard'
assert res[1]['author'] == 'Will'
assert response.status_code == 200
assert type(res) is list
....
This test snippet ensures that you are able to receive a list of books. You do this by asserting that the response received by the test_client() is indeed your expected list of books. You are also asserting that the response contains a list of dictionaries, which are the individual books in the defined book object. You are also asserting that the first book in the list has the author Havard and the second book has the author Will. That is all you need to write a test to fetch all the books from the /bookapi/books endpoint.
Running tests with Pytest
To run Pytest tests, you can either use py.test or pytest in the terminal. You can also run a single file by explicitly specifying the filename after the Pytest command: pytest test_api.py. When these commands are executed, Pytest finds all the tests in either the root directory, or in the specified single file.
When no test files have been explicitly defined, Pytest executes any tests in the root directory that follow the standard naming pattern, without you needing to specify filenames or directories.
You have verification that the test was executed successfully! Pytest marks a passed test with a green dot .; it marks a failed test with a red F. Count the number of dots or Fs to figure out how many tests passed and failed, and in what execution order.
Note: If you are debugging the tests to the console and you need to print out a response, you can use $ pytest -s test_*.py to log to stdout. When the Pytest s option is defined, you can console messages inside the test and debug the output during the test execution.
Now that you have successfully executed your first test, you can integrate CircleCI to run the tests automatically when you push to the main branch.
Setting up CircleCI
Create a .circleci directory in your root directory and then add a config.yml file there. The config file contains the CircleCI configuration for every project. Use the CircleCI Python orb to execute the tests as shown here:
version: 2.1
orbs:
python: circleci/python@2.1.1
jobs:
build-and-test:
docker:
- image: cimg/python:3.11.0
steps:
- checkout
- python/install-packages:
pkg-manager: pip
- run:
name: Run tests
command: pytest
workflows:
sample:
jobs:
- build-and-test
CircleCI orbs are reusable packages of YAML code. Orbs condense multiple lines of code into a single line. In this example, that line of code is: python: circleci/python@2.1.1. You might need to enable organization settings to allow the use of third-party orbs in the CircleCI dashboard, or request permission from your organization’s CircleCI admin.
After setting up the configuration, commit your changes and then push your changes to GitHub.
Log into CircleCI and go to the Projects dashboard. From the list of GitHub repositories associated with your username or organization, find the specific repository that you want to set up in CircleCI. In this case it is testing-flask-with-pytest.
Next, click Set Up Project to start building the project on CircleCI. This displays a modal pop up with a default option to use the configuration file within your project’s repository. Enter the name of the branch where the configuration file is housed.
Click Set Up Project to complete the process.
Voila! From the CircleCI dashboard, click the build to review details. You can verify that running the first Pytest test and integrating it into CircleCI was a success.
Now that you have successfully set up continuous integration, the next step is grouping tests and running batches of grouped tests.
Grouping and running batches of tests
As application features are added, you need to increase the number of tests to make sure that everything works. It is easy to get overwhelmed by the large number of test scripts available. You do not need to worry though, Pytest already has that figured out. Pytest allows you to run multiple tests from a single file by grouping tests.
Pytest provides markers that you use to set attributes and features for testing functions.
Using Pytest test markers
We can use markers to give tests different behaviors, like skipping tests, running a subset of tests, or even failing them. Some of the default markers bundled with Pytest include xfail, skip, and parameterize. For this project, you will create a custom marker that will group all the tests that perform GET requests to the /bookapi/books and /bookapi/book/:id endpoints.
Here is an example of the structure you will use to create a custom marker:
@pytest.mark.<markername>
def test_method():
# test code
To use the custom marker in Pytest, define it as an argument in the pytest command:
$ pytest -m <markername>
-m <markername> is the custom marker name you will use for your tests.
You need to import pytest in your test file to use Pytest markers. Markers also need to be registered so that Pytest can suppress warnings about them. Add this to your pytest.ini file:
[pytest]
markers =
<markername>: Marker description
Using markers to group tests
For the tutorial, you want to group the tests that make the GET request to the /bookapi endpoints.
Create a customer marker called get_request and add it to the tests:
import pytest
...
# Other imports here
@pytest.mark.get_request
def test_get_book_by_id():
response = app.test_client().get('/bookapi/books/1')
res = json.loads(response.data.decode('utf-8')).get("Book")
print(res)
assert res['id'] == 1
assert res['author'] == 'Havard'
assert res['title'] == 'CS50'
assert response.status_code == 200
Running the test with the pytest -m get_request argument executes all the tests marked with the @pytest.mark.get_request decorator in the test file. You can also verify this by running it in the terminal.
Now, commit and push your changes to GitHub and check that the pipeline executes successfully.
Excellent! All your tests were executed successfully.
Conclusion
In this tutorial, you have set up tests with Pytest, executed them, learned to group tests using Pytest markers. You learned how to use Pytest command line arguments to execute tests and how to use the test_client() method to make HTTP requests. You know how to use the received responses in your tests. I hope you enjoyed this tutorial. If you did, share what you learned with your team. Nothing solidifies learning like teaching someone else. Until the next one, enjoy your coding projects!
Waweru Mwaura is a software engineer and a life-long learner who specializes in quality engineering. He is an author at Packt and enjoys reading about engineering, finance, and technology. You can read more about him on his web profile.
