This tutorial covers:
- The Flask logging module
- Logging Flask events by severity level
- Testing the logging module
Without logs, or a good understanding of them, debugging an application or looking through an error stack trace can be challenging. Luckily, Flask logging can change the way you understand debugging and how you interact with logs produced by the application. The Flask logging module gives you a way to record errors over different severity levels. A default logging module is included in the Python standard library, and it provides both simple and advanced logging functions.
In this tutorial, I will cover how to log Flask application events based on their severity levels and how to test the logging module.
For this tutorial the following technologies are be required:
- Basic understanding of the Python programming language
- Understanding of the Flask framework
- Basic understanding of testing methods
You also need to have the following installed:
- Python Version >= 3.5 installed in your machine.
- A GitHub account. You can create one here.
- A CircleCI account. Create one here.
Our tutorials are platform-agnostic, but use CircleCI as an example. If you don’t have a CircleCI account, sign up for a free one here.
Cloning the sample project repository
Get started by cloning the project repository from GitHub.
Once the project is cloned, you also need to install the dependencies. Use the command
pip install -r requirements.txt from the root folder of the project.
Understanding Flask logging
Flask uses the standard Python logging module to log messages:
[app.logger](https://docs.python.org/3/library/logging.html#module-logging). This logger can be extended and used to log custom messages. Implementing a flexible, event logging system for Flask applications gives you the ability to know when something goes wrong while your applications are executing or when they encounter errors. The diagram below shows the different parts of the Flask logging module and how they contribute to handling application logs.
The Python logger uses four sub-modules:
- Logger is the primary interface that logs events from your application. These events, when recorded, are referred to as log records.
- Handlers direct log events/records into respective destinations.
- Formatters specify the layout of your messages when they are written by the logger.
- Filters help developers manage the log record using parameters. These parameters can be in addition to the log levels.
Implementing a Flask logger
Logging allows developers to monitor the flow of a program with actions taken. You can use loggers to track application flows like tracking transactional data in ecommerce applications or recording events when an API call interacts with a service.
To start with logging in Flask, first import the logging module from Python. This logger module comes out of the box from the Python installation and does not need configuration. The Python logging module logs events based on pre-defined levels. The recorded log events are known as log records. Each record level has a different severity level:
- Debug : 10
- Info: 20
- Warning: 30
- Error: 40
- Critical: 50
The logger records logs only when the severity is bigger than their log levels. Then the logger passes them to the handlers.
This snippet shows the different types of loggers and their usage in a Flask route
@app.route('/') def main(): app.logger.debug("Debug log level") app.logger.info("Program running correctly") app.logger.warning("Warning; low disk space!") app.logger.error("Error!") app.logger.critical("Program halt!") return "logger levels!"
You can find this snippet in the
app.py file. While the Flask application is running, navigate to the
/ home route in your browser to review the logs.
Here is how to use the loggers:
Debugprovides developers with detailed information for diagnosing program error.
Infodisplays a confirmation message that a program’s flow behavior is executing as expected.
Warningshows that something unexpected occurred, or that a problem might occur in the near future (low disk space, for example).
Errorindicates a serious problem, like the program failed to execute some functionality.
Criticalshows the occurrence of a serious error in the application, such as a program failure.
Configuring a basic logger
A logger that provides just the basics is enough for many applications. To configure this type of logging in your
app.py file, add this:
from flask import Flask import logging logging.basicConfig(filename='record.log', level=logging.DEBUG) app = Flask(__name__) @app.route('/') def main(): # showing different logging levels app.logger.debug("debug log info") app.logger.info("Info log information") app.logger.warning("Warning log info") app.logger.error("Error log info") app.logger.critical("Critical log info") return "testing logging levels." if __name__ == '__main__': app.run(debug=True)
This snippet specifies where Flask will log your application based on the levels from
DEBUG. It also sets up the message that will be logged when you call your
/ home route using a client like Postman.
Note: Logging configuration should be completed before you create the Flask app object. If the
app.logger is accessed before configuration, it uses the default Python handlers.
This basic configuration using
logging.basicConfig logs messages and stores the log information in a
.log file. For our sample project it is the
Now, execute your Flask application using this command:
FLASK_APP=app.py flask run
Open your client application and make a
GET request to your route for the running application. In this case it is
http://127.0.0.1:5000/. When the
main function in your program is called, it creates the
record.log file, and then it sets the logging level to DEBUG. The logging activity should appear in the file
record.log and output should be something like this:
## record.log file output DEBUG:app:debug log info INFO:app:Info log information WARNING:app:Warning log info ERROR:app:Error log info CRITICAL:app:Critical log info INFO:werkzeug:127.0.0.1 - - [01/Mar/2022 12:35:19] "GET / HTTP/1.1" 200 -
You were able to manipulate the logger object into logging all the configured loggers based on the different logger levels. When you have different loggers set for recording different levels of information, you can disable some logs from being displayed on the console and enable others. For this configuration to print out the logs on the terminal, you can remove the file configuration
filename='record.log' in the
logging.basicConfig() object where logs get recorded.
While these log outputs are readable, they may not be very useful, especially because you do not know when the events occurred. To fix this, you can add a format to your logs, as described in the next section.
Formatting log outputs
The Python formatter formats the structure of the record into specific structures that make it easy to read logs and tie them to specific events. It can be applied inside the
The log formatter consists of the following configuration:
%(asctime)sconfigures the timestamp as a string
%(levelname)sconfigures the logging level as a string.
%(name)sconfigures the logger name as a string.
%(threadName)sis the thread name.
%(message)sconfigures log messages as a string.
You can apply these formatting options to your configuration for a more accurate object output:
......... logging.basicConfig(filename='record.log', level=logging.DEBUG, format='%(asctime)s %(levelname)s %(name)s %(threadName)s : %(message)s') app = Flask(__name__) .........
format configuration specified in the previous snippet, your log output can be tied to a specific timestamp, a specific thread, and even a particular threadname. The resulting output log makes more sense and looks cleaner when you run the application and observe your
2022-03-01 13:09:11,787 DEBUG app Thread-3 : debug log info 2022-03-01 13:09:11,788 INFO app Thread-3 : Info log information 2022-03-01 13:09:11,788 WARNING app Thread-3 : Warning log info 2022-03-01 13:09:11,788 ERROR app Thread-3 : Error log info 2022-03-01 13:09:11,788 CRITICAL app Thread-3 : Critical log info 2022-03-01 13:09:11,788 INFO werkzeug Thread-3 : 127.0.0.1 - - [01/Mar/2022 13:09:11] "GET / HTTP/1.1" 200 -
The log levels have the messages but also a timestamp, level of the log, name of the application, the thread of the process, and finally the message of your defined logs. If you encountered an error, this makes it easy to step through the timestamp to identify the specific process, timestamp and even message. This process is much more useful while debugging than just reading plain error logs.
Now that you have created logs with the Python
basicConfig, you can write tests for your logger module.
Testing Python loggers is nearly the same as writing tests for normal Python functions. To write your first test, create a file named
test_app.py. Open it and add your first test snippet:
from flask import Flask from app import app import logging import unittest class TestLogConfiguration(unittest.TestCase): """[config set up] """ def test_INFO__level_log(self): """ Verify log for INFO level """ self.app = app self.client = self.app.test_client with self.assertLogs() as log: user_logs = self.client().get('/') self.assertEqual(len(log.output), 4) self.assertEqual(len(log.records), 4) self.assertIn('Info log information', log.output)
In the test snippet above, we use the
test_client to first make a request to the
/ route just as we are doing while running our application, and after we do this, we can verify that the log output logs the
INFO level log information. As the wise men say, the only way to know whether our tests run is by executing them, we can do that with the command below on your terminal of choice:
Review the results of your run.
Congratulations! You have executed your first test successfully. Now you can extend your tests to other log levels that are defined in the application. Here’s an example:
def test_WARNING__level_log(self): """ Verify log for WARNING level """ self.app = app self.client = self.app.test_client with self.assertLogs() as log: user_logs = self.client().get('/') self.assertEqual(len(log.output), 4) self.assertIn('Warning log info', log.output) def test_ERROR__level_log(self): """ Verify log for ERROR level """ self.app = app self.client = self.app.test_client with self.assertLogs() as log: user_logs = self.client().get('/') self.assertEqual(len(log.output), 4) self.assertIn('Error log info', log.output)
These test the log levels at each different stage in the snippets.
Note: The log object contains more information that can then be tested and asserted based on the needs of the application.
Your last task is to share your tests with the world. You can accomplish that using CircleCI as your continuous integration platform.
Setting up Git and pushing to CircleCI
To set up CircleCI, initialize a Git repository in the project by running:
.gitignore file in the root directory. Inside the file, add any modules you want to prevent from being added to your remote repository. Add a commit and then push your project to GitHub.
Now, log into the CircleCI dashboard and navigate to
Projects. There will be a list of all the GitHub repositories associated with your GitHub username or organization. The specific repository for this tutorial is
From the Projects dashboard, select the option to set up the selected project and use the option for an existing configuration, which uses
config.yml in the repository. Start the build.
Tip After initiating the build, expect your pipeline to fail. That is because you have not yet added your customized
.circleci/config.yml configuration file to GitHub, which is needed for the project to build properly.
Setting up CircleCI
.circleci directory in your root directory, then add a
config.yml file to it. The config file contains the CircleCI configuration for every project. For this set up, you will use CircleCI Python orb. Use this configuration to execute your tests:
version: 2.1 orbs: python: firstname.lastname@example.org workflows: build-app-with-test: jobs: - build-and-test jobs: build-and-test: docker: - image: cimg/python:3.9 steps: - checkout - python/install-packages: pkg-manager: pip - run: name: Run tests command: pytest
CircleCI orbs are reusable packages of YAML configurations that condense multiple lines of code into a single one:
python: email@example.com. You may need to enable organization settings (if you are the administrator) or request permission from your organization’s CircleCI admin to allow the use of third party orbs in the CircleCI dashboard.
After setting up the configuration, push your configuration to GitHub. CircleCI will automatically start building your project.
Check the CircleCI dashboard and expand the build details to verify that you have run your first PyTest test successfully. It should be integrated into CircleCI.
Excellent! Not only have you written Flask loggers, but you have also tested them, and shared with the world – or at least with the rest of your team.
In this tutorial you have learned how to configure an API for different log output levels and how to format log output in a way that is not only straightforward but makes it easy to identify issues when they occur. Plus, you learned to write tests for logger methods and to assert different messages that have been defined to be logged. That brings us to the end of this tutorial! As always, I enjoyed creating it and I hope that you enjoyed following along.
Waweru Mwaura is a software engineer and a life-long learner who specializes in quality engineering. He is an author at Packt and enjoys reading about engineering, finance, and technology. You can read more about him on his web profile.