How to Use Unified Logging Config Across the Project
Contents
When we are working on a project, we often need to log some message for easier debugging. How do we configure the logging settings once and use it across the project?
Set up logging for the project
Suppose we have the following file in our project:
main.py
module1.py
module2.py
main.py
is the entry point of our script and use functions or classes from
module1
and module2.
The best practice is to set up logging in main.py
like this:
import module1
import module2
import logging
# other imports
logging_level = logging.DEBUG
main_logger = logging.getLogger()
main_logger.setLevel(logging_level)
# Set up a stream handler to log to the console
stream_handler = logging.StreamHandler()
stream_handler.setLevel(logging_level)
formatter = logging.Formatter("%(name)s - %(levelname)s - %(message)s")
stream_handler.setFormatter(formatter)
# Add handler to logger
main_logger.addHandler(stream_handler)
# other codes
In the above code, we define a root logger main_logger
via
logging.getLogger()
(notice that no name is provided to this method), and set
up the handlers and logging level properly.
In module1
and module2
, we only need to define a logger like this:
import logging
logger = logging.getLogger(__name__)
# then use logger.info() or logger.debug() in your code.
In other modules, if we define the logger using logging.getLogger(__name__)
,
it will automatically use the settings from the root logger. This ensures that
we only need to config the logging once in our project.
Changing logging level to disable some logging message
There is also another benifit in using a unified config: we can easily control
the logging message verbosity by changing the logging level. For example, in
the above example, we set logging level in main.py
to logging.DEBUG
, so all
logger.debug()
and logger.info()
message will be printed to the console. If
we do not want to see logger.debug("xxx")
messages anymore, we can set
logging level to logging.INFO
, which will disable any debug message across
the whole project.
Disable logging message from other packages
When we import 3rd party packages in our own project and set proper logging level for the project, if a logging message in 3rd party packages is below that level, the message will also be printed. Since we ususally only care about logging messages in our project, we can disable logging message in 3rd party packages by setting their logger level higher or disable logging entirely for 3rd party packages. Here is the code:
import logging
# suppose that the logger level of our module is logging.DEBUG, we set
# logger level of other module higher than logging.DEBUG
for m in ("urllib3", "foo", "bar"):
logging.getLogger(m).setLevel(logging.CRITICAL)
# To disable it entirely, use the following.
# logging.getLogger(m).disabled = True