From 013a1824d3c57b7ffc5240bc03d219a4a290aa3d Mon Sep 17 00:00:00 2001 From: Daniele Procida Date: Sat, 3 Jul 2021 22:31:26 +0200 Subject: [PATCH] Refs #32880 -- Created a new logging how-to document. Moved how-to material from topic document into a new document, and added new material. Introduced minor improvements to logging reference document. --- docs/howto/index.txt | 1 + docs/howto/logging.txt | 343 ++++++++++++++++++++++++++++++++++++++++ docs/ref/logging.txt | 21 ++- docs/topics/logging.txt | 102 +----------- 4 files changed, 364 insertions(+), 103 deletions(-) create mode 100644 docs/howto/logging.txt diff --git a/docs/howto/index.txt b/docs/howto/index.txt index ffe4c5519e7..75304f68272 100644 --- a/docs/howto/index.txt +++ b/docs/howto/index.txt @@ -22,6 +22,7 @@ you quickly accomplish common tasks. error-reporting initial-data legacy-databases + logging outputting-csv outputting-pdf overriding-templates diff --git a/docs/howto/logging.txt b/docs/howto/logging.txt new file mode 100644 index 00000000000..f2bd0b84854 --- /dev/null +++ b/docs/howto/logging.txt @@ -0,0 +1,343 @@ +.. _logging-how-to: + +================================ +How to configure and use logging +================================ + +.. seealso:: + + * :ref:`Django logging reference ` + * :ref:`Django logging overview ` + +Django provides a working :ref:`default logging configuration +` that is readily extended. + +Make a basic logging call +========================= + +To send a log message from within your code, you place a logging call into it. + +.. admonition:: Don't be tempted to use logging calls in ``settings.py``. + + The way that Django logging is configured as part of the ``setup()`` + function means that logging calls placed in ``settings.py`` may not work as + expected, because *logging will not be set up at that point*. To explore + logging, use a view function as suggested in the example below. + +First, import the Python logging library, and then obtain a logger instance +with :py:func:`logging.getLogger`. Provide the ``getLogger()`` method with a +name to identify it and the records it emits. A good option is to use +``__name__`` (see :ref:`naming-loggers` below for more on this) which will +provide the name of the current Python module as a dotted path:: + + import logging + + logger = logging.getLogger(__name__) + +It's a good convention to perform this declaration at module level. + +And then in a function, for example in a view, send a record to the logger:: + + def some_view(request): + ... + if some_risky_state: + logger.warning('Platform is running at risk') + +When this code is executed, a :py:class:`~logging.LogRecord` containing that +message will be sent to the logger. If you're using Django's default logging +configuration, the message will appear in the console. + +The ``WARNING`` level used in the example above is one of several +:ref:`logging severity levels `: ``DEBUG``, +``INFO``, ``WARNING``, ``ERROR``, ``CRITICAL``. So, another example might be:: + + logger.critical('Payment system is not responding') + +.. important:: + + Records with a level lower than ``WARNING`` will not appear in the console + by default. Changing this behavior requires additional configuration. + +Customize logging configuration +=============================== + +Although Django's logging configuration works out of the box, you can control +exactly how your logs are sent to various destinations - to log files, external +services, email and so on - with some additional configuration. + +You can configure: + +* logger mappings, to determine which records are sent to which handlers +* handlers, to determine what they do with the records they receive +* filters, to provide additional control over the transfer of records, and + even modify records in-place +* formatters, to convert :class:`~logging.LogRecord` objects to a string or + other form for consumption by human beings or another system + +There are various ways of configuring logging. In Django, the +:setting:`LOGGING` setting is most commonly used. The setting uses the +:ref:`dictConfig format `, and extends the +:ref:`default logging configuration `. + +See :ref:`configuring-logging` for an explanation of how your custom settings +are merged with Django's defaults. + +See the :mod:`Python logging documentation ` for +details of other ways of configuring logging. For the sake of simplicity, this +documentation will only consider configuration via the ``LOGGING`` setting. + +.. _basic-logger-configuration: + +Basic logging configuration +--------------------------- + +When configuring logging, it makes sense to + +Create a ``LOGGING`` dictionary +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +In your ``settings.py``:: + + LOGGING = { + 'version': 1, # the dictConfig format version + 'disable_existing_loggers': False, # retain the default loggers + } + +It nearly always makes sense to retain and extend the default logging +configuration by setting ``disable_existing_loggers`` to ``False``. + +Configure a handler +~~~~~~~~~~~~~~~~~~~ + +This example configures a single handler named ``file``, that uses Python's +:class:`~logging.FileHandler` to save logs of level ``DEBUG`` and higher to the +file ``general.log`` (at the project root): + +.. code-block:: python + :emphasize-lines: 3-8 + + LOGGING = { + [...] + 'handlers': { + 'file': { + 'class': 'logging.FileHandler', + 'filename': 'general.log', + }, + }, + } + +Different handler classes take different configuration options. For more +information on available handler classes, see the +:class:`~django.utils.log.AdminEmailHandler` provided by Django and the various +:py:mod:`handler classes ` provided by Python. + +Logging levels can also be set on the handlers (by default, they accept log +messages of all levels). Using the example above, adding: + +.. code-block:: python + :emphasize-lines: 4 + + { + 'class': 'logging.FileHandler', + 'filename': 'general.log', + 'level': 'DEBUG', + } + +would define a handler configuration that only accepts records of level +``DEBUG`` and higher. + +Configure a logger mapping +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To send records to this handler, configure a logger mapping to use it for +example: + +.. code-block:: python + :emphasize-lines: 3-8 + + LOGGING = { + [...] + 'loggers': { + '': { + 'level': 'DEBUG', + 'handlers': ['file'], + }, + }, + } + +The mapping's name determines which log records it will process. This +configuration (``''``) is *unnamed*. That means that it will process records +from *all* loggers (see :ref:`naming-loggers` below on how to use the mapping +name to determine the loggers for which it will process records). + +It will forward messages of levels ``DEBUG`` and higher to the handler named +``file``. + +Note that a logger can forward messages to multiple handlers, so the relation +between loggers and handlers is many-to-many. + +If you execute:: + + logger.debug('Attempting to connect to API') + +in your code, you will find that message in the file ``general.log`` in the +root of the project. + +Configure a formatter +~~~~~~~~~~~~~~~~~~~~~ + +By default, the final log output contains the message part of each :class:`log +record `. Use a formatter if you want to include additional +data. First name and define your formatters - this example defines +formatters named ``verbose`` and ``simple``: + +.. code-block:: python + :emphasize-lines: 3-12 + + LOGGING = { + [...] + 'formatters': { + 'verbose': { + 'format': '{name} {levelname} {asctime} {module} {process:d} {thread:d} {message}', + 'style': '{', + }, + 'simple': { + 'format': '{levelname} {message}', + 'style': '{', + }, + }, + } + +The ``style`` keyword allows you to specify ``{`` for :meth:`str.format` or +``$`` for :class:`string.Template` formatting; the default is ``$``. + +See :ref:`logrecord-attributes` for the :class:`~logging.LogRecord` attributes +you can include. + +To apply a formatter to a handler, add a ``formatter`` entry to the handler's +dictionary referring to the formatter by name, for example: + +.. code-block:: python + :emphasize-lines: 5 + + 'handlers': { + 'file': { + 'class': 'logging.FileHandler', + 'filename': 'general.log', + 'formatter': 'verbose', + }, + }, + +.. _naming-loggers: + +Use logger namespacing +~~~~~~~~~~~~~~~~~~~~~~ + +The unnamed logging configuration ``''`` captures logs from any Python +application. A named logging configuration will capture logs only from loggers +with matching names. + +The namespace of a logger instance is defined using +:py:func:`~logging.getLogger`. For example in ``views.py`` of ``my_app``:: + + logger = logging.getLogger(__name__) + +will create a logger in the ``my_app.views`` namespace. ``__name__`` allows you +to organize log messages according to their provenance within your project's +applications automatically. It also ensures that you will not experience name +collisions. + +A logger mapping named ``my_app.views`` will capture records from this logger: + +.. code-block:: python + :emphasize-lines: 4 + + LOGGING = { + [...] + 'loggers': { + 'my_app.views': { + ... + }, + }, + } + +A logger mapping named ``my_app`` will be more permissive, capturing records +from loggers anywhere within the ``my_app`` namespace (including +``my_app.views``, ``my_app.utils``, and so on): + +.. code-block:: python + :emphasize-lines: 4 + + LOGGING = { + [...] + 'loggers': { + 'my_app': { + ... + }, + }, + } + +You can also define logger namespacing explicitly:: + + logger = logging.getLogger('project.payment') + +and set up logger mappings accordingly. + +.. _naming-loggers-hierarchy: + +Using logger hierarchies and propagation +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Logger naming is *hierarchical*. ``my_app`` is the parent of ``my_app.views``, +which is the parent of ``my_app.views.private``. Unless specified otherwise, +logger mappings will propagate the records they process to their parents - a +record from a logger in the ``my_app.views.private`` namespace will be handled +by a mapping for both ``my_app`` and ``my_app.views``. + +To manage this behavior, set the propagation key on the mappings you define:: + + LOGGING = { + [...] + 'loggers': { + 'my_app': { + [...] + }, + 'my_app.views': { + [...] + }, + 'my_app.views.private': { + [...] + 'propagate': False, + }, + }, + } + +``propagate`` defaults to ``True``. In this example, the logs from +``my_app.views.private`` will not be handled by the parent, but logs from +``my_app.views`` will. + +Configure responsive logging +---------------------------- + +Logging is most useful when it contains as much information as possible, but +not information that you don't need - and how much you need depends upon what +you're doing. When you're debugging, you need a level of information that would +be excessive and unhelpful if you had to deal with it in production. + +You can configure logging to provide you with the level of detail you need, +when you need it. Rather than manually change configuration to achieve this, a +better way is to apply configuration automatically according to the +environment. + +For example, you could set an environment variable ``DJANGO_LOG_LEVEL`` +appropriately in your development and staging environments, and make use of it +in a logger mapping thus:: + + 'level': os.getenv('DJANGO_LOG_LEVEL', 'WARNING') + +\- so that unless the environment specifies a lower log level, this +configuration will only forward records of severity ``WARNING`` and above to +its handler. + +Other options in the configuration (such as the ``level`` or ``formatter`` +option of handlers) can be similarly managed. diff --git a/docs/ref/logging.txt b/docs/ref/logging.txt index 8c682dfcebe..7fdb7791f55 100644 --- a/docs/ref/logging.txt +++ b/docs/ref/logging.txt @@ -1,9 +1,14 @@ -.. _logging_ref: +.. _logging-ref: ======= Logging ======= +.. seealso:: + + * :ref:`logging-how-to` + * :ref:`Django logging overview ` + .. module:: django.utils.log :synopsis: Logging tools for Django applications @@ -46,11 +51,17 @@ parents, up to the root ``django`` logger. The ``console`` and ``mail_admins`` handlers are attached to the root logger to provide the behavior described above. +Python's own defaults send records of level ``WARNING`` and higher +to the console. + +.. _default-logging-definition: + Default logging definition -------------------------- -The default configuration is available as ``django.utils.log.DEFAULT_LOGGING`` -and defined in :source:`django/utils/log.py`:: +Django's default logging configuration inherits Python's defaults. It's +available as ``django.utils.log.DEFAULT_LOGGING`` and defined in +:source:`django/utils/log.py`:: { 'version': 1, @@ -246,8 +257,8 @@ as explained in :ref:`django-db-logger`. Handlers -------- -Django provides one log handler in addition to those provided by the -Python logging module. +Django provides one log handler in addition to :mod:`those provided by the +Python logging module `. .. class:: AdminEmailHandler(include_html=False, email_backend=None, reporter_class=None) diff --git a/docs/topics/logging.txt b/docs/topics/logging.txt index b9a47ff3247..1e06ce356aa 100644 --- a/docs/topics/logging.txt +++ b/docs/topics/logging.txt @@ -1,10 +1,13 @@ +.. _logging-explanation: + ======= Logging ======= .. seealso:: - :ref:`Django logging reference `. + * :ref:`logging-how-to` + * :ref:`Django logging reference ` Python programmers will often use ``print()`` in their code as a quick and convenient debugging tool. Using the logging framework is only a little more @@ -163,10 +166,6 @@ and has access to the information, and so on. Configuring logging =================== -It isn't enough to just put logging calls into your code. You also need to -configure the loggers, handlers, filters, and formatters to ensure you can use -the logging output. - Python's logging library provides several techniques to configure logging, ranging from a programmatic interface to configuration files. By default, Django uses the :ref:`dictConfig format @@ -457,96 +456,3 @@ Note that the default configuration process only calls configuring the logging in your settings file will load your logging config immediately. As such, your logging config must appear *after* any settings on which it depends. - -.. _logging-how-to: - -How to use logging -================== - -Django provides a :ref:`default logging configuration -`, so you don't need to provide any additional -configuration in order to start using logging (it's the default configuration -that for example generates the messages that appear in the console when using -the :djadmin:`runserver`). - -Make a basic logging call -------------------------- - -To send a log message from within your code, you place a logging call into it. - -.. admonition:: Don't be tempted to use logging calls in ``settings.py`` - - The way that Django logging is configured as part of the ``setup()`` - function means that logging calls placed in ``settings.py`` may not work as - expected, because *logging will not be set up at that point*. To explore - logging, use a view function as suggested in the example below. - -First, import the Python logging library, and then obtain a logger instance -with :py:func:`logging.getLogger`. The ``getLogger()`` method must be provided -with a name. A good option is to use ``__name__``, which will provide the name -of the current Python module (see :ref:`naming-loggers` for use of explicit -naming):: - - import logging - - logger = logging.getLogger(__name__) - -And then in a function, for example in a view, send a message to the logger:: - - def some_view(request): - ... - if some_risky_state: - logger.warning('Platform is running at risk') - -When this code is executed, that message will be sent to the logger (and if -you're using Django's default logging configuration, it will appear in the -console). - -The ``WARNING`` level used in the example above is one of several -:ref:`logging severity levels `: ``DEBUG``, -``INFO``, ``WARNING``, ``ERROR``, ``CRITICAL``. So, another example might be:: - - logger.critical('Payment system is not responding') - -The default logging configuration, which Django inherits from the Python -logging module, prints all messages of level ``WARNING`` and higher to the -console. Django's own defaults will *not* pass ``INFO`` or lower severity -messages from applications other than Django itself to the console - that will -need to be configured explicitly. - -.. _naming-loggers: - -Name logger instances ---------------------- - -Every logger instance has a name. By convention, the logger name is usually -``__name__``, the name of the Python module in which -:func:`logging.getLogger()` is called. This allows you to filter and handle -logging calls on a per-module basis. However, if you have some other way of -organizing your logging messages, you can provide any dot-separated name to -identify your logger:: - - # Get an instance of a specific named logger - logger = logging.getLogger('project.interesting.stuff') - -.. _naming-loggers-hierarchy: - -Logger hierarchy -~~~~~~~~~~~~~~~~ - -The dotted paths of logger names define a hierarchy. The -``project.interesting`` logger is considered to be a parent of the -``project.interesting.stuff`` logger; the ``project`` logger is a parent of the -``project.interesting`` logger. (Note that this hierarchy does not need to -reflect the actual Python module hierarchy.) - -Why is the hierarchy important? Well, because loggers can be set to -*propagate* their logging calls to their parents. In this way, you can -define a single set of handlers at the root of a logger tree, and -capture all logging calls in the subtree of loggers. A logger defined -in the ``project`` namespace will catch all logging messages issued on -the ``project.interesting`` and ``project.interesting.stuff`` loggers. - -This propagation can be controlled on a per-logger basis. If -you don't want a particular logger to propagate to its parents, you -can turn off this behavior.