test_ok2/doc/en/warnings.rst

436 lines
13 KiB
ReStructuredText
Raw Normal View History

.. _`warnings`:
Warnings Capture
================
Starting from version ``3.1``, pytest now automatically catches warnings during test execution
2019-04-12 19:50:26 +08:00
and displays them at the end of the session:
.. code-block:: python
# content of test_show_warnings.py
import warnings
2019-04-12 19:50:26 +08:00
def api_v1():
warnings.warn(UserWarning("api v1, should use functions from v2"))
return 1
2019-04-12 19:50:26 +08:00
def test_one():
assert api_v1() == 1
2018-11-24 13:41:22 +08:00
Running pytest now produces this output:
.. code-block:: pytest
$ pytest test_show_warnings.py
=========================== test session starts ============================
2019-07-05 08:01:16 +08:00
platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y
2019-01-31 00:25:38 +08:00
cachedir: $PYTHON_PREFIX/.pytest_cache
2019-04-15 22:24:17 +08:00
rootdir: $REGENDOC_TMPDIR
2017-07-04 07:29:13 +08:00
collected 1 item
test_show_warnings.py . [100%]
============================= warnings summary =============================
2018-10-27 21:07:54 +08:00
test_show_warnings.py::test_one
2019-04-15 22:24:17 +08:00
$REGENDOC_TMPDIR/test_show_warnings.py:5: UserWarning: api v1, should use functions from v2
2018-10-27 21:07:54 +08:00
warnings.warn(UserWarning("api v1, should use functions from v2"))
2018-08-29 23:57:54 +08:00
-- Docs: https://docs.pytest.org/en/latest/warnings.html
2019-11-19 23:43:51 +08:00
======================= 1 passed, 1 warning in 0.12s =======================
The ``-W`` flag can be passed to control which warnings will be displayed or even turn
2018-11-24 13:41:22 +08:00
them into errors:
.. code-block:: pytest
$ pytest -q test_show_warnings.py -W error::UserWarning
F [100%]
================================= FAILURES =================================
_________________________________ test_one _________________________________
2017-05-20 06:12:59 +08:00
def test_one():
2017-05-31 05:19:34 +08:00
> assert api_v1() == 1
2019-04-15 22:24:17 +08:00
test_show_warnings.py:10:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2017-05-31 05:19:34 +08:00
def api_v1():
> warnings.warn(UserWarning("api v1, should use functions from v2"))
E UserWarning: api v1, should use functions from v2
2019-04-15 22:24:17 +08:00
test_show_warnings.py:5: UserWarning
2019-09-18 21:11:59 +08:00
1 failed in 0.12s
The same option can be set in the ``pytest.ini`` file using the ``filterwarnings`` ini option.
For example, the configuration below will ignore all user warnings, but will transform
all other warnings into errors.
.. code-block:: ini
[pytest]
filterwarnings =
error
ignore::UserWarning
When a warning matches more than one option in the list, the action for the last matching option
is performed.
Both ``-W`` command-line option and ``filterwarnings`` ini option are based on Python's own
`-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python
documentation for other examples and advanced usage.
.. _`filterwarnings`:
``@pytest.mark.filterwarnings``
-------------------------------
You can use the ``@pytest.mark.filterwarnings`` to add warning filters to specific test items,
allowing you to have finer control of which warnings should be captured at test, class or
even module level:
.. code-block:: python
import warnings
def api_v1():
warnings.warn(UserWarning("api v1, should use functions from v2"))
return 1
@pytest.mark.filterwarnings("ignore:api v1")
def test_one():
assert api_v1() == 1
Filters applied using a mark take precedence over filters passed on the command line or configured
by the ``filterwarnings`` ini option.
You may apply a filter to all tests of a class by using the ``filterwarnings`` mark as a class
decorator or to all tests in a module by setting the ``pytestmark`` variable:
.. code-block:: python
# turns all warnings into errors for this module
pytestmark = pytest.mark.filterwarnings("error")
*Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_
*plugin.*
.. _`-W option`: https://docs.python.org/3/using/cmdline.html#cmdoption-w
.. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter
.. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings
Disabling warnings summary
--------------------------
Although not recommended, you can use the ``--disable-warnings`` command-line option to suppress the
warning summary entirely from the test run output.
Disabling warning capture entirely
----------------------------------
This plugin is enabled by default but can be disabled entirely in your ``pytest.ini`` file with:
.. code-block:: ini
[pytest]
addopts = -p no:warnings
Or passing ``-p no:warnings`` in the command-line. This might be useful if your test suites handles warnings
using an external system.
.. _`deprecation-warnings`:
DeprecationWarning and PendingDeprecationWarning
------------------------------------------------
By default pytest will display ``DeprecationWarning`` and ``PendingDeprecationWarning`` warnings from
2019-01-04 08:50:36 +08:00
user code and third-party libraries, as recommended by `PEP-0565 <https://www.python.org/dev/peps/pep-0565>`_.
This helps users keep their code modern and avoid breakages when deprecated warnings are effectively removed.
Sometimes it is useful to hide some specific deprecation warnings that happen in code that you have no control over
(such as third-party libraries), in which case you might use the warning filters options (ini or marks) to ignore
those warnings.
For example:
.. code-block:: ini
[pytest]
filterwarnings =
ignore:.*U.*mode is deprecated:DeprecationWarning
This will ignore all warnings of type ``DeprecationWarning`` where the start of the message matches
the regular expression ``".*U.*mode is deprecated"``.
.. note::
2019-08-07 04:25:54 +08:00
If warnings are configured at the interpreter level, using
the `PYTHONWARNINGS <https://docs.python.org/3/using/cmdline.html#envvar-PYTHONWARNINGS>`_ environment variable or the
``-W`` command-line option, pytest will not configure any filters by default.
Also pytest doesn't follow ``PEP-0506`` suggestion of resetting all warning filters because
it might break test suites that configure warning filters themselves
by calling ``warnings.simplefilter`` (see issue `#2430 <https://github.com/pytest-dev/pytest/issues/2430>`_
for an example of that).
.. _`ensuring a function triggers a deprecation warning`:
2017-07-21 09:02:21 +08:00
.. _ensuring_function_triggers:
2017-07-21 09:02:21 +08:00
Ensuring code triggers a deprecation warning
--------------------------------------------
2017-07-21 09:02:21 +08:00
You can also call a global helper for checking
that a certain function call triggers a ``DeprecationWarning`` or
2019-04-12 19:50:26 +08:00
``PendingDeprecationWarning``:
.. code-block:: python
2017-07-21 09:02:21 +08:00
import pytest
2017-07-21 09:02:21 +08:00
2019-04-12 19:50:26 +08:00
def test_global():
pytest.deprecated_call(myfunction, 17)
By default, ``DeprecationWarning`` and ``PendingDeprecationWarning`` will not be
caught when using ``pytest.warns`` or ``recwarn`` because default Python warnings filters hide
them. If you wish to record them in your own code, use the
2019-04-12 19:50:26 +08:00
command ``warnings.simplefilter('always')``:
.. code-block:: python
import warnings
import pytest
2019-04-12 19:50:26 +08:00
def test_deprecation(recwarn):
2019-04-12 19:50:26 +08:00
warnings.simplefilter("always")
warnings.warn("deprecated", DeprecationWarning)
assert len(recwarn) == 1
assert recwarn.pop(DeprecationWarning)
2019-04-12 19:50:26 +08:00
You can also use it as a contextmanager:
.. code-block:: python
def test_global():
with pytest.deprecated_call():
myobject.deprecated_method()
.. _`asserting warnings`:
.. _assertwarnings:
.. _`asserting warnings with the warns function`:
.. _warns:
Asserting warnings with the warns function
------------------------------------------
You can check that code raises a particular warning using ``pytest.warns``,
2019-04-12 19:50:26 +08:00
which works in a similar manner to :ref:`raises <assertraises>`:
.. code-block:: python
import warnings
import pytest
2019-04-12 19:50:26 +08:00
def test_warning():
with pytest.warns(UserWarning):
warnings.warn("my warning", UserWarning)
2017-09-07 16:28:52 +08:00
The test will fail if the warning in question is not raised. The keyword
argument ``match`` to assert that the exception matches a text or regex::
>>> with warns(UserWarning, match='must be 0 or None'):
... warnings.warn("value must be 0 or None", UserWarning)
>>> with warns(UserWarning, match=r'must be \d+$'):
... warnings.warn("value must be 42", UserWarning)
>>> with warns(UserWarning, match=r'must be \d+$'):
... warnings.warn("this is not here", UserWarning)
Traceback (most recent call last):
...
Failed: DID NOT WARN. No warnings of type ...UserWarning... was emitted...
You can also call ``pytest.warns`` on a function or code string:
2019-08-07 04:25:54 +08:00
.. code-block:: python
pytest.warns(expected_warning, func, *args, **kwargs)
pytest.warns(expected_warning, "func(*args, **kwargs)")
The function also returns a list of all raised warnings (as
``warnings.WarningMessage`` objects), which you can query for
2019-04-12 19:50:26 +08:00
additional information:
.. code-block:: python
with pytest.warns(RuntimeWarning) as record:
warnings.warn("another warning", RuntimeWarning)
# check that only one warning was raised
assert len(record) == 1
# check that the message matches
assert record[0].message.args[0] == "another warning"
Alternatively, you can examine raised warnings in detail using the
:ref:`recwarn <recwarn>` fixture (see below).
.. note::
``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated
differently; see :ref:`ensuring_function_triggers`.
.. _`recording warnings`:
.. _recwarn:
Recording warnings
------------------
You can record raised warnings either using ``pytest.warns`` or with
the ``recwarn`` fixture.
To record with ``pytest.warns`` without asserting anything about the warnings,
2019-04-12 19:50:26 +08:00
pass ``None`` as the expected warning type:
.. code-block:: python
with pytest.warns(None) as record:
warnings.warn("user", UserWarning)
warnings.warn("runtime", RuntimeWarning)
assert len(record) == 2
assert str(record[0].message) == "user"
assert str(record[1].message) == "runtime"
2019-04-12 19:50:26 +08:00
The ``recwarn`` fixture will record warnings for the whole function:
.. code-block:: python
import warnings
2019-04-12 19:50:26 +08:00
def test_hello(recwarn):
warnings.warn("hello", UserWarning)
assert len(recwarn) == 1
w = recwarn.pop(UserWarning)
assert issubclass(w.category, UserWarning)
assert str(w.message) == "hello"
assert w.filename
assert w.lineno
Both ``recwarn`` and ``pytest.warns`` return the same interface for recorded
warnings: a WarningsRecorder instance. To view the recorded warnings, you can
iterate over this instance, call ``len`` on it to get the number of recorded
warnings, or index into it to get a particular recorded warning.
.. currentmodule:: _pytest.warnings
Full API: :class:`WarningsRecorder`.
.. _custom_failure_messages:
Custom failure messages
-----------------------
Recording warnings provides an opportunity to produce custom test
failure messages for when no warnings are issued or other conditions
are met.
.. code-block:: python
def test():
with pytest.warns(Warning) as record:
f()
if not record:
pytest.fail("Expected a warning!")
If no warnings are issued when calling ``f``, then ``not record`` will
evaluate to ``True``. You can then call ``pytest.fail`` with a
custom error message.
.. _internal-warnings:
Internal pytest warnings
------------------------
pytest may generate its own warnings in some situations, such as improper usage or deprecated features.
For example, pytest will emit a warning if it encounters a class that matches :confval:`python_classes` but also
defines an ``__init__`` constructor, as this prevents the class from being instantiated:
.. code-block:: python
# content of test_pytest_warnings.py
class Test:
def __init__(self):
pass
def test_foo(self):
assert 1 == 1
2018-11-24 13:41:22 +08:00
.. code-block:: pytest
$ pytest test_pytest_warnings.py -q
2018-09-06 05:06:32 +08:00
============================= warnings summary =============================
2018-10-27 21:07:54 +08:00
test_pytest_warnings.py:1
2019-06-01 00:01:05 +08:00
$REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestCollectionWarning: cannot collect test class 'Test' because it has a __init__ constructor (from: test_pytest_warnings.py)
2018-10-27 21:07:54 +08:00
class Test:
2018-09-06 05:06:32 +08:00
-- Docs: https://docs.pytest.org/en/latest/warnings.html
2019-11-19 23:43:51 +08:00
1 warning in 0.12s
These warnings might be filtered using the same builtin mechanisms used to filter other types of warnings.
Please read our :ref:`backwards-compatibility` to learn how we proceed about deprecating and eventually removing
features.
2019-08-07 04:25:54 +08:00
The following warning types are used by pytest and are part of the public API:
.. autoclass:: pytest.PytestWarning
.. autoclass:: pytest.PytestAssertRewriteWarning
.. autoclass:: pytest.PytestCacheWarning
.. autoclass:: pytest.PytestCollectionWarning
.. autoclass:: pytest.PytestConfigWarning
.. autoclass:: pytest.PytestDeprecationWarning
.. autoclass:: pytest.PytestExperimentalApiWarning
.. autoclass:: pytest.PytestUnhandledCoroutineWarning
.. autoclass:: pytest.PytestUnknownMarkWarning