test_ok2/doc/en/reference.rst

1601 lines
37 KiB
ReStructuredText
Raw Normal View History

.. _`reference`:
2019-07-15 22:25:33 +08:00
API Reference
=============
This page contains the full reference to pytest's API.
.. contents::
2018-02-28 04:32:34 +08:00
:depth: 3
2018-02-27 07:27:15 +08:00
:local:
2018-03-02 05:07:06 +08:00
Functions
---------
2018-02-27 07:27:15 +08:00
pytest.approx
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.python_api.approx
2018-02-28 04:15:08 +08:00
pytest.fail
~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-03-02 05:07:06 +08:00
**Tutorial**: :ref:`skipping`
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.outcomes.fail
2018-02-27 07:34:53 +08:00
2018-02-28 04:15:08 +08:00
pytest.skip
~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
.. autofunction:: _pytest.outcomes.skip(msg, [allow_module_level=False])
2018-02-27 07:34:53 +08:00
2019-08-10 03:35:03 +08:00
.. _`pytest.importorskip ref`:
2018-02-28 04:15:08 +08:00
pytest.importorskip
~~~~~~~~~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.outcomes.importorskip
2018-02-27 07:34:53 +08:00
2018-02-28 04:15:08 +08:00
pytest.xfail
~~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.outcomes.xfail
2018-02-27 07:34:53 +08:00
2018-02-28 04:15:08 +08:00
pytest.exit
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.outcomes.exit
2018-02-27 07:34:53 +08:00
2018-02-28 04:15:08 +08:00
pytest.main
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-27 07:27:15 +08:00
.. autofunction:: _pytest.config.main
2018-03-01 07:34:20 +08:00
pytest.param
~~~~~~~~~~~~
2018-03-01 07:34:20 +08:00
.. autofunction:: pytest.param(*values, [id], [marks])
2018-03-01 07:34:20 +08:00
2018-02-28 04:15:08 +08:00
pytest.raises
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~~~
2018-02-28 04:58:51 +08:00
**Tutorial**: :ref:`assertraises`.
.. autofunction:: pytest.raises(expected_exception: Exception [, *, match])
:with: excinfo
2018-02-27 07:27:15 +08:00
2018-02-28 04:15:08 +08:00
pytest.deprecated_call
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~~~~~~~~~~~~
2018-02-27 07:34:53 +08:00
2018-02-28 04:58:51 +08:00
**Tutorial**: :ref:`ensuring_function_triggers`.
.. autofunction:: pytest.deprecated_call()
:with:
pytest.register_assert_rewrite
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`assertion-rewriting`.
.. autofunction:: pytest.register_assert_rewrite
pytest.warns
2018-03-02 05:07:06 +08:00
~~~~~~~~~~~~
**Tutorial**: :ref:`assertwarnings`
.. autofunction:: pytest.warns(expected_warning: Exception, [match])
:with:
pytest.freeze_includes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`freezing-pytest`.
.. autofunction:: pytest.freeze_includes
.. _`marks ref`:
2018-03-01 07:34:20 +08:00
Marks
-----
Marks can be used apply meta data to *test functions* (but not fixtures), which can then be accessed by
fixtures or plugins.
2018-04-06 00:39:57 +08:00
.. _`pytest.mark.filterwarnings ref`:
pytest.mark.filterwarnings
~~~~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`filterwarnings`.
Add warning filters to marked test items.
.. py:function:: pytest.mark.filterwarnings(filter)
:keyword str filter:
A *warning specification string*, which is composed of contents of the tuple ``(action, message, category, module, lineno)``
as specified in `The Warnings filter <https://docs.python.org/3/library/warnings.html#warning-filter>`_ section of
the Python documentation, separated by ``":"``. Optional fields can be omitted.
Module names passed for filtering are not regex-escaped.
For example:
.. code-block:: python
2020-03-12 16:38:00 +08:00
@pytest.mark.filterwarnings("ignore:.*usage will be deprecated.*:DeprecationWarning")
def test_foo():
...
2018-03-01 07:34:20 +08:00
.. _`pytest.mark.parametrize ref`:
pytest.mark.parametrize
~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :doc:`parametrize`.
.. automethod:: _pytest.python.Metafunc.parametrize
.. _`pytest.mark.skip ref`:
pytest.mark.skip
~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`skip`.
Unconditionally skip a test function.
.. py:function:: pytest.mark.skip(*, reason=None)
:keyword str reason: Reason why the test function is being skipped.
.. _`pytest.mark.skipif ref`:
pytest.mark.skipif
~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`skipif`.
2018-03-01 07:34:20 +08:00
Skip a test function if a condition is ``True``.
.. py:function:: pytest.mark.skipif(condition, *, reason=None)
:type condition: bool or str
:param condition: ``True/False`` if the condition should be skipped or a :ref:`condition string <string conditions>`.
:keyword str reason: Reason why the test function is being skipped.
.. _`pytest.mark.usefixtures ref`:
pytest.mark.usefixtures
~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`usefixtures`.
Mark a test function as using the given fixture names.
.. warning::
This mark has no effect when applied
to a **fixture** function.
.. py:function:: pytest.mark.usefixtures(*names)
:param args: the names of the fixture to use, as strings
2018-03-01 07:34:20 +08:00
.. _`pytest.mark.xfail ref`:
pytest.mark.xfail
~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`xfail`.
Marks a test function as *expected to fail*.
.. py:function:: pytest.mark.xfail(condition=None, *, reason=None, raises=None, run=True, strict=False)
:type condition: bool or str
:param condition:
Condition for marking the test function as xfail (``True/False`` or a
:ref:`condition string <string conditions>`). If a bool, you also have
to specify ``reason`` (see :ref:`condition string <string conditions>`).
2018-03-01 07:34:20 +08:00
:keyword str reason: Reason why the test function is marked as xfail.
:keyword Exception raises: Exception subclass expected to be raised by the test function; other exceptions will fail the test.
:keyword bool run:
If the test function should actually be executed. If ``False``, the function will always xfail and will
not be executed (useful if a function is segfaulting).
2018-03-01 07:34:20 +08:00
:keyword bool strict:
* If ``False`` (the default) the function will be shown in the terminal output as ``xfailed`` if it fails
and as ``xpass`` if it passes. In both cases this will not cause the test suite to fail as a whole. This
is particularly useful to mark *flaky* tests (tests that fail at random) to be tackled later.
2018-03-01 07:34:20 +08:00
* If ``True``, the function will be shown in the terminal output as ``xfailed`` if it fails, but if it
unexpectedly passes then it will **fail** the test suite. This is particularly useful to mark functions
that are always failing and there should be a clear indication if they unexpectedly start to pass (for example
a new release of a library fixes a known bug).
custom marks
~~~~~~~~~~~~
Marks are created dynamically using the factory object ``pytest.mark`` and applied as a decorator.
For example:
.. code-block:: python
2018-06-03 11:29:28 +08:00
@pytest.mark.timeout(10, "slow", method="thread")
2018-03-01 07:34:20 +08:00
def test_function():
...
2018-04-06 00:39:57 +08:00
Will create and attach a :class:`Mark <_pytest.mark.structures.Mark>` object to the collected
2018-03-01 07:34:20 +08:00
:class:`Item <_pytest.nodes.Item>`, which can then be accessed by fixtures or hooks with
2018-04-06 00:39:57 +08:00
:meth:`Node.iter_markers <_pytest.nodes.Node.iter_markers>`. The ``mark`` object will have the following attributes:
2018-03-01 07:34:20 +08:00
.. code-block:: python
2018-06-03 11:29:28 +08:00
mark.args == (10, "slow")
mark.kwargs == {"method": "thread"}
.. _`fixtures-api`:
Fixtures
--------
2018-02-28 04:58:51 +08:00
**Tutorial**: :ref:`fixture`.
Fixtures are requested by test functions or other fixtures by declaring them as argument names.
Example of a test requiring a fixture:
.. code-block:: python
def test_output(capsys):
2018-06-03 11:29:28 +08:00
print("hello")
out, err = capsys.readouterr()
2018-06-03 11:29:28 +08:00
assert out == "hello\n"
Example of a fixture requiring another fixture:
.. code-block:: python
@pytest.fixture
def db_session(tmpdir):
2018-06-03 11:29:28 +08:00
fn = tmpdir / "db.file"
return connect(str(fn))
For more details, consult the full :ref:`fixtures docs <fixture>`.
.. _`pytest.fixture-api`:
2018-03-02 05:07:06 +08:00
@pytest.fixture
~~~~~~~~~~~~~~~
2018-03-02 05:07:06 +08:00
.. autofunction:: pytest.fixture
:decorator:
2020-03-15 08:01:50 +08:00
.. fixture:: cache
config.cache
~~~~~~~~~~~~
2018-02-28 04:58:51 +08:00
**Tutorial**: :ref:`cache`.
The ``config.cache`` object allows other plugins and fixtures
to store and retrieve values across test runs. To access it from fixtures
request ``pytestconfig`` into your fixture and get it with ``pytestconfig.cache``.
Under the hood, the cache plugin uses the simple
``dumps``/``loads`` API of the :py:mod:`json` stdlib module.
.. currentmodule:: _pytest.cacheprovider
.. automethod:: Cache.get
.. automethod:: Cache.set
.. automethod:: Cache.makedir
.. fixture:: capsys
capsys
~~~~~~
2018-02-28 04:58:51 +08:00
**Tutorial**: :doc:`capture`.
.. currentmodule:: _pytest.capture
.. autofunction:: capsys()
:no-auto-options:
Returns an instance of :py:class:`CaptureFixture`.
Example:
.. code-block:: python
def test_output(capsys):
print("hello")
captured = capsys.readouterr()
assert captured.out == "hello\n"
2018-03-02 10:33:21 +08:00
.. autoclass:: CaptureFixture()
:members:
.. fixture:: capsysbinary
capsysbinary
~~~~~~~~~~~~
2018-02-28 04:58:51 +08:00
**Tutorial**: :doc:`capture`.
.. autofunction:: capsysbinary()
:no-auto-options:
Returns an instance of :py:class:`CaptureFixture`.
Example:
.. code-block:: python
def test_output(capsysbinary):
print("hello")
captured = capsysbinary.readouterr()
assert captured.out == b"hello\n"
.. fixture:: capfd
capfd
~~~~~~
2018-02-28 04:58:51 +08:00
**Tutorial**: :doc:`capture`.
.. autofunction:: capfd()
:no-auto-options:
Returns an instance of :py:class:`CaptureFixture`.
Example:
.. code-block:: python
def test_system_echo(capfd):
os.system('echo "hello"')
2019-12-03 14:29:22 +08:00
captured = capfd.readouterr()
assert captured.out == "hello\n"
.. fixture:: capfdbinary
capfdbinary
~~~~~~~~~~~~
**Tutorial**: :doc:`capture`.
.. autofunction:: capfdbinary()
:no-auto-options:
Returns an instance of :py:class:`CaptureFixture`.
Example:
.. code-block:: python
def test_system_echo(capfdbinary):
os.system('echo "hello"')
captured = capfdbinary.readouterr()
assert captured.out == b"hello\n"
.. fixture:: doctest_namespace
doctest_namespace
~~~~~~~~~~~~~~~~~
**Tutorial**: :doc:`doctest`.
.. autofunction:: _pytest.doctest.doctest_namespace()
Usually this fixture is used in conjunction with another ``autouse`` fixture:
.. code-block:: python
@pytest.fixture(autouse=True)
def add_np(doctest_namespace):
2018-06-03 11:29:28 +08:00
doctest_namespace["np"] = numpy
For more details: :ref:`doctest_namespace`.
.. fixture:: request
request
~~~~~~~
**Tutorial**: :ref:`request example`.
The ``request`` fixture is a special fixture providing information of the requesting test function.
.. autoclass:: _pytest.fixtures.FixtureRequest()
:members:
.. fixture:: pytestconfig
pytestconfig
~~~~~~~~~~~~
.. autofunction:: _pytest.fixtures.pytestconfig()
.. fixture:: record_property
record_property
~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`record_property example`.
.. autofunction:: _pytest.junitxml.record_property()
.. fixture:: record_testsuite_property
record_testsuite_property
~~~~~~~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`record_testsuite_property example`.
.. autofunction:: _pytest.junitxml.record_testsuite_property()
.. fixture:: caplog
caplog
~~~~~~
**Tutorial**: :doc:`logging`.
.. autofunction:: _pytest.logging.caplog()
:no-auto-options:
This returns a :class:`_pytest.logging.LogCaptureFixture` instance.
.. autoclass:: _pytest.logging.LogCaptureFixture
:members:
.. fixture:: monkeypatch
monkeypatch
~~~~~~~~~~~
.. currentmodule:: _pytest.monkeypatch
**Tutorial**: :doc:`monkeypatch`.
.. autofunction:: _pytest.monkeypatch.monkeypatch()
:no-auto-options:
This returns a :class:`MonkeyPatch` instance.
.. autoclass:: _pytest.monkeypatch.MonkeyPatch
:members:
.. fixture:: testdir
testdir
~~~~~~~
.. currentmodule:: _pytest.pytester
This fixture provides a :class:`Testdir` instance useful for black-box testing of test files, making it ideal to
test plugins.
To use it, include in your top-most ``conftest.py`` file:
2019-08-07 04:25:54 +08:00
.. code-block:: python
2019-08-07 04:34:58 +08:00
pytest_plugins = "pytester"
.. autoclass:: Testdir()
2018-08-20 12:11:31 +08:00
:members:
.. autoclass:: RunResult()
:members:
.. autoclass:: LineMatcher()
:members:
.. fixture:: recwarn
recwarn
~~~~~~~
**Tutorial**: :ref:`assertwarnings`
.. currentmodule:: _pytest.recwarn
.. autofunction:: recwarn()
:no-auto-options:
.. autoclass:: _pytest.recwarn.WarningsRecorder()
:members:
Each recorded warning is an instance of :class:`warnings.WarningMessage`.
.. note::
:class:`RecordedWarning` was changed from a plain class to a namedtuple in pytest 3.1
.. note::
``DeprecationWarning`` and ``PendingDeprecationWarning`` are treated
differently; see :ref:`ensuring_function_triggers`.
.. fixture:: tmp_path
tmp_path
~~~~~~~~
**Tutorial**: :doc:`tmpdir`
.. currentmodule:: _pytest.tmpdir
.. autofunction:: tmp_path()
:no-auto-options:
.. fixture:: tmp_path_factory
tmp_path_factory
~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`tmp_path_factory example`
.. _`tmp_path_factory factory api`:
``tmp_path_factory`` instances have the following methods:
.. currentmodule:: _pytest.tmpdir
.. automethod:: TempPathFactory.mktemp
.. automethod:: TempPathFactory.getbasetemp
.. fixture:: tmpdir
tmpdir
~~~~~~
**Tutorial**: :doc:`tmpdir`
.. currentmodule:: _pytest.tmpdir
.. autofunction:: tmpdir()
:no-auto-options:
.. fixture:: tmpdir_factory
tmpdir_factory
~~~~~~~~~~~~~~
**Tutorial**: :ref:`tmpdir factory example`
.. _`tmpdir factory api`:
``tmpdir_factory`` instances have the following methods:
.. currentmodule:: _pytest.tmpdir
.. automethod:: TempdirFactory.mktemp
.. automethod:: TempdirFactory.getbasetemp
.. _`hook-reference`:
Hooks
-----
**Tutorial**: :doc:`writing_plugins`.
.. currentmodule:: _pytest.hookspec
Reference to all hooks which can be implemented by :ref:`conftest.py files <localplugin>` and :ref:`plugins <plugins>`.
Bootstrapping hooks
~~~~~~~~~~~~~~~~~~~
Bootstrapping hooks called for plugins registered early enough (internal and setuptools plugins).
.. autofunction:: pytest_load_initial_conftests
.. autofunction:: pytest_cmdline_preparse
.. autofunction:: pytest_cmdline_parse
.. autofunction:: pytest_cmdline_main
.. _`initialization-hooks`:
Initialization hooks
~~~~~~~~~~~~~~~~~~~~
Initialization hooks called for plugins and ``conftest.py`` files.
.. autofunction:: pytest_addoption
2018-03-02 04:55:45 +08:00
.. autofunction:: pytest_addhooks
.. autofunction:: pytest_configure
.. autofunction:: pytest_unconfigure
.. autofunction:: pytest_sessionstart
.. autofunction:: pytest_sessionfinish
2018-02-28 04:58:51 +08:00
.. autofunction:: pytest_plugin_registered
Test running hooks
~~~~~~~~~~~~~~~~~~
All runtest related hooks receive a :py:class:`pytest.Item <_pytest.main.Item>` object.
.. autofunction:: pytest_runtestloop
.. autofunction:: pytest_runtest_protocol
.. autofunction:: pytest_runtest_logstart
.. autofunction:: pytest_runtest_logfinish
.. autofunction:: pytest_runtest_setup
.. autofunction:: pytest_runtest_call
.. autofunction:: pytest_runtest_teardown
.. autofunction:: pytest_runtest_makereport
For deeper understanding you may look at the default implementation of
these hooks in :py:mod:`_pytest.runner` and maybe also
in :py:mod:`_pytest.pdb` which interacts with :py:mod:`_pytest.capture`
and its input/output capturing in order to immediately drop
into interactive debugging when a test failure occurs.
The :py:mod:`_pytest.terminal` reported specifically uses
the reporting hook to print information about a test run.
.. autofunction:: pytest_pyfunc_call
Collection hooks
~~~~~~~~~~~~~~~~
``pytest`` calls the following hooks for collecting files and directories:
.. autofunction:: pytest_collection
.. autofunction:: pytest_ignore_collect
.. autofunction:: pytest_collect_directory
.. autofunction:: pytest_collect_file
.. autofunction:: pytest_pycollect_makemodule
2018-02-28 04:58:51 +08:00
For influencing the collection of objects in Python modules
you can use the following hook:
.. autofunction:: pytest_pycollect_makeitem
.. autofunction:: pytest_generate_tests
.. autofunction:: pytest_make_parametrize_id
After collection is complete, you can modify the order of
items, delete or otherwise amend the test items:
.. autofunction:: pytest_collection_modifyitems
.. autofunction:: pytest_collection_finish
Reporting hooks
~~~~~~~~~~~~~~~
2018-02-27 07:27:15 +08:00
Session related reporting hooks:
2018-02-27 07:27:15 +08:00
.. autofunction:: pytest_collectstart
.. autofunction:: pytest_make_collect_report
.. autofunction:: pytest_itemcollected
.. autofunction:: pytest_collectreport
.. autofunction:: pytest_deselected
.. autofunction:: pytest_report_header
.. autofunction:: pytest_report_collectionfinish
.. autofunction:: pytest_report_teststatus
.. autofunction:: pytest_terminal_summary
.. autofunction:: pytest_fixture_setup
.. autofunction:: pytest_fixture_post_finalizer
.. autofunction:: pytest_warning_captured
2018-02-27 07:27:15 +08:00
2019-06-27 06:10:54 +08:00
Central hook for reporting about test execution:
2018-02-28 04:58:51 +08:00
.. autofunction:: pytest_runtest_logreport
2018-02-27 07:27:15 +08:00
2019-06-27 06:10:54 +08:00
Assertion related hooks:
2018-02-28 04:13:14 +08:00
.. autofunction:: pytest_assertrepr_compare
2019-06-27 06:10:54 +08:00
.. autofunction:: pytest_assertion_pass
2018-02-28 04:13:14 +08:00
Debugging/Interaction hooks
~~~~~~~~~~~~~~~~~~~~~~~~~~~
There are few hooks which can be used for special
reporting or interaction with exceptions:
.. autofunction:: pytest_internalerror
.. autofunction:: pytest_keyboard_interrupt
.. autofunction:: pytest_exception_interact
.. autofunction:: pytest_enter_pdb
2018-02-28 04:58:51 +08:00
2018-02-28 04:32:34 +08:00
Objects
-------
2018-02-28 04:32:34 +08:00
Full reference to objects accessible from :ref:`fixtures <fixture>` or :ref:`hooks <hook-reference>`.
2018-02-28 04:32:34 +08:00
CallInfo
~~~~~~~~
2018-02-28 04:32:34 +08:00
.. autoclass:: _pytest.runner.CallInfo()
2018-02-28 04:32:34 +08:00
:members:
Class
~~~~~
.. autoclass:: _pytest.python.Class()
:members:
:show-inheritance:
2018-02-28 05:03:35 +08:00
Collector
~~~~~~~~~
.. autoclass:: _pytest.nodes.Collector()
:members:
:show-inheritance:
Config
~~~~~~
.. autoclass:: _pytest.config.Config()
:members:
ExceptionInfo
~~~~~~~~~~~~~
2018-02-28 05:03:35 +08:00
.. autoclass:: _pytest._code.ExceptionInfo
:members:
2018-02-28 05:03:35 +08:00
2019-06-15 23:18:21 +08:00
pytest.ExitCode
~~~~~~~~~~~~~~~
.. autoclass:: _pytest.config.ExitCode
2019-06-15 23:18:21 +08:00
:members:
FixtureDef
~~~~~~~~~~
2018-02-28 05:03:35 +08:00
.. autoclass:: _pytest.fixtures.FixtureDef()
:members:
:show-inheritance:
2018-02-28 05:03:35 +08:00
FSCollector
~~~~~~~~~~~
2018-02-28 05:03:35 +08:00
.. autoclass:: _pytest.nodes.FSCollector()
:members:
:show-inheritance:
2018-02-28 05:03:35 +08:00
Function
~~~~~~~~
2018-02-28 05:03:35 +08:00
.. autoclass:: _pytest.python.Function()
:members:
:show-inheritance:
2018-02-28 05:03:35 +08:00
Item
~~~~
.. autoclass:: _pytest.nodes.Item()
2018-02-28 05:03:35 +08:00
:members:
:show-inheritance:
2018-02-28 05:03:35 +08:00
MarkDecorator
~~~~~~~~~~~~~
.. autoclass:: _pytest.mark.MarkDecorator
2018-02-28 05:03:35 +08:00
:members:
2018-04-06 00:39:57 +08:00
MarkGenerator
~~~~~~~~~~~~~
.. autoclass:: _pytest.mark.MarkGenerator
:members:
2018-04-06 00:39:57 +08:00
Mark
~~~~
.. autoclass:: _pytest.mark.structures.Mark
:members:
Metafunc
~~~~~~~~
.. autoclass:: _pytest.python.Metafunc
:members:
Module
~~~~~~
.. autoclass:: _pytest.python.Module()
:members:
:show-inheritance:
Node
~~~~
.. autoclass:: _pytest.nodes.Node()
:members:
Parser
~~~~~~
2018-06-11 22:20:24 +08:00
.. autoclass:: _pytest.config.argparsing.Parser()
:members:
PluginManager
~~~~~~~~~~~~~
.. autoclass:: pluggy.PluginManager()
:members:
PytestPluginManager
~~~~~~~~~~~~~~~~~~~
.. autoclass:: _pytest.config.PytestPluginManager()
:members:
:undoc-members:
:show-inheritance:
Session
~~~~~~~
.. autoclass:: _pytest.main.Session()
:members:
:show-inheritance:
TestReport
~~~~~~~~~~
.. autoclass:: _pytest.runner.TestReport()
:members:
2020-03-05 13:00:11 +08:00
:show-inheritance:
:inherited-members:
_Result
~~~~~~~
Result used within :ref:`hook wrappers <hookwrapper>`.
2018-07-30 07:50:24 +08:00
.. autoclass:: pluggy.callers._Result
.. automethod:: pluggy.callers._Result.get_result
.. automethod:: pluggy.callers._Result.force_result
Special Variables
-----------------
pytest treats some global variables in a special manner when defined in a test module.
collect_ignore
~~~~~~~~~~~~~~
**Tutorial**: :ref:`customizing-test-collection`
Can be declared in *conftest.py files* to exclude test directories or modules.
Needs to be ``list[str]``.
.. code-block:: python
collect_ignore = ["setup.py"]
collect_ignore_glob
~~~~~~~~~~~~~~~~~~~
**Tutorial**: :ref:`customizing-test-collection`
Can be declared in *conftest.py files* to exclude test directories or modules
with Unix shell-style wildcards. Needs to be ``list[str]`` where ``str`` can
contain glob patterns.
.. code-block:: python
collect_ignore_glob = ["*_ignore.py"]
pytest_plugins
~~~~~~~~~~~~~~
**Tutorial**: :ref:`available installable plugins`
Can be declared at the **global** level in *test modules* and *conftest.py files* to register additional plugins.
Can be either a ``str`` or ``Sequence[str]``.
.. code-block:: python
pytest_plugins = "myapp.testsupport.myplugin"
.. code-block:: python
pytest_plugins = ("myapp.testsupport.tools", "myapp.testsupport.regression")
2020-02-03 15:56:37 +08:00
pytestmark
~~~~~~~~~~
**Tutorial**: :ref:`scoped-marking`
Can be declared at the **global** level in *test modules* to apply one or more :ref:`marks <marks ref>` to all
Fix pytestmark syntax in reference.rst pytest 4.3.1 throws an error if `pytestmark` is set to a tuple of marks; it appears to insist on a list. With `pytestmark = [pytest.mark.api, pytest.mark.good]`: ``` ============================== test session starts ============================== platform win32 -- Python 3.6.3, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 rootdir: C:\Temp\git\sphobjinv, inifile: tox.ini plugins: timeout-1.3.3 collected 48 items / 41 deselected / 7 selected tests\test_api_good.py ....... [100%] ==================== 7 passed, 41 deselected in 0.15 seconds ==================== ``` With `pytestmark = (pytest.mark.api, pytest.mark.good)`: ``` ==================================== ERRORS ===================================== ____________________ ERROR collecting tests/test_api_good.py ____________________ env\lib\site-packages\_pytest\runner.py:226: in from_call result = func() env\lib\site-packages\_pytest\runner.py:289: in <lambda> call = CallInfo.from_call(lambda: list(collector.collect()), "collect") env\lib\site-packages\_pytest\python.py:435: in collect self._inject_setup_module_fixture() env\lib\site-packages\_pytest\python.py:447: in _inject_setup_module_fixture setup_module = _get_non_fixture_func(self.obj, "setUpModule") env\lib\site-packages\_pytest\python.py:255: in obj self.own_markers.extend(get_unpacked_marks(self.obj)) env\lib\site-packages\_pytest\mark\structures.py:244: in get_unpacked_marks return normalize_mark_list(mark_list) env\lib\site-packages\_pytest\mark\structures.py:259: in normalize_mark_list raise TypeError("got {!r} instead of Mark".format(mark)) E TypeError: got (MarkDecorator(mark=Mark(name='api', args=(), kwargs={})), MarkDecorator(mark=Mark(name='good', args=(), kwargs={}))) instead of Mark !!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!! ==================== 19 deselected, 1 error in 0.27 seconds ===================== ```
2019-03-22 03:14:28 +08:00
test functions and methods. Can be either a single mark or a list of marks.
.. code-block:: python
import pytest
2018-06-03 11:29:28 +08:00
pytestmark = pytest.mark.webtest
.. code-block:: python
import pytest
2018-06-03 11:29:28 +08:00
Fix pytestmark syntax in reference.rst pytest 4.3.1 throws an error if `pytestmark` is set to a tuple of marks; it appears to insist on a list. With `pytestmark = [pytest.mark.api, pytest.mark.good]`: ``` ============================== test session starts ============================== platform win32 -- Python 3.6.3, pytest-4.3.1, py-1.8.0, pluggy-0.9.0 rootdir: C:\Temp\git\sphobjinv, inifile: tox.ini plugins: timeout-1.3.3 collected 48 items / 41 deselected / 7 selected tests\test_api_good.py ....... [100%] ==================== 7 passed, 41 deselected in 0.15 seconds ==================== ``` With `pytestmark = (pytest.mark.api, pytest.mark.good)`: ``` ==================================== ERRORS ===================================== ____________________ ERROR collecting tests/test_api_good.py ____________________ env\lib\site-packages\_pytest\runner.py:226: in from_call result = func() env\lib\site-packages\_pytest\runner.py:289: in <lambda> call = CallInfo.from_call(lambda: list(collector.collect()), "collect") env\lib\site-packages\_pytest\python.py:435: in collect self._inject_setup_module_fixture() env\lib\site-packages\_pytest\python.py:447: in _inject_setup_module_fixture setup_module = _get_non_fixture_func(self.obj, "setUpModule") env\lib\site-packages\_pytest\python.py:255: in obj self.own_markers.extend(get_unpacked_marks(self.obj)) env\lib\site-packages\_pytest\mark\structures.py:244: in get_unpacked_marks return normalize_mark_list(mark_list) env\lib\site-packages\_pytest\mark\structures.py:259: in normalize_mark_list raise TypeError("got {!r} instead of Mark".format(mark)) E TypeError: got (MarkDecorator(mark=Mark(name='api', args=(), kwargs={})), MarkDecorator(mark=Mark(name='good', args=(), kwargs={}))) instead of Mark !!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!! ==================== 19 deselected, 1 error in 0.27 seconds ===================== ```
2019-03-22 03:14:28 +08:00
pytestmark = [pytest.mark.integration, pytest.mark.slow]
2018-03-07 07:40:07 +08:00
PYTEST_DONT_REWRITE (module docstring)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The text ``PYTEST_DONT_REWRITE`` can be add to any **module docstring** to disable
:ref:`assertion rewriting <assert introspection>` for that module.
Environment Variables
---------------------
Environment variables that can be used to change pytest's behavior.
.. envvar:: PYTEST_ADDOPTS
2018-03-07 07:40:07 +08:00
This contains a command-line (parsed by the py:mod:`shlex` module) that will be **prepended** to the command line given
by the user, see :ref:`adding default options` for more information.
.. envvar:: PYTEST_DEBUG
2018-03-07 07:40:07 +08:00
When set, pytest will print tracing and debug information.
.. envvar:: PYTEST_PLUGINS
2018-03-07 07:40:07 +08:00
Contains comma-separated list of modules that should be loaded as plugins:
.. code-block:: bash
2018-06-06 06:27:59 +08:00
export PYTEST_PLUGINS=mymodule.plugin,xdist
2018-03-07 07:40:07 +08:00
.. envvar:: PYTEST_DISABLE_PLUGIN_AUTOLOAD
When set, disables plugin auto-loading through setuptools entrypoints. Only explicitly specified plugins will be
loaded.
2018-03-07 07:40:07 +08:00
.. envvar:: PYTEST_CURRENT_TEST
2018-03-07 07:40:07 +08:00
This is not meant to be set by users, but is set by pytest internally with the name of the current test so other
processes can inspect it, see :ref:`pytest current test env` for more information.
Exceptions
----------
UsageError
~~~~~~~~~~
.. autoclass:: _pytest.config.UsageError()
.. _`ini options ref`:
Configuration Options
---------------------
Here is a list of builtin configuration options that may be written in a ``pytest.ini``, ``tox.ini`` or ``setup.cfg``
file, usually located at the root of your repository. All options must be under a ``[pytest]`` section
(``[tool:pytest]`` for ``setup.cfg`` files).
.. warning::
Usage of ``setup.cfg`` is not recommended unless for very simple use cases. ``.cfg``
files use a different parser than ``pytest.ini`` and ``tox.ini`` which might cause hard to track
down problems.
When possible, it is recommended to use the latter files to hold your pytest configuration.
2020-03-29 21:55:56 +08:00
Configuration file options may be overwritten in the command-line by using ``-o/--override-ini``, which can also be
passed multiple times. The expected format is ``name=value``. For example::
pytest -o console_output_style=classic -o cache_dir=/tmp/mycache
.. confval:: addopts
Add the specified ``OPTS`` to the set of command line arguments as if they
had been specified by the user. Example: if you have this ini file content:
.. code-block:: ini
# content of pytest.ini
[pytest]
addopts = --maxfail=2 -rf # exit after 2 failures, report fail info
2019-10-01 22:17:26 +08:00
issuing ``pytest test_hello.py`` actually means:
2019-08-07 04:25:54 +08:00
.. code-block:: bash
pytest --maxfail=2 -rf test_hello.py
Default is to add no options.
2018-03-13 06:33:34 +08:00
.. confval:: cache_dir
2018-03-13 06:33:34 +08:00
Sets a directory where stores content of cache plugin. Default directory is
``.pytest_cache`` which is created in :ref:`rootdir <rootdir>`. Directory may be
2018-03-13 06:33:34 +08:00
relative or absolute path. If setting relative path, then directory is created
relative to :ref:`rootdir <rootdir>`. Additionally path may contain environment
variables, that will be expanded. For more information about cache plugin
please refer to :ref:`cache_provider`.
2018-03-13 06:33:34 +08:00
.. confval:: confcutdir
2018-03-13 06:33:34 +08:00
Sets a directory where search upwards for ``conftest.py`` files stops.
By default, pytest will stop searching for ``conftest.py`` files upwards
from ``pytest.ini``/``tox.ini``/``setup.cfg`` of the project if any,
or up to the file-system root.
2018-03-13 06:33:34 +08:00
.. confval:: console_output_style
2018-03-13 06:33:34 +08:00
Sets the console output style while running tests:
2018-03-13 06:33:34 +08:00
* ``classic``: classic pytest output.
* ``progress``: like classic pytest output, but with a progress indicator.
* ``count``: like progress, but shows progress as the number of tests completed instead of a percent.
2018-03-13 06:33:34 +08:00
The default is ``progress``, but you can fallback to ``classic`` if you prefer or
the new mode is causing unexpected problems:
.. code-block:: ini
# content of pytest.ini
[pytest]
2018-03-13 06:33:34 +08:00
console_output_style = classic
2018-03-13 06:33:34 +08:00
.. confval:: doctest_encoding
2018-03-13 06:33:34 +08:00
Default encoding to use to decode text files with docstrings.
:doc:`See how pytest handles doctests <doctest>`.
.. confval:: doctest_optionflags
One or more doctest flag names from the standard ``doctest`` module.
:doc:`See how pytest handles doctests <doctest>`.
2018-03-13 06:33:34 +08:00
.. confval:: empty_parameter_set_mark
2018-03-13 06:33:34 +08:00
Allows to pick the action for empty parametersets in parameterization
2018-05-13 18:06:09 +08:00
* ``skip`` skips tests with an empty parameterset (default)
* ``xfail`` marks tests with an empty parameterset as xfail(run=False)
2018-09-12 18:18:44 +08:00
* ``fail_at_collect`` raises an exception if parametrize collects an empty parameter set
2018-03-13 06:33:34 +08:00
.. code-block:: ini
# content of pytest.ini
[pytest]
empty_parameter_set_mark = xfail
.. note::
The default value of this option is planned to change to ``xfail`` in future releases
as this is considered less error prone, see `#3155 <https://github.com/pytest-dev/pytest/issues/3155>`_
for more details.
.. confval:: faulthandler_timeout
Dumps the tracebacks of all threads if a test takes longer than ``X`` seconds to run (including
fixture setup and teardown). Implemented using the `faulthandler.dump_traceback_later`_ function,
so all caveats there apply.
.. code-block:: ini
# content of pytest.ini
[pytest]
faulthandler_timeout=5
For more information please refer to :ref:`faulthandler`.
.. _`faulthandler.dump_traceback_later`: https://docs.python.org/3/library/faulthandler.html#faulthandler.dump_traceback_later
.. confval:: filterwarnings
Sets a list of filters and actions that should be taken for matched
warnings. By default all warnings emitted during the test session
will be displayed in a summary at the end of the test session.
.. code-block:: ini
# content of pytest.ini
[pytest]
filterwarnings =
error
ignore::DeprecationWarning
This tells pytest to ignore deprecation warnings and turn all other warnings
into errors. For more information please refer to :ref:`warnings`.
.. confval:: junit_duration_report
.. versionadded:: 4.1
Configures how durations are recorded into the JUnit XML report:
* ``total`` (the default): duration times reported include setup, call, and teardown times.
* ``call``: duration times reported include only call times, excluding setup and teardown.
.. code-block:: ini
[pytest]
junit_duration_report = call
2019-02-08 23:10:47 +08:00
.. confval:: junit_family
.. versionadded:: 4.2
Configures the format of the generated JUnit XML file. The possible options are:
* ``xunit1`` (or ``legacy``): produces old style output, compatible with the xunit 1.0 format. **This is the default**.
* ``xunit2``: produces `xunit 2.0 style output <https://github.com/jenkinsci/xunit-plugin/blob/xunit-2.3.2/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd>`__,
which should be more compatible with latest Jenkins versions.
.. code-block:: ini
[pytest]
junit_family = xunit2
.. confval:: junit_logging
.. versionadded:: 3.5
.. versionchanged:: 5.4
``log``, ``all``, ``out-err`` options added.
Configures if captured output should be written to the JUnit XML file. Valid values are:
* ``log``: write only ``logging`` captured output.
* ``system-out``: write captured ``stdout`` contents.
* ``system-err``: write captured ``stderr`` contents.
* ``out-err``: write both captured ``stdout`` and ``stderr`` contents.
* ``all``: write captured ``logging``, ``stdout`` and ``stderr`` contents.
* ``no`` (the default): no captured output is written.
.. code-block:: ini
[pytest]
junit_logging = system-out
.. confval:: junit_log_passing_tests
.. versionadded:: 4.6
If ``junit_logging != "no"``, configures if the captured output should be written
to the JUnit XML file for **passing** tests. Default is ``True``.
.. code-block:: ini
[pytest]
junit_log_passing_tests = False
.. confval:: junit_suite_name
2018-03-13 06:33:34 +08:00
To set the name of the root test suite xml item, you can configure the ``junit_suite_name`` option in your config file:
2018-03-13 06:33:34 +08:00
.. code-block:: ini
2018-03-13 06:33:34 +08:00
[pytest]
junit_suite_name = my_suite
.. confval:: log_auto_indent
Allow selective auto-indentation of multiline log messages.
Supports command line option ``--log-auto-indent [value]``
and config option ``log_auto_indent = [value]`` to set the
auto-indentation behavior for all logging.
``[value]`` can be:
* True or "On" - Dynamically auto-indent multiline log messages
* False or "Off" or 0 - Do not auto-indent multiline log messages (the default behavior)
* [positive integer] - auto-indent multiline log messages by [value] spaces
.. code-block:: ini
[pytest]
log_auto_indent = False
Supports passing kwarg ``extra={"auto_indent": [value]}`` to
calls to ``logging.log()`` to specify auto-indentation behavior for
a specific entry in the log. ``extra`` kwarg overrides the value specified
on the command line or in the config.
2019-11-25 05:33:51 +08:00
.. confval:: log_cli
Enable log display during test run (also known as :ref:`"live logging" <live_logs>`).
The default is ``False``.
.. code-block:: ini
[pytest]
log_cli = True
2018-03-13 07:09:05 +08:00
.. confval:: log_cli_date_format
2018-03-13 07:09:05 +08:00
Sets a :py:func:`time.strftime`-compatible string that will be used when formatting dates for live logging.
.. code-block:: ini
[pytest]
log_cli_date_format = %Y-%m-%d %H:%M:%S
For more information, see :ref:`live_logs`.
.. confval:: log_cli_format
2018-03-13 07:09:05 +08:00
Sets a :py:mod:`logging`-compatible string used to format live logging messages.
.. code-block:: ini
[pytest]
log_cli_format = %(asctime)s %(levelname)s %(message)s
For more information, see :ref:`live_logs`.
.. confval:: log_cli_level
2018-03-13 07:09:05 +08:00
Sets the minimum log message level that should be captured for live logging. The integer value or
the names of the levels can be used.
.. code-block:: ini
[pytest]
log_cli_level = INFO
2018-03-13 07:09:05 +08:00
For more information, see :ref:`live_logs`.
.. confval:: log_date_format
2018-03-13 07:09:05 +08:00
Sets a :py:func:`time.strftime`-compatible string that will be used when formatting dates for logging capture.
.. code-block:: ini
[pytest]
log_date_format = %Y-%m-%d %H:%M:%S
For more information, see :ref:`logging`.
.. confval:: log_file
2018-03-13 07:09:05 +08:00
Sets a file name relative to the ``pytest.ini`` file where log messages should be written to, in addition
to the other logging facilities that are active.
.. code-block:: ini
[pytest]
log_file = logs/pytest-logs.txt
For more information, see :ref:`logging`.
.. confval:: log_file_date_format
2018-03-13 07:09:05 +08:00
Sets a :py:func:`time.strftime`-compatible string that will be used when formatting dates for the logging file.
.. code-block:: ini
[pytest]
log_file_date_format = %Y-%m-%d %H:%M:%S
For more information, see :ref:`logging`.
.. confval:: log_file_format
2018-03-13 07:09:05 +08:00
Sets a :py:mod:`logging`-compatible string used to format logging messages redirected to the logging file.
.. code-block:: ini
[pytest]
log_file_format = %(asctime)s %(levelname)s %(message)s
For more information, see :ref:`logging`.
.. confval:: log_file_level
2018-03-13 07:09:05 +08:00
Sets the minimum log message level that should be captured for the logging file. The integer value or
the names of the levels can be used.
.. code-block:: ini
[pytest]
log_file_level = INFO
For more information, see :ref:`logging`.
.. confval:: log_format
2018-03-13 07:09:05 +08:00
Sets a :py:mod:`logging`-compatible string used to format captured logging messages.
.. code-block:: ini
[pytest]
log_format = %(asctime)s %(levelname)s %(message)s
For more information, see :ref:`logging`.
.. confval:: log_level
2018-03-13 07:09:05 +08:00
Sets the minimum log message level that should be captured for logging capture. The integer value or
the names of the levels can be used.
.. code-block:: ini
[pytest]
log_level = INFO
For more information, see :ref:`logging`.
.. confval:: log_print
2018-03-13 07:09:05 +08:00
If set to ``False``, will disable displaying captured logging messages for failed tests.
.. code-block:: ini
[pytest]
log_print = False
For more information, see :ref:`logging`.
.. confval:: markers
When the ``--strict-markers`` or ``--strict`` command-line arguments are used,
only known markers - defined in code by core pytest or some plugin - are allowed.
2019-02-25 19:30:49 +08:00
You can list additional markers in this setting to add them to the whitelist,
in which case you probably want to add ``--strict-markers`` to ``addopts``
to avoid future regressions:
.. code-block:: ini
[pytest]
addopts = --strict-markers
markers =
slow
serial
2018-03-13 06:33:34 +08:00
.. confval:: minversion
2018-03-13 06:33:34 +08:00
Specifies a minimal pytest version required for running tests.
.. code-block:: ini
# content of pytest.ini
[pytest]
2018-03-13 06:33:34 +08:00
minversion = 3.0 # will fail if we run with pytest-2.8
2018-03-13 06:33:34 +08:00
.. confval:: norecursedirs
2018-03-13 06:33:34 +08:00
Set the directory basename patterns to avoid when recursing
for test discovery. The individual (fnmatch-style) patterns are
applied to the basename of a directory to decide if to recurse into it.
Pattern matching characters::
2018-03-13 06:33:34 +08:00
* matches everything
? matches any single character
[seq] matches any character in seq
[!seq] matches any char not in seq
2018-03-13 06:33:34 +08:00
Default patterns are ``'.*', 'build', 'dist', 'CVS', '_darcs', '{arch}', '*.egg', 'venv'``.
Setting a ``norecursedirs`` replaces the default. Here is an example of
how to avoid certain directories:
2018-03-13 06:33:34 +08:00
.. code-block:: ini
2018-03-13 06:33:34 +08:00
[pytest]
norecursedirs = .svn _build tmp*
2018-03-13 06:33:34 +08:00
This would tell ``pytest`` to not look into typical subversion or
sphinx-build directories or into any ``tmp`` prefixed directory.
2018-03-13 06:33:34 +08:00
Additionally, ``pytest`` will attempt to intelligently identify and ignore a
virtualenv by the presence of an activation script. Any directory deemed to
be the root of a virtual environment will not be considered during test
collection unless ``collectinvirtualenv`` is given. Note also that
``norecursedirs`` takes precedence over ``collectinvirtualenv``; e.g. if
you intend to run tests in a virtualenv with a base directory that matches
``'.*'`` you *must* override ``norecursedirs`` in addition to using the
``collectinvirtualenv`` flag.
2018-03-13 06:33:34 +08:00
.. confval:: python_classes
2018-03-13 06:33:34 +08:00
One or more name prefixes or glob-style patterns determining which classes
2018-08-22 21:37:35 +08:00
are considered for test collection. Search for multiple glob patterns by
adding a space between patterns. By default, pytest will consider any
2018-03-13 06:33:34 +08:00
class prefixed with ``Test`` as a test collection. Here is an example of how
to collect tests from classes that end in ``Suite``:
2018-03-13 06:33:34 +08:00
.. code-block:: ini
2018-03-13 06:33:34 +08:00
[pytest]
python_classes = *Suite
Note that ``unittest.TestCase`` derived classes are always collected
regardless of this option, as ``unittest``'s own collection framework is used
to collect those tests.
.. confval:: python_files
One or more Glob-style file patterns determining which python files
2018-08-22 21:37:35 +08:00
are considered as test modules. Search for multiple glob patterns by
adding a space between patterns:
2018-08-22 21:37:35 +08:00
.. code-block:: ini
[pytest]
python_files = test_*.py check_*.py example_*.py
Or one per line:
.. code-block:: ini
[pytest]
python_files =
test_*.py
check_*.py
example_*.py
By default, files matching ``test_*.py`` and ``*_test.py`` will be considered
test modules.
2018-03-13 06:33:34 +08:00
.. confval:: python_functions
One or more name prefixes or glob-patterns determining which test functions
2018-08-22 21:37:35 +08:00
and methods are considered tests. Search for multiple glob patterns by
adding a space between patterns. By default, pytest will consider any
2018-03-13 06:33:34 +08:00
function prefixed with ``test`` as a test. Here is an example of how
to collect test functions and methods that end in ``_test``:
.. code-block:: ini
[pytest]
2018-03-13 06:33:34 +08:00
python_functions = *_test
Note that this has no effect on methods that live on a ``unittest
.TestCase`` derived class, as ``unittest``'s own collection framework is used
to collect those tests.
See :ref:`change naming conventions` for more detailed examples.
.. confval:: testpaths
2018-03-13 06:33:34 +08:00
Sets list of directories that should be searched for tests when
no specific directories, files or test ids are given in the command line when
executing pytest from the :ref:`rootdir <rootdir>` directory.
Useful when all project tests are in a known location to speed up
test collection and to avoid picking up undesired tests by accident.
.. code-block:: ini
2018-03-13 06:33:34 +08:00
[pytest]
testpaths = testing doc
2018-03-13 06:33:34 +08:00
This tells pytest to only look for tests in ``testing`` and ``doc``
directories when executing from the root directory.
.. confval:: usefixtures
List of fixtures that will be applied to all test functions; this is semantically the same to apply
the ``@pytest.mark.usefixtures`` marker to all test functions.
.. code-block:: ini
[pytest]
usefixtures =
clean_db
.. confval:: xfail_strict
If set to ``True``, tests marked with ``@pytest.mark.xfail`` that actually succeed will by default fail the
test suite.
For more information, see :ref:`xfail strict tutorial`.
.. code-block:: ini
[pytest]
xfail_strict = True