p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Better document xfail(condition) (#6957)

This commit is contained in:
Tim Hoffmann 2020-03-27 20:46:42 +01:00 committed by GitHub
parent 83e18776f6
commit 0e4a44db3b
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 35 additions and 22 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
doc/en/reference.rst
View File

@ -204,7 +204,8 @@ Marks a test function as *expected to fail*.
:type condition: bool or str :type condition: bool or str
:param condition: :param condition:
Condition for marking the test function as xfail (``True/False`` or a Condition for marking the test function as xfail (``True/False`` or a
:ref:`condition string <string conditions>`). :ref:`condition string <string conditions>`). If a bool, you also have
to specify ``reason`` (see :ref:`condition string <string conditions>`).
:keyword str reason: Reason why the test function is marked as xfail. :keyword str reason: Reason why the test function is marked as xfail.
:keyword Exception raises: Exception subclass expected to be raised by the test function; other exceptions will fail the test. :keyword Exception raises: Exception subclass expected to be raised by the test function; other exceptions will fail the test.
:keyword bool run: :keyword bool run:

54
doc/en/skipping.rst
View File

@ -265,33 +265,20 @@ internally by raising a known exception.
**Reference**: :ref:`pytest.mark.xfail ref` **Reference**: :ref:`pytest.mark.xfail ref`
.. _`xfail strict tutorial`: ``condition`` parameter
~~~~~~~~~~~~~~~~~~~~~~~
``strict`` parameter If a test is only expected to fail under a certain condition, you can pass
~~~~~~~~~~~~~~~~~~~~ that condition as the first parameter:
Both ``XFAIL`` and ``XPASS`` don't fail the test suite by default.
You can change this by setting the ``strict`` keyword-only parameter to ``True``:
.. code-block:: python .. code-block:: python
@pytest.mark.xfail(strict=True) @pytest.mark.xfail(sys.platform == "win32", reason="bug in a 3rd party library")
def test_function(): def test_function():
... ...
Note that you have to pass a reason as well (see the parameter description at
This will make ``XPASS`` ("unexpectedly passing") results from this test to fail the test suite. :ref:`pytest.mark.xfail ref`).
You can change the default value of the ``strict`` parameter using the
``xfail_strict`` ini option:
.. code-block:: ini
[pytest]
xfail_strict=true
``reason`` parameter ``reason`` parameter
~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~
@ -301,7 +288,7 @@ on a particular platform:
.. code-block:: python .. code-block:: python
@pytest.mark.xfail(sys.version_info >= (3, 6), reason="python3.6 api changes") @pytest.mark.xfail(reason="known parser issue")
def test_function(): def test_function():
... ...
@ -336,6 +323,31 @@ even executed, use the ``run`` parameter as ``False``:
This is specially useful for xfailing tests that are crashing the interpreter and should be This is specially useful for xfailing tests that are crashing the interpreter and should be
investigated later. investigated later.
.. _`xfail strict tutorial`:
``strict`` parameter
~~~~~~~~~~~~~~~~~~~~
Both ``XFAIL`` and ``XPASS`` don't fail the test suite by default.
You can change this by setting the ``strict`` keyword-only parameter to ``True``:
.. code-block:: python
@pytest.mark.xfail(strict=True)
def test_function():
...
This will make ``XPASS`` ("unexpectedly passing") results from this test to fail the test suite.
You can change the default value of the ``strict`` parameter using the
``xfail_strict`` ini option:
.. code-block:: ini
[pytest]
xfail_strict=true
Ignoring xfail Ignoring xfail
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~