Merged in bubenkoff/pytest/329-skipif-requires-expression-as-a-string (pull request #43)
re #329 add test for skipif failure when you pass boolean without the reason. add emphasize to the docs.
This commit is contained in:
commit
f7a9beaefb
|
@ -10,11 +10,11 @@ or that you expect to fail you can mark them accordingly or you
|
|||
may call helper functions during execution of setup or test functions.
|
||||
|
||||
A *skip* means that you expect your test to pass unless the environment
|
||||
(e.g. wrong Python interpreter, missing dependency) prevents it to run.
|
||||
(e.g. wrong Python interpreter, missing dependency) prevents it to run.
|
||||
And *xfail* means that your test can run but you expect it to fail
|
||||
because there is an implementation problem.
|
||||
|
||||
py.test counts and lists *skip* and *xfail* tests separately. Detailed
|
||||
py.test counts and lists *skip* and *xfail* tests separately. Detailed
|
||||
information about skipped/xfailed tests is not shown by default to avoid
|
||||
cluttering the output. You can use the ``-r`` option to see details
|
||||
corresponding to the "short" letters shown in the test progress::
|
||||
|
@ -35,15 +35,16 @@ Here is an example of marking a test function to be skipped
|
|||
when run on a Python3.3 interpreter::
|
||||
|
||||
import sys
|
||||
@pytest.mark.skipif(sys.version_info >= (3,3),
|
||||
@pytest.mark.skipif(sys.version_info >= (3,3),
|
||||
reason="requires python3.3")
|
||||
def test_function():
|
||||
...
|
||||
|
||||
During test function setup the condition ("sys.version_info >= (3,3)") is
|
||||
checked. If it evaluates to True, the test function will be skipped
|
||||
with the specified reason. Note that pytest enforces specifying a reason
|
||||
with the specified reason. Note that pytest enforces specifying a reason
|
||||
in order to report meaningful "skip reasons" (e.g. when using ``-rs``).
|
||||
If the condition is a string, it will be evaluated as python expression.
|
||||
|
||||
You can share skipif markers between modules. Consider this test module::
|
||||
|
||||
|
@ -69,7 +70,7 @@ For larger test suites it's usually a good idea to have one file
|
|||
where you define the markers which you then consistently apply
|
||||
throughout your test suite.
|
||||
|
||||
Alternatively, the pre pytest-2.4 way to specify `condition strings <condition strings>`_ instead of booleans will remain fully supported in future
|
||||
Alternatively, the pre pytest-2.4 way to specify `condition strings <condition strings>`_ instead of booleans will remain fully supported in future
|
||||
versions of pytest. It couldn't be easily used for importing markers
|
||||
between test modules so it's no longer advertised as the primary method.
|
||||
|
||||
|
@ -78,10 +79,10 @@ Skip all test functions of a class or module
|
|||
---------------------------------------------
|
||||
|
||||
As with all function :ref:`marking <mark>` you can skip test functions at the
|
||||
`whole class- or module level`_. If your code targets python2.6 or above you
|
||||
`whole class- or module level`_. If your code targets python2.6 or above you
|
||||
use the skipif decorator (and any other marker) on classes::
|
||||
|
||||
@pytest.mark.skipif(sys.platform == 'win32',
|
||||
@pytest.mark.skipif(sys.platform == 'win32',
|
||||
reason="requires windows")
|
||||
class TestPosixCalls:
|
||||
|
||||
|
@ -102,7 +103,7 @@ you can set the ``pytestmark`` attribute of a class::
|
|||
"will not be setup or run under 'win32' platform"
|
||||
|
||||
As with the class-decorator, the ``pytestmark`` special name tells
|
||||
py.test to apply it to each test function in the class.
|
||||
py.test to apply it to each test function in the class.
|
||||
|
||||
If you want to skip all test functions of a module, you must use
|
||||
the ``pytestmark`` name on the global level::
|
||||
|
@ -142,7 +143,7 @@ as if it weren't marked at all.
|
|||
As with skipif_ you can also mark your expectation of a failure
|
||||
on a particular platform::
|
||||
|
||||
@pytest.mark.xfail(sys.version_info >= (3,3),
|
||||
@pytest.mark.xfail(sys.version_info >= (3,3),
|
||||
reason="python3.3 api changes")
|
||||
def test_function():
|
||||
...
|
||||
|
@ -159,12 +160,12 @@ Running it with the report-on-xfail option gives this output::
|
|||
=========================== test session starts ============================
|
||||
platform linux2 -- Python 2.7.3 -- pytest-2.3.5
|
||||
collected 6 items
|
||||
|
||||
|
||||
xfail_demo.py xxxxxx
|
||||
========================= short test summary info ==========================
|
||||
XFAIL xfail_demo.py::test_hello
|
||||
XFAIL xfail_demo.py::test_hello2
|
||||
reason: [NOTRUN]
|
||||
reason: [NOTRUN]
|
||||
XFAIL xfail_demo.py::test_hello3
|
||||
condition: hasattr(os, 'sep')
|
||||
XFAIL xfail_demo.py::test_hello4
|
||||
|
@ -173,7 +174,7 @@ Running it with the report-on-xfail option gives this output::
|
|||
condition: pytest.__version__[0] != "17"
|
||||
XFAIL xfail_demo.py::test_hello6
|
||||
reason: reason
|
||||
|
||||
|
||||
======================== 6 xfailed in 0.05 seconds =========================
|
||||
|
||||
.. _`skip/xfail with parametrize`:
|
||||
|
@ -202,7 +203,7 @@ test instances when using parametrize:
|
|||
Imperative xfail from within a test or setup function
|
||||
------------------------------------------------------
|
||||
|
||||
If you cannot declare xfail- of skipif conditions at import
|
||||
If you cannot declare xfail- of skipif conditions at import
|
||||
time you can also imperatively produce an according outcome
|
||||
imperatively, in test or setup code::
|
||||
|
||||
|
@ -227,7 +228,7 @@ version number of a library::
|
|||
|
||||
docutils = pytest.importorskip("docutils", minversion="0.3")
|
||||
|
||||
The version will be read from the specified
|
||||
The version will be read from the specified
|
||||
module's ``__version__`` attribute.
|
||||
|
||||
|
||||
|
@ -244,18 +245,18 @@ to use strings::
|
|||
def test_function():
|
||||
...
|
||||
|
||||
During test function setup the skipif condition is evaluated by calling
|
||||
``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains
|
||||
During test function setup the skipif condition is evaluated by calling
|
||||
``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains
|
||||
all the module globals, and ``os`` and ``sys`` as a minimum.
|
||||
|
||||
Since pytest-2.4 `condition booleans`_ are considered preferable
|
||||
Since pytest-2.4 `condition booleans`_ are considered preferable
|
||||
because markers can then be freely imported between test modules.
|
||||
With strings you need to import not only the marker but all variables
|
||||
everything used by the marker, which violates encapsulation.
|
||||
|
||||
The reason for specifying the condition as a string was that py.test can
|
||||
report a summary of skip conditions based purely on the condition string.
|
||||
With conditions as booleans you are required to specify a ``reason`` string.
|
||||
report a summary of skip conditions based purely on the condition string.
|
||||
With conditions as booleans you are required to specify a ``reason`` string.
|
||||
|
||||
Note that string conditions will remain fully supported and you are free
|
||||
to use them if you have no need for cross-importing markers.
|
||||
|
@ -266,7 +267,7 @@ dictionary which is constructed as follows:
|
|||
|
||||
* the namespace is initialized by putting the ``sys`` and ``os`` modules
|
||||
and the pytest ``config`` object into it.
|
||||
|
||||
|
||||
* updated with the module globals of the test function for which the
|
||||
expression is applied.
|
||||
|
||||
|
|
|
@ -100,6 +100,18 @@ class TestEvaluator:
|
|||
expl = ev.getexplanation()
|
||||
assert expl == "condition: not hasattr(os, 'murks')"
|
||||
|
||||
def test_marked_skip_with_not_string(self, testdir):
|
||||
item = testdir.getitem("""
|
||||
import pytest
|
||||
@pytest.mark.skipif(False)
|
||||
def test_func():
|
||||
pass
|
||||
""")
|
||||
ev = MarkEvaluator(item, 'skipif')
|
||||
with pytest.raises(pytest.fail.Exception) as exc:
|
||||
assert ev.istrue()
|
||||
assert """Failed: you need to specify reason=STRING when using booleans as conditions.""" in exc.value.msg
|
||||
|
||||
def test_skipif_class(self, testdir):
|
||||
item, = testdir.getitems("""
|
||||
import pytest
|
||||
|
|
Loading…
Reference in New Issue