parent
d46006f791
commit
6fe7069cbb
|
@ -5,14 +5,6 @@ Cache: working with cross-testrun state
|
||||||
|
|
||||||
.. versionadded:: 2.8
|
.. versionadded:: 2.8
|
||||||
|
|
||||||
.. warning::
|
|
||||||
|
|
||||||
The functionality of this core plugin was previously distributed
|
|
||||||
as a third party plugin named ``pytest-cache``. The core plugin
|
|
||||||
is compatible regarding command line options and API usage except that you
|
|
||||||
can only store/receive data between test runs that is json-serializable.
|
|
||||||
|
|
||||||
|
|
||||||
Usage
|
Usage
|
||||||
---------
|
---------
|
||||||
|
|
||||||
|
|
|
@ -38,6 +38,7 @@ Full pytest documentation
|
||||||
bash-completion
|
bash-completion
|
||||||
|
|
||||||
backwards-compatibility
|
backwards-compatibility
|
||||||
|
historical-notes
|
||||||
license
|
license
|
||||||
contributing
|
contributing
|
||||||
talks
|
talks
|
||||||
|
|
|
@ -173,14 +173,14 @@ Or to select "http" and "quick" tests::
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
If you are using expressions such as "X and Y" then both X and Y
|
If you are using expressions such as ``"X and Y"`` then both ``X`` and ``Y``
|
||||||
need to be simple non-keyword names. For example, "pass" or "from"
|
need to be simple non-keyword names. For example, ``"pass"`` or ``"from"``
|
||||||
will result in SyntaxErrors because "-k" evaluates the expression.
|
will result in SyntaxErrors because ``"-k"`` evaluates the expression using Python's ``eval`` function.
|
||||||
|
|
||||||
However, if the "-k" argument is a simple string, no such restrictions
|
However, if the ``"-k"`` argument is a simple string, no such restrictions
|
||||||
apply. Also "-k 'not STRING'" has no restrictions. You can also
|
apply. Also ``"-k 'not STRING'"`` has no restrictions. You can also
|
||||||
specify numbers like "-k 1.3" to match tests which are parametrized
|
specify numbers like ``"-k 1.3"`` to match tests which are parametrized
|
||||||
with the float "1.3".
|
with the float ``"1.3"``.
|
||||||
|
|
||||||
Registering markers
|
Registering markers
|
||||||
-------------------------------------
|
-------------------------------------
|
||||||
|
|
|
@ -109,7 +109,7 @@ Note that if you misspell a function argument or want
|
||||||
to use one that isn't available, you'll see an error
|
to use one that isn't available, you'll see an error
|
||||||
with a list of available function arguments.
|
with a list of available function arguments.
|
||||||
|
|
||||||
.. Note::
|
.. note::
|
||||||
|
|
||||||
You can always issue::
|
You can always issue::
|
||||||
|
|
||||||
|
@ -117,12 +117,6 @@ with a list of available function arguments.
|
||||||
|
|
||||||
to see available fixtures.
|
to see available fixtures.
|
||||||
|
|
||||||
In versions prior to 2.3 there was no ``@pytest.fixture`` marker
|
|
||||||
and you had to use a magic ``pytest_funcarg__NAME`` prefix
|
|
||||||
for the fixture factory. This remains and will remain supported
|
|
||||||
but is not anymore advertised as the primary means of declaring fixture
|
|
||||||
functions.
|
|
||||||
|
|
||||||
Fixtures: a prime example of dependency injection
|
Fixtures: a prime example of dependency injection
|
||||||
---------------------------------------------------
|
---------------------------------------------------
|
||||||
|
|
||||||
|
@ -292,14 +286,6 @@ the ``with`` statement ends.
|
||||||
Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the
|
Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the
|
||||||
*teardown* code (after the ``yield``) will not be called.
|
*teardown* code (after the ``yield``) will not be called.
|
||||||
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Prior to version 2.10, in order to use a ``yield`` statement to execute teardown code one
|
|
||||||
had to mark a fixture using the ``yield_fixture`` marker. From 2.10 onward, normal
|
|
||||||
fixtures can use ``yield`` directly so the ``yield_fixture`` decorator is no longer needed
|
|
||||||
and considered deprecated.
|
|
||||||
|
|
||||||
|
|
||||||
An alternative option for executing *teardown* code is to
|
An alternative option for executing *teardown* code is to
|
||||||
make use of the ``addfinalizer`` method of the `request-context`_ object to register
|
make use of the ``addfinalizer`` method of the `request-context`_ object to register
|
||||||
finalization functions.
|
finalization functions.
|
||||||
|
|
|
@ -249,15 +249,6 @@ by putting them into a ``[tool:pytest]`` section:
|
||||||
python_files = testing/*/*.py
|
python_files = testing/*/*.py
|
||||||
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
Prior to 3.0, the supported section name was ``[pytest]``. Due to how
|
|
||||||
this may collide with some distutils commands, the recommended
|
|
||||||
section name for ``setup.cfg`` files is now ``[tool:pytest]``.
|
|
||||||
|
|
||||||
Note that for ``pytest.ini`` and ``tox.ini`` files the section
|
|
||||||
name is ``[pytest]``.
|
|
||||||
|
|
||||||
|
|
||||||
Manual Integration
|
Manual Integration
|
||||||
^^^^^^^^^^^^^^^^^^
|
^^^^^^^^^^^^^^^^^^
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,177 @@
|
||||||
|
Historical Notes
|
||||||
|
================
|
||||||
|
|
||||||
|
This page lists features or behavior from previous versions of pytest which have changed over the years. They are
|
||||||
|
kept here as a historical note so users looking at old code can find documentation related to them.
|
||||||
|
|
||||||
|
cache plugin integrated into the core
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
.. versionadded:: 2.8
|
||||||
|
|
||||||
|
The functionality of the :ref:`core cache <cache>` plugin was previously distributed
|
||||||
|
as a third party plugin named ``pytest-cache``. The core plugin
|
||||||
|
is compatible regarding command line options and API usage except that you
|
||||||
|
can only store/receive data between test runs that is json-serializable.
|
||||||
|
|
||||||
|
|
||||||
|
funcargs and ``pytest_funcarg__``
|
||||||
|
---------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.3
|
||||||
|
|
||||||
|
In versions prior to 2.3 there was no ``@pytest.fixture`` marker
|
||||||
|
and you had to use a magic ``pytest_funcarg__NAME`` prefix
|
||||||
|
for the fixture factory. This remains and will remain supported
|
||||||
|
but is not anymore advertised as the primary means of declaring fixture
|
||||||
|
functions.
|
||||||
|
|
||||||
|
|
||||||
|
``@pytest.yield_fixture`` decorator
|
||||||
|
-----------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.10
|
||||||
|
|
||||||
|
Prior to version 2.10, in order to use a ``yield`` statement to execute teardown code one
|
||||||
|
had to mark a fixture using the ``yield_fixture`` marker. From 2.10 onward, normal
|
||||||
|
fixtures can use ``yield`` directly so the ``yield_fixture`` decorator is no longer needed
|
||||||
|
and considered deprecated.
|
||||||
|
|
||||||
|
|
||||||
|
``[pytest]`` header in ``setup.cfg``
|
||||||
|
------------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 3.0
|
||||||
|
|
||||||
|
Prior to 3.0, the supported section name was ``[pytest]``. Due to how
|
||||||
|
this may collide with some distutils commands, the recommended
|
||||||
|
section name for ``setup.cfg`` files is now ``[tool:pytest]``.
|
||||||
|
|
||||||
|
Note that for ``pytest.ini`` and ``tox.ini`` files the section
|
||||||
|
name is ``[pytest]``.
|
||||||
|
|
||||||
|
|
||||||
|
Applying marks to ``@pytest.mark.parametrize`` parameters
|
||||||
|
---------------------------------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 3.1
|
||||||
|
|
||||||
|
Prior to version 3.1 the supported mechanism for marking values
|
||||||
|
used the syntax::
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
@pytest.mark.parametrize("test_input,expected", [
|
||||||
|
("3+5", 8),
|
||||||
|
("2+4", 6),
|
||||||
|
pytest.mark.xfail(("6*9", 42),),
|
||||||
|
])
|
||||||
|
def test_eval(test_input, expected):
|
||||||
|
assert eval(test_input) == expected
|
||||||
|
|
||||||
|
|
||||||
|
This was an initial hack to support the feature but soon was demonstrated to be incomplete,
|
||||||
|
broken for passing functions or applying multiple marks with the same name but different parameters.
|
||||||
|
|
||||||
|
The old syntax is planned to be removed in pytest-4.0.
|
||||||
|
|
||||||
|
|
||||||
|
``@pytest.mark.parametrize`` argument names as a tuple
|
||||||
|
------------------------------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.4
|
||||||
|
|
||||||
|
In versions prior to 2.4 one needed to specify the argument
|
||||||
|
names as a tuple. This remains valid but the simpler ``"name1,name2,..."``
|
||||||
|
comma-separated-string syntax is now advertised first because
|
||||||
|
it's easier to write and produces less line noise.
|
||||||
|
|
||||||
|
|
||||||
|
setup: is now an "autouse fixture"
|
||||||
|
----------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.3
|
||||||
|
|
||||||
|
During development prior to the pytest-2.3 release the name
|
||||||
|
``pytest.setup`` was used but before the release it was renamed
|
||||||
|
and moved to become part of the general fixture mechanism,
|
||||||
|
namely :ref:`autouse fixtures`
|
||||||
|
|
||||||
|
|
||||||
|
.. _string conditions:
|
||||||
|
|
||||||
|
Conditions as strings instead of booleans
|
||||||
|
-----------------------------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.4
|
||||||
|
|
||||||
|
Prior to pytest-2.4 the only way to specify skipif/xfail conditions was
|
||||||
|
to use strings::
|
||||||
|
|
||||||
|
import sys
|
||||||
|
@pytest.mark.skipif("sys.version_info >= (3,3)")
|
||||||
|
def test_function():
|
||||||
|
...
|
||||||
|
|
||||||
|
During test function setup the skipif condition is evaluated by calling
|
||||||
|
``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains
|
||||||
|
all the module globals, and ``os`` and ``sys`` as a minimum.
|
||||||
|
|
||||||
|
Since pytest-2.4 :ref:`boolean conditions <condition booleans>` are considered preferable
|
||||||
|
because markers can then be freely imported between test modules.
|
||||||
|
With strings you need to import not only the marker but all variables
|
||||||
|
used by the marker, which violates encapsulation.
|
||||||
|
|
||||||
|
The reason for specifying the condition as a string was that ``pytest`` can
|
||||||
|
report a summary of skip conditions based purely on the condition string.
|
||||||
|
With conditions as booleans you are required to specify a ``reason`` string.
|
||||||
|
|
||||||
|
Note that string conditions will remain fully supported and you are free
|
||||||
|
to use them if you have no need for cross-importing markers.
|
||||||
|
|
||||||
|
The evaluation of a condition string in ``pytest.mark.skipif(conditionstring)``
|
||||||
|
or ``pytest.mark.xfail(conditionstring)`` takes place in a namespace
|
||||||
|
dictionary which is constructed as follows:
|
||||||
|
|
||||||
|
* the namespace is initialized by putting the ``sys`` and ``os`` modules
|
||||||
|
and the pytest ``config`` object into it.
|
||||||
|
|
||||||
|
* updated with the module globals of the test function for which the
|
||||||
|
expression is applied.
|
||||||
|
|
||||||
|
The pytest ``config`` object allows you to skip based on a test
|
||||||
|
configuration value which you might have added::
|
||||||
|
|
||||||
|
@pytest.mark.skipif("not config.getvalue('db')")
|
||||||
|
def test_function(...):
|
||||||
|
...
|
||||||
|
|
||||||
|
The equivalent with "boolean conditions" is::
|
||||||
|
|
||||||
|
@pytest.mark.skipif(not pytest.config.getvalue("db"),
|
||||||
|
reason="--db was not specified")
|
||||||
|
def test_function(...):
|
||||||
|
pass
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
You cannot use ``pytest.config.getvalue()`` in code
|
||||||
|
imported before pytest's argument parsing takes place. For example,
|
||||||
|
``conftest.py`` files are imported before command line parsing and thus
|
||||||
|
``config.getvalue()`` will not execute correctly.
|
||||||
|
|
||||||
|
``pytest.set_trace()``
|
||||||
|
----------------------
|
||||||
|
|
||||||
|
.. versionchanged:: 2.4
|
||||||
|
|
||||||
|
Previous to version 2.4 to set a break point in code one needed to use ``pytest.set_trace()``::
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
def test_function():
|
||||||
|
...
|
||||||
|
pytest.set_trace() # invoke PDB debugger and tracing
|
||||||
|
|
||||||
|
|
||||||
|
This is no longer needed and one can use the native ``import pdb;pdb.set_trace()`` call directly.
|
||||||
|
|
||||||
|
For more details see :ref:`breakpoints`.
|
|
@ -99,26 +99,6 @@ for example with the builtin ``mark.xfail``::
|
||||||
def test_eval(test_input, expected):
|
def test_eval(test_input, expected):
|
||||||
assert eval(test_input) == expected
|
assert eval(test_input) == expected
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
prior to version 3.1 the supported mechanism for marking values
|
|
||||||
used the syntax::
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
@pytest.mark.parametrize("test_input,expected", [
|
|
||||||
("3+5", 8),
|
|
||||||
("2+4", 6),
|
|
||||||
pytest.mark.xfail(("6*9", 42),),
|
|
||||||
])
|
|
||||||
def test_eval(test_input, expected):
|
|
||||||
assert eval(test_input) == expected
|
|
||||||
|
|
||||||
|
|
||||||
This was an initial hack to support the feature but soon was demonstrated to be incomplete,
|
|
||||||
broken for passing functions or applying multiple marks with the same name but different parameters.
|
|
||||||
The old syntax will be removed in pytest-4.0.
|
|
||||||
|
|
||||||
|
|
||||||
Let's run this::
|
Let's run this::
|
||||||
|
|
||||||
$ pytest
|
$ pytest
|
||||||
|
@ -143,15 +123,8 @@ To get all combinations of multiple parametrized arguments you can stack
|
||||||
def test_foo(x, y):
|
def test_foo(x, y):
|
||||||
pass
|
pass
|
||||||
|
|
||||||
This will run the test with the arguments set to x=0/y=2, x=0/y=3, x=1/y=2 and
|
This will run the test with the arguments set to ``x=0/y=2``, ``x=0/y=3``, ``x=1/y=2`` and
|
||||||
x=1/y=3.
|
``x=1/y=3``.
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
In versions prior to 2.4 one needed to specify the argument
|
|
||||||
names as a tuple. This remains valid but the simpler ``"name1,name2,..."``
|
|
||||||
comma-separated-string syntax is now advertised first because
|
|
||||||
it's easier to write and produces less line noise.
|
|
||||||
|
|
||||||
.. _`pytest_generate_tests`:
|
.. _`pytest_generate_tests`:
|
||||||
|
|
||||||
|
|
|
@ -1,10 +0,0 @@
|
||||||
|
|
||||||
setup: is now an "autouse fixture"
|
|
||||||
========================================================
|
|
||||||
|
|
||||||
During development prior to the pytest-2.3 release the name
|
|
||||||
``pytest.setup`` was used but before the release it was renamed
|
|
||||||
and moved to become part of the general fixture mechanism,
|
|
||||||
namely :ref:`autouse fixtures`
|
|
||||||
|
|
||||||
|
|
|
@ -347,64 +347,3 @@ test instances when using parametrize:
|
||||||
assert n + 1 == expected
|
assert n + 1 == expected
|
||||||
|
|
||||||
|
|
||||||
.. _string conditions:
|
|
||||||
|
|
||||||
Conditions as strings instead of booleans
|
|
||||||
-----------------------------------------
|
|
||||||
|
|
||||||
Prior to pytest-2.4 the only way to specify skipif/xfail conditions was
|
|
||||||
to use strings::
|
|
||||||
|
|
||||||
import sys
|
|
||||||
@pytest.mark.skipif("sys.version_info >= (3,3)")
|
|
||||||
def test_function():
|
|
||||||
...
|
|
||||||
|
|
||||||
During test function setup the skipif condition is evaluated by calling
|
|
||||||
``eval('sys.version_info >= (3,0)', namespace)``. The namespace contains
|
|
||||||
all the module globals, and ``os`` and ``sys`` as a minimum.
|
|
||||||
|
|
||||||
Since pytest-2.4 `condition booleans`_ are considered preferable
|
|
||||||
because markers can then be freely imported between test modules.
|
|
||||||
With strings you need to import not only the marker but all variables
|
|
||||||
used by the marker, which violates encapsulation.
|
|
||||||
|
|
||||||
The reason for specifying the condition as a string was that ``pytest`` can
|
|
||||||
report a summary of skip conditions based purely on the condition string.
|
|
||||||
With conditions as booleans you are required to specify a ``reason`` string.
|
|
||||||
|
|
||||||
Note that string conditions will remain fully supported and you are free
|
|
||||||
to use them if you have no need for cross-importing markers.
|
|
||||||
|
|
||||||
The evaluation of a condition string in ``pytest.mark.skipif(conditionstring)``
|
|
||||||
or ``pytest.mark.xfail(conditionstring)`` takes place in a namespace
|
|
||||||
dictionary which is constructed as follows:
|
|
||||||
|
|
||||||
* the namespace is initialized by putting the ``sys`` and ``os`` modules
|
|
||||||
and the pytest ``config`` object into it.
|
|
||||||
|
|
||||||
* updated with the module globals of the test function for which the
|
|
||||||
expression is applied.
|
|
||||||
|
|
||||||
The pytest ``config`` object allows you to skip based on a test
|
|
||||||
configuration value which you might have added::
|
|
||||||
|
|
||||||
@pytest.mark.skipif("not config.getvalue('db')")
|
|
||||||
def test_function(...):
|
|
||||||
...
|
|
||||||
|
|
||||||
The equivalent with "boolean conditions" is::
|
|
||||||
|
|
||||||
@pytest.mark.skipif(not pytest.config.getvalue("db"),
|
|
||||||
reason="--db was not specified")
|
|
||||||
def test_function(...):
|
|
||||||
pass
|
|
||||||
|
|
||||||
.. note::
|
|
||||||
|
|
||||||
You cannot use ``pytest.config.getvalue()`` in code
|
|
||||||
imported before pytest's argument parsing takes place. For example,
|
|
||||||
``conftest.py`` files are imported before command line parsing and thus
|
|
||||||
``config.getvalue()`` will not execute correctly.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -191,12 +191,12 @@ was executed ahead of the ``test_method``.
|
||||||
|
|
||||||
.. note::
|
.. note::
|
||||||
|
|
||||||
While pytest supports receiving fixtures via :ref:`test function arguments <funcargs>` for non-unittest test methods, ``unittest.TestCase`` methods cannot directly receive fixture
|
``unittest.TestCase`` methods cannot directly receive fixture
|
||||||
function arguments as implementing that is likely to inflict
|
arguments as implementing that is likely to inflict
|
||||||
on the ability to run general unittest.TestCase test suites.
|
on the ability to run general unittest.TestCase test suites.
|
||||||
Maybe optional support would be possible, though. If unittest finally
|
|
||||||
grows a plugin system that should help as well. In the meanwhile, the
|
The above ``usefixtures`` and ``autouse`` examples should help to mix in
|
||||||
above ``usefixtures`` and ``autouse`` examples should help to mix in
|
pytest fixtures into unittest suites.
|
||||||
pytest fixtures into unittest suites. And of course you can also start
|
|
||||||
to selectively leave away the ``unittest.TestCase`` subclassing, use
|
You can also consider incrementally leave away ``unittest.TestCase`` subclassing,
|
||||||
plain asserts and get the unlimited pytest feature set.
|
using plain asserts and get the full pytest feature set.
|
||||||
|
|
|
@ -123,22 +123,15 @@ for example::
|
||||||
>>> sys.last_value
|
>>> sys.last_value
|
||||||
AssertionError('assert result == "ok"',)
|
AssertionError('assert result == "ok"',)
|
||||||
|
|
||||||
Setting a breakpoint / aka ``set_trace()``
|
.. _breakpoints:
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
If you want to set a breakpoint and enter the ``pdb.set_trace()`` you
|
Setting breakpoints
|
||||||
can use a helper::
|
-------------------
|
||||||
|
|
||||||
import pytest
|
.. versionadded: 2.4.0
|
||||||
def test_function():
|
|
||||||
...
|
|
||||||
pytest.set_trace() # invoke PDB debugger and tracing
|
|
||||||
|
|
||||||
.. versionadded: 2.0.0
|
To set a breakpoint in your code use the native Python ``import pdb;pdb.set_trace()`` call
|
||||||
|
in your code and pytest automatically disables its output capture for that test:
|
||||||
Prior to pytest version 2.0.0 you could only enter PDB_ tracing if you disabled
|
|
||||||
capturing on the command line via ``pytest -s``. In later versions, pytest
|
|
||||||
automatically disables its output capture when you enter PDB_ tracing:
|
|
||||||
|
|
||||||
* Output capture in other tests is not affected.
|
* Output capture in other tests is not affected.
|
||||||
* Any prior test output that has already been captured and will be processed as
|
* Any prior test output that has already been captured and will be processed as
|
||||||
|
@ -148,12 +141,6 @@ automatically disables its output capture when you enter PDB_ tracing:
|
||||||
for test output occurring after you exit the interactive PDB_ tracing session
|
for test output occurring after you exit the interactive PDB_ tracing session
|
||||||
and continue with the regular test run.
|
and continue with the regular test run.
|
||||||
|
|
||||||
.. versionadded: 2.4.0
|
|
||||||
|
|
||||||
Since pytest version 2.4.0 you can also use the native Python
|
|
||||||
``import pdb;pdb.set_trace()`` call to enter PDB_ tracing without having to use
|
|
||||||
the ``pytest.set_trace()`` wrapper or explicitly disable pytest's output
|
|
||||||
capturing via ``pytest -s``.
|
|
||||||
|
|
||||||
.. _durations:
|
.. _durations:
|
||||||
|
|
||||||
|
|
|
@ -90,7 +90,7 @@ Here is how you might run it::
|
||||||
pytest test_flat.py # will not show "setting up"
|
pytest test_flat.py # will not show "setting up"
|
||||||
pytest a/test_sub.py # will show "setting up"
|
pytest a/test_sub.py # will show "setting up"
|
||||||
|
|
||||||
.. Note::
|
.. note::
|
||||||
If you have ``conftest.py`` files which do not reside in a
|
If you have ``conftest.py`` files which do not reside in a
|
||||||
python package directory (i.e. one containing an ``__init__.py``) then
|
python package directory (i.e. one containing an ``__init__.py``) then
|
||||||
"import conftest" can be ambiguous because there might be other
|
"import conftest" can be ambiguous because there might be other
|
||||||
|
|
Loading…
Reference in New Issue