p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Regendocs

This commit is contained in:
Bruno Oliveira 2019-01-05 17:32:16 +00:00
parent 5a8e674e92
commit d0e9b4812f
20 changed files with 532 additions and 384 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

30
doc/en/assert.rst
View File

@ -27,15 +27,17 @@ you will see the return value of the function call:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_assert1.py $ pytest test_assert1.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_assert1.py F [100%] test_assert1.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ test_function _______________________________ ___________________________________ test_function ____________________________________
def test_function(): def test_function():
> assert f() == 4 > assert f() == 4
@ -43,7 +45,7 @@ you will see the return value of the function call:
E + where 3 = f() E + where 3 = f()
test_assert1.py:5: AssertionError test_assert1.py:5: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
``pytest`` has support for showing the values of the most common subexpressions ``pytest`` has support for showing the values of the most common subexpressions
including calls, attributes, comparisons, and binary and unary including calls, attributes, comparisons, and binary and unary
@ -171,15 +173,17 @@ if you run this module:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_assert2.py $ pytest test_assert2.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_assert2.py F [100%] test_assert2.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
___________________________ test_set_comparison ____________________________ ________________________________ test_set_comparison _________________________________
def test_set_comparison(): def test_set_comparison():
set1 = set("1308") set1 = set("1308")
@ -193,7 +197,7 @@ if you run this module:
E Use -v to get the full diff E Use -v to get the full diff
test_assert2.py:5: AssertionError test_assert2.py:5: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
Special comparisons are done for a number of cases: Special comparisons are done for a number of cases:
@ -243,9 +247,9 @@ the conftest file:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_foocompare.py $ pytest -q test_foocompare.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_compare _______________________________ ____________________________________ test_compare ____________________________________
def test_compare(): def test_compare():
f1 = Foo(1) f1 = Foo(1)

58
doc/en/cache.rst
View File

@ -48,9 +48,9 @@ If you run this for the first time you will see two failures:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
.................F.......F........................ [100%] .................F.......F........................ [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_num[17] _______________________________ ____________________________________ test_num[17] ____________________________________
i = 17 i = 17
@ -61,7 +61,7 @@ If you run this for the first time you will see two failures:
E Failed: bad luck E Failed: bad luck
test_50.py:6: Failed test_50.py:6: Failed
_______________________________ test_num[25] _______________________________ ____________________________________ test_num[25] ____________________________________
i = 25 i = 25
@ -79,16 +79,18 @@ If you then run it with ``--lf``:
.. code-block:: pytest .. code-block:: pytest
$ pytest --lf $ pytest --lf
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 50 items / 48 deselected collected 50 items / 48 deselected
run-last-failure: rerun previous 2 failures run-last-failure: rerun previous 2 failures
test_50.py FF [100%] test_50.py FF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_num[17] _______________________________ ____________________________________ test_num[17] ____________________________________
i = 17 i = 17
@ -99,7 +101,7 @@ If you then run it with ``--lf``:
E Failed: bad luck E Failed: bad luck
test_50.py:6: Failed test_50.py:6: Failed
_______________________________ test_num[25] _______________________________ ____________________________________ test_num[25] ____________________________________
i = 25 i = 25
@ -110,7 +112,7 @@ If you then run it with ``--lf``:
E Failed: bad luck E Failed: bad luck
test_50.py:6: Failed test_50.py:6: Failed
================= 2 failed, 48 deselected in 0.12 seconds ================== ====================== 2 failed, 48 deselected in 0.12 seconds =======================
You have run only the two failing test from the last run, while 48 tests have You have run only the two failing test from the last run, while 48 tests have
not been run ("deselected"). not been run ("deselected").
@ -122,16 +124,18 @@ of ``FF`` and dots):
.. code-block:: pytest .. code-block:: pytest
$ pytest --ff $ pytest --ff
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 50 items collected 50 items
run-last-failure: rerun previous 2 failures first run-last-failure: rerun previous 2 failures first
test_50.py FF................................................ [100%] test_50.py FF................................................ [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_num[17] _______________________________ ____________________________________ test_num[17] ____________________________________
i = 17 i = 17
@ -142,7 +146,7 @@ of ``FF`` and dots):
E Failed: bad luck E Failed: bad luck
test_50.py:6: Failed test_50.py:6: Failed
_______________________________ test_num[25] _______________________________ ____________________________________ test_num[25] ____________________________________
i = 25 i = 25
@ -153,7 +157,7 @@ of ``FF`` and dots):
E Failed: bad luck E Failed: bad luck
test_50.py:6: Failed test_50.py:6: Failed
=================== 2 failed, 48 passed in 0.12 seconds ==================== ======================== 2 failed, 48 passed in 0.12 seconds =========================
.. _`config.cache`: .. _`config.cache`:
@ -205,9 +209,9 @@ If you run this command for the first time, you can see the print statement:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ test_function _______________________________ ___________________________________ test_function ____________________________________
mydata = 42 mydata = 42
@ -216,7 +220,7 @@ If you run this command for the first time, you can see the print statement:
E assert 42 == 23 E assert 42 == 23
test_caching.py:17: AssertionError test_caching.py:17: AssertionError
-------------------------- Captured stdout setup --------------------------- ------------------------------- Captured stdout setup --------------------------------
running expensive computation... running expensive computation...
1 failed in 0.12 seconds 1 failed in 0.12 seconds
@ -226,9 +230,9 @@ the cache and nothing will be printed:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ test_function _______________________________ ___________________________________ test_function ____________________________________
mydata = 42 mydata = 42
@ -251,11 +255,13 @@ You can always peek at the content of the cache using the
.. code-block:: pytest .. code-block:: pytest
$ pytest --cache-show $ pytest --cache-show
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
cachedir: $REGENDOC_TMPDIR/.pytest_cache cachedir: $REGENDOC_TMPDIR/.pytest_cache
------------------------------- cache values ------------------------------- ------------------------------------ cache values ------------------------------------
cache/lastfailed contains: cache/lastfailed contains:
{'test_caching.py::test_function': True} {'test_caching.py::test_function': True}
cache/nodeids contains: cache/nodeids contains:
@ -265,7 +271,7 @@ You can always peek at the content of the cache using the
example/value contains: example/value contains:
42 42
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
Clearing Cache content Clearing Cache content
------------------------------- -------------------------------

14
doc/en/capture.rst
View File

@ -66,24 +66,26 @@ of the failing function and hide the other one:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py .F [100%] test_module.py .F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________________ test_func2 ________________________________ _____________________________________ test_func2 _____________________________________
def test_func2(): def test_func2():
> assert False > assert False
E assert False E assert False
test_module.py:9: AssertionError test_module.py:9: AssertionError
-------------------------- Captured stdout setup --------------------------- ------------------------------- Captured stdout setup --------------------------------
setting up <function test_func2 at 0xdeadbeef> setting up <function test_func2 at 0xdeadbeef>
==================== 1 failed, 1 passed in 0.12 seconds ==================== ========================= 1 failed, 1 passed in 0.12 seconds =========================
Accessing captured output from a test function Accessing captured output from a test function
--------------------------------------------------- ---------------------------------------------------

8
doc/en/doctest.rst
View File

@ -63,14 +63,16 @@ then you can just invoke ``pytest`` without command line options:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
mymodule.py . [100%] mymodule.py . [100%]
========================= 1 passed in 0.12 seconds ========================= ============================== 1 passed in 0.12 seconds ==============================
It is possible to use fixtures using the ``getfixture`` helper:: It is possible to use fixtures using the ``getfixture`` helper::

144
doc/en/example/markers.rst
View File

@ -32,32 +32,36 @@ You can then restrict a test run to only run tests marked with ``webtest``:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v -m webtest $ pytest -v -m webtest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 4 items / 3 deselected collecting ... collected 4 items / 3 deselected
test_server.py::test_send_http PASSED [100%] test_server.py::test_send_http PASSED [100%]
================== 1 passed, 3 deselected in 0.12 seconds ================== ======================= 1 passed, 3 deselected in 0.12 seconds =======================
Or the inverse, running all tests except the webtest ones: Or the inverse, running all tests except the webtest ones:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v -m "not webtest" $ pytest -v -m "not webtest"
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 4 items / 1 deselected collecting ... collected 4 items / 1 deselected
test_server.py::test_something_quick PASSED [ 33%] test_server.py::test_something_quick PASSED [ 33%]
test_server.py::test_another PASSED [ 66%] test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%] test_server.py::TestClass::test_method PASSED [100%]
================== 3 passed, 1 deselected in 0.12 seconds ================== ======================= 3 passed, 1 deselected in 0.12 seconds =======================
Selecting tests based on their node ID Selecting tests based on their node ID
-------------------------------------- --------------------------------------
@ -69,46 +73,52 @@ tests based on their module, class, method, or function name:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v test_server.py::TestClass::test_method $ pytest -v test_server.py::TestClass::test_method
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 1 item collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%] test_server.py::TestClass::test_method PASSED [100%]
========================= 1 passed in 0.12 seconds ========================= ============================== 1 passed in 0.12 seconds ==============================
You can also select on the class: You can also select on the class:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v test_server.py::TestClass $ pytest -v test_server.py::TestClass
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 1 item collecting ... collected 1 item
test_server.py::TestClass::test_method PASSED [100%] test_server.py::TestClass::test_method PASSED [100%]
========================= 1 passed in 0.12 seconds ========================= ============================== 1 passed in 0.12 seconds ==============================
Or select multiple nodes: Or select multiple nodes:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v test_server.py::TestClass test_server.py::test_send_http $ pytest -v test_server.py::TestClass test_server.py::test_send_http
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 2 items collecting ... collected 2 items
test_server.py::TestClass::test_method PASSED [ 50%] test_server.py::TestClass::test_method PASSED [ 50%]
test_server.py::test_send_http PASSED [100%] test_server.py::test_send_http PASSED [100%]
========================= 2 passed in 0.12 seconds ========================= ============================== 2 passed in 0.12 seconds ==============================
.. _node-id: .. _node-id:
@ -139,48 +149,54 @@ select tests based on their names:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v -k http # running with the above defined example module $ pytest -v -k http # running with the above defined example module
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 4 items / 3 deselected collecting ... collected 4 items / 3 deselected
test_server.py::test_send_http PASSED [100%] test_server.py::test_send_http PASSED [100%]
================== 1 passed, 3 deselected in 0.12 seconds ================== ======================= 1 passed, 3 deselected in 0.12 seconds =======================
And you can also run all tests except the ones that match the keyword: And you can also run all tests except the ones that match the keyword:
.. code-block:: pytest .. code-block:: pytest
$ pytest -k "not send_http" -v $ pytest -k "not send_http" -v
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 4 items / 1 deselected collecting ... collected 4 items / 1 deselected
test_server.py::test_something_quick PASSED [ 33%] test_server.py::test_something_quick PASSED [ 33%]
test_server.py::test_another PASSED [ 66%] test_server.py::test_another PASSED [ 66%]
test_server.py::TestClass::test_method PASSED [100%] test_server.py::TestClass::test_method PASSED [100%]
================== 3 passed, 1 deselected in 0.12 seconds ================== ======================= 3 passed, 1 deselected in 0.12 seconds =======================
Or to select "http" and "quick" tests: Or to select "http" and "quick" tests:
.. code-block:: pytest .. code-block:: pytest
$ pytest -k "http or quick" -v $ pytest -k "http or quick" -v
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 4 items / 2 deselected collecting ... collected 4 items / 2 deselected
test_server.py::test_send_http PASSED [ 50%] test_server.py::test_send_http PASSED [ 50%]
test_server.py::test_something_quick PASSED [100%] test_server.py::test_something_quick PASSED [100%]
================== 2 passed, 2 deselected in 0.12 seconds ================== ======================= 2 passed, 2 deselected in 0.12 seconds =======================
.. note:: .. note::
@ -216,6 +232,8 @@ You can ask which markers exist for your test suite - the list includes our just
$ pytest --markers $ pytest --markers
@pytest.mark.webtest: mark a test as a webtest. @pytest.mark.webtest: mark a test as a webtest.
@pytest.mark.hypothesis: Tests which use hypothesis.
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/warnings.html#pytest-mark-filterwarnings @pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test. @pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@ -363,34 +381,40 @@ the test needs:
.. code-block:: pytest .. code-block:: pytest
$ pytest -E stage2 $ pytest -E stage2
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_someenv.py s [100%] test_someenv.py s [100%]
======================== 1 skipped in 0.12 seconds ========================= ============================= 1 skipped in 0.12 seconds ==============================
and here is one that specifies exactly the environment needed: and here is one that specifies exactly the environment needed:
.. code-block:: pytest .. code-block:: pytest
$ pytest -E stage1 $ pytest -E stage1
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_someenv.py . [100%] test_someenv.py . [100%]
========================= 1 passed in 0.12 seconds ========================= ============================== 1 passed in 0.12 seconds ==============================
The ``--markers`` option always gives you a list of available markers:: The ``--markers`` option always gives you a list of available markers::
$ pytest --markers $ pytest --markers
@pytest.mark.env(name): mark test to run only on named environment @pytest.mark.env(name): mark test to run only on named environment
@pytest.mark.hypothesis: Tests which use hypothesis.
@pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/warnings.html#pytest-mark-filterwarnings @pytest.mark.filterwarnings(warning): add a warning filter to the given test. see https://docs.pytest.org/en/latest/warnings.html#pytest-mark-filterwarnings
@pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test. @pytest.mark.skip(reason=None): skip the given test function with an optional reason. Example: skip(reason="no way of currently testing this") skips the test.
@ -544,30 +568,34 @@ then you will see two tests skipped and two executed tests as expected:
.. code-block:: pytest .. code-block:: pytest
$ pytest -rs # this option reports skip reasons $ pytest -rs # this option reports skip reasons
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items collected 4 items
test_plat.py s.s. [100%] test_plat.py s.s. [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
SKIP [2] $REGENDOC_TMPDIR/conftest.py:12: cannot run on platform linux SKIP [2] $REGENDOC_TMPDIR/conftest.py:12: cannot run on platform linux
=================== 2 passed, 2 skipped in 0.12 seconds ==================== ======================== 2 passed, 2 skipped in 0.12 seconds =========================
Note that if you specify a platform via the marker-command line option like this: Note that if you specify a platform via the marker-command line option like this:
.. code-block:: pytest .. code-block:: pytest
$ pytest -m linux $ pytest -m linux
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items / 3 deselected collected 4 items / 3 deselected
test_plat.py . [100%] test_plat.py . [100%]
================== 1 passed, 3 deselected in 0.12 seconds ================== ======================= 1 passed, 3 deselected in 0.12 seconds =======================
then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests. then the unmarked-tests will not be run. It is thus a way to restrict the run to the specific tests.
@ -613,47 +641,51 @@ We can now use the ``-m option`` to select one set:
.. code-block:: pytest .. code-block:: pytest
$ pytest -m interface --tb=short $ pytest -m interface --tb=short
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items / 2 deselected collected 4 items / 2 deselected
test_module.py FF [100%] test_module.py FF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
__________________________ test_interface_simple ___________________________ _______________________________ test_interface_simple ________________________________
test_module.py:3: in test_interface_simple test_module.py:3: in test_interface_simple
assert 0 assert 0
E assert 0 E assert 0
__________________________ test_interface_complex __________________________ _______________________________ test_interface_complex _______________________________
test_module.py:6: in test_interface_complex test_module.py:6: in test_interface_complex
assert 0 assert 0
E assert 0 E assert 0
================== 2 failed, 2 deselected in 0.12 seconds ================== ======================= 2 failed, 2 deselected in 0.12 seconds =======================
or to select both "event" and "interface" tests: or to select both "event" and "interface" tests:
.. code-block:: pytest .. code-block:: pytest
$ pytest -m "interface or event" --tb=short $ pytest -m "interface or event" --tb=short
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items / 1 deselected collected 4 items / 1 deselected
test_module.py FFF [100%] test_module.py FFF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
__________________________ test_interface_simple ___________________________ _______________________________ test_interface_simple ________________________________
test_module.py:3: in test_interface_simple test_module.py:3: in test_interface_simple
assert 0 assert 0
E assert 0 E assert 0
__________________________ test_interface_complex __________________________ _______________________________ test_interface_complex _______________________________
test_module.py:6: in test_interface_complex test_module.py:6: in test_interface_complex
assert 0 assert 0
E assert 0 E assert 0
____________________________ test_event_simple _____________________________ _________________________________ test_event_simple __________________________________
test_module.py:9: in test_event_simple test_module.py:9: in test_event_simple
assert 0 assert 0
E assert 0 E assert 0
================== 3 failed, 1 deselected in 0.12 seconds ================== ======================= 3 failed, 1 deselected in 0.12 seconds =======================

32
doc/en/example/nonpython.rst
View File

@ -28,19 +28,21 @@ now execute the test specification:
.. code-block:: pytest .. code-block:: pytest
nonpython $ pytest test_simple.yml nonpython $ pytest test_simple.yml
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/nonpython/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR/nonpython, inifile: rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_simple.yml F. [100%] test_simple.yml F. [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ usecase: hello ______________________________ ___________________________________ usecase: hello ___________________________________
usecase execution failed usecase execution failed
spec failed: 'some': 'other' spec failed: 'some': 'other'
no further details known at this point. no further details known at this point.
==================== 1 failed, 1 passed in 0.12 seconds ==================== ========================= 1 failed, 1 passed in 0.12 seconds =========================
.. regendoc:wipe .. regendoc:wipe
@ -62,21 +64,23 @@ consulted when reporting in ``verbose`` mode:
.. code-block:: pytest .. code-block:: pytest
nonpython $ pytest -v nonpython $ pytest -v
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/nonpython/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR/nonpython, inifile: rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 2 items collecting ... collected 2 items
test_simple.yml::hello FAILED [ 50%] test_simple.yml::hello FAILED [ 50%]
test_simple.yml::ok PASSED [100%] test_simple.yml::ok PASSED [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ usecase: hello ______________________________ ___________________________________ usecase: hello ___________________________________
usecase execution failed usecase execution failed
spec failed: 'some': 'other' spec failed: 'some': 'other'
no further details known at this point. no further details known at this point.
==================== 1 failed, 1 passed in 0.12 seconds ==================== ========================= 1 failed, 1 passed in 0.12 seconds =========================
.. regendoc:wipe .. regendoc:wipe
@ -86,13 +90,15 @@ interesting to just look at the collection tree:
.. code-block:: pytest .. code-block:: pytest
nonpython $ pytest --collect-only nonpython $ pytest --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/nonpython/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR/nonpython, inifile: rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
<Package $REGENDOC_TMPDIR/nonpython> <Package $REGENDOC_TMPDIR/nonpython>
<YamlFile test_simple.yml> <YamlFile test_simple.yml>
<YamlItem hello> <YamlItem hello>
<YamlItem ok> <YamlItem ok>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================

78
doc/en/example/parametrize.rst
View File

@ -47,7 +47,7 @@ This means that we only run 2 tests if we do not pass ``--all``:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_compute.py $ pytest -q test_compute.py
.. [100%] .. [100%]
2 passed in 0.12 seconds 2 passed in 0.12 seconds
We run only two computations, so we see two dots. We run only two computations, so we see two dots.
@ -56,9 +56,9 @@ let's run the full monty:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q --all $ pytest -q --all
....F [100%] ....F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_____________________________ test_compute[4] ______________________________ __________________________________ test_compute[4] ___________________________________
param1 = 4 param1 = 4
@ -143,9 +143,11 @@ objects, they are still using the default pytest representation:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_time.py --collect-only $ pytest test_time.py --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 8 items collected 8 items
<Module test_time.py> <Module test_time.py>
<Function test_timedistance_v0[a0-b0-expected0]> <Function test_timedistance_v0[a0-b0-expected0]>
@ -157,7 +159,7 @@ objects, they are still using the default pytest representation:
<Function test_timedistance_v3[forward]> <Function test_timedistance_v3[forward]>
<Function test_timedistance_v3[backward]> <Function test_timedistance_v3[backward]>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
In ``test_timedistance_v3``, we used ``pytest.param`` to specify the test IDs In ``test_timedistance_v3``, we used ``pytest.param`` to specify the test IDs
together with the actual data, instead of listing them separately. together with the actual data, instead of listing them separately.
@ -201,23 +203,27 @@ this is a fully self-contained example which you can run with:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_scenarios.py $ pytest test_scenarios.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items collected 4 items
test_scenarios.py .... [100%] test_scenarios.py .... [100%]
========================= 4 passed in 0.12 seconds ========================= ============================== 4 passed in 0.12 seconds ==============================
If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function: If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function:
.. code-block:: pytest .. code-block:: pytest
$ pytest --collect-only test_scenarios.py $ pytest --collect-only test_scenarios.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items collected 4 items
<Module test_scenarios.py> <Module test_scenarios.py>
<Class TestSampleWithScenarios> <Class TestSampleWithScenarios>
@ -226,7 +232,7 @@ If you just collect tests you'll also nicely see 'advanced' and 'basic' as varia
<Function test_demo1[advanced]> <Function test_demo1[advanced]>
<Function test_demo2[advanced]> <Function test_demo2[advanced]>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
Note that we told ``metafunc.parametrize()`` that your scenario values Note that we told ``metafunc.parametrize()`` that your scenario values
should be considered class-scoped. With pytest-2.3 this leads to a should be considered class-scoped. With pytest-2.3 this leads to a
@ -281,24 +287,26 @@ Let's first see how it looks like at collection time:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_backends.py --collect-only $ pytest test_backends.py --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
<Module test_backends.py> <Module test_backends.py>
<Function test_db_initialized[d1]> <Function test_db_initialized[d1]>
<Function test_db_initialized[d2]> <Function test_db_initialized[d2]>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
And then when we run the test: And then when we run the test:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_backends.py $ pytest -q test_backends.py
.F [100%] .F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_________________________ test_db_initialized[d2] __________________________ ______________________________ test_db_initialized[d2] _______________________________
db = <conftest.DB2 object at 0xdeadbeef> db = <conftest.DB2 object at 0xdeadbeef>
@ -346,14 +354,16 @@ The result of this test will be successful:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_indirect_list.py --collect-only $ pytest test_indirect_list.py --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
<Module test_indirect_list.py> <Module test_indirect_list.py>
<Function test_indirect[a-b]> <Function test_indirect[a-b]>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. regendoc:wipe .. regendoc:wipe
@ -397,9 +407,9 @@ argument sets to use for each test function. Let's run it:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
F.. [100%] F.. [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________ TestClass.test_equals[1-2] ________________________ _____________________________ TestClass.test_equals[1-2] _____________________________
self = <test_parametrize.TestClass object at 0xdeadbeef>, a = 1, b = 2 self = <test_parametrize.TestClass object at 0xdeadbeef>, a = 1, b = 2
@ -429,8 +439,8 @@ Running it results in some skips if we don't have all the python interpreters in
.. code-block:: pytest .. code-block:: pytest
. $ pytest -rs -q multipython.py . $ pytest -rs -q multipython.py
...sss...sssssssss...sss... [100%] ...sss...sssssssss...sss... [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
SKIP [15] $REGENDOC_TMPDIR/CWD/multipython.py:30: 'python3.4' not found SKIP [15] $REGENDOC_TMPDIR/CWD/multipython.py:30: 'python3.4' not found
12 passed, 15 skipped in 0.12 seconds 12 passed, 15 skipped in 0.12 seconds
@ -480,16 +490,18 @@ If you run this with reporting for skips enabled:
.. code-block:: pytest .. code-block:: pytest
$ pytest -rs test_module.py $ pytest -rs test_module.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py .s [100%] test_module.py .s [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
SKIP [1] $REGENDOC_TMPDIR/conftest.py:11: could not import 'opt2' SKIP [1] $REGENDOC_TMPDIR/conftest.py:11: could not import 'opt2'
=================== 1 passed, 1 skipped in 0.12 seconds ==================== ======================== 1 passed, 1 skipped in 0.12 seconds =========================
You'll see that we don't have an ``opt2`` module and thus the second test run You'll see that we don't have an ``opt2`` module and thus the second test run
of our ``test_func1`` was skipped. A few notes: of our ``test_func1`` was skipped. A few notes:
@ -537,17 +549,19 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v -m basic $ pytest -v -m basic
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 17 items / 14 deselected collecting ... collected 17 items / 14 deselected
test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%] test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%]
test_pytest_param_example.py::test_eval[basic_2+4] PASSED [ 66%] test_pytest_param_example.py::test_eval[basic_2+4] PASSED [ 66%]
test_pytest_param_example.py::test_eval[basic_6*9] xfail [100%] test_pytest_param_example.py::test_eval[basic_6*9] xfail [100%]
============ 2 passed, 14 deselected, 1 xfailed in 0.12 seconds ============ ================= 2 passed, 14 deselected, 1 xfailed in 0.12 seconds =================
As the result: As the result:

18
doc/en/example/pythoncollection.rst
View File

@ -130,16 +130,18 @@ The test collection would look like this:
.. code-block:: pytest .. code-block:: pytest
$ pytest --collect-only $ pytest --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
<Module check_myapp.py> <Module check_myapp.py>
<Class CheckMyApp> <Class CheckMyApp>
<Function simple_check> <Function simple_check>
<Function complex_check> <Function complex_check>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
You can check for multiple glob patterns by adding a space between the patterns:: You can check for multiple glob patterns by adding a space between the patterns::
@ -185,9 +187,11 @@ You can always peek at the collection tree without running tests like this:
.. code-block:: pytest .. code-block:: pytest
. $ pytest --collect-only pythoncollection.py . $ pytest --collect-only pythoncollection.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/CWD/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
plugins: hypothesis-3.x.y
collected 3 items collected 3 items
<Module CWD/pythoncollection.py> <Module CWD/pythoncollection.py>
<Function test_function> <Function test_function>
@ -195,7 +199,7 @@ You can always peek at the collection tree without running tests like this:
<Function test_method> <Function test_method>
<Function test_anothermethod> <Function test_anothermethod>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. _customizing-test-collection: .. _customizing-test-collection:
@ -257,9 +261,11 @@ file will be left out:
.. code-block:: pytest .. code-block:: pytest
$ pytest --collect-only $ pytest --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================

128
doc/en/example/reportingdemo.rst
View File

@ -12,15 +12,17 @@ get on the terminal - we are working on that):
.. code-block:: pytest .. code-block:: pytest
assertion $ pytest failure_demo.py assertion $ pytest failure_demo.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/assertion/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR/assertion, inifile: rootdir: $REGENDOC_TMPDIR/assertion, inifile:
plugins: hypothesis-3.x.y
collected 44 items collected 44 items
failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [100%] failure_demo.py FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
___________________________ test_generative[3-6] ___________________________ ________________________________ test_generative[3-6] ________________________________
param1 = 3, param2 = 6 param1 = 3, param2 = 6
@ -30,7 +32,7 @@ get on the terminal - we are working on that):
E assert (3 * 2) < 6 E assert (3 * 2) < 6
failure_demo.py:22: AssertionError failure_demo.py:22: AssertionError
_________________________ TestFailing.test_simple __________________________ ______________________________ TestFailing.test_simple _______________________________
self = <failure_demo.TestFailing object at 0xdeadbeef> self = <failure_demo.TestFailing object at 0xdeadbeef>
@ -47,7 +49,7 @@ get on the terminal - we are working on that):
E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef>() E + and 43 = <function TestFailing.test_simple.<locals>.g at 0xdeadbeef>()
failure_demo.py:33: AssertionError failure_demo.py:33: AssertionError
____________________ TestFailing.test_simple_multiline _____________________ _________________________ TestFailing.test_simple_multiline __________________________
self = <failure_demo.TestFailing object at 0xdeadbeef> self = <failure_demo.TestFailing object at 0xdeadbeef>
@ -55,7 +57,7 @@ get on the terminal - we are working on that):
> otherfunc_multi(42, 6 * 9) > otherfunc_multi(42, 6 * 9)
failure_demo.py:36: failure_demo.py:36:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = 42, b = 54 a = 42, b = 54
@ -64,7 +66,7 @@ get on the terminal - we are working on that):
E assert 42 == 54 E assert 42 == 54
failure_demo.py:17: AssertionError failure_demo.py:17: AssertionError
___________________________ TestFailing.test_not ___________________________ ________________________________ TestFailing.test_not ________________________________
self = <failure_demo.TestFailing object at 0xdeadbeef> self = <failure_demo.TestFailing object at 0xdeadbeef>
@ -77,7 +79,7 @@ get on the terminal - we are working on that):
E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef>() E + where 42 = <function TestFailing.test_not.<locals>.f at 0xdeadbeef>()
failure_demo.py:42: AssertionError failure_demo.py:42: AssertionError
_________________ TestSpecialisedExplanations.test_eq_text _________________ ______________________ TestSpecialisedExplanations.test_eq_text ______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -88,7 +90,7 @@ get on the terminal - we are working on that):
E + eggs E + eggs
failure_demo.py:47: AssertionError failure_demo.py:47: AssertionError
_____________ TestSpecialisedExplanations.test_eq_similar_text _____________ __________________ TestSpecialisedExplanations.test_eq_similar_text __________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -101,7 +103,7 @@ get on the terminal - we are working on that):
E ? ^ E ? ^
failure_demo.py:50: AssertionError failure_demo.py:50: AssertionError
____________ TestSpecialisedExplanations.test_eq_multiline_text ____________ _________________ TestSpecialisedExplanations.test_eq_multiline_text _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -114,7 +116,7 @@ get on the terminal - we are working on that):
E bar E bar
failure_demo.py:53: AssertionError failure_demo.py:53: AssertionError
______________ TestSpecialisedExplanations.test_eq_long_text _______________ ___________________ TestSpecialisedExplanations.test_eq_long_text ____________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -131,7 +133,7 @@ get on the terminal - we are working on that):
E ? ^ E ? ^
failure_demo.py:58: AssertionError failure_demo.py:58: AssertionError
_________ TestSpecialisedExplanations.test_eq_long_text_multiline __________ ______________ TestSpecialisedExplanations.test_eq_long_text_multiline _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -151,7 +153,7 @@ get on the terminal - we are working on that):
E ...Full output truncated (7 lines hidden), use '-vv' to show E ...Full output truncated (7 lines hidden), use '-vv' to show
failure_demo.py:63: AssertionError failure_demo.py:63: AssertionError
_________________ TestSpecialisedExplanations.test_eq_list _________________ ______________________ TestSpecialisedExplanations.test_eq_list ______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -162,7 +164,7 @@ get on the terminal - we are working on that):
E Use -v to get the full diff E Use -v to get the full diff
failure_demo.py:66: AssertionError failure_demo.py:66: AssertionError
______________ TestSpecialisedExplanations.test_eq_list_long _______________ ___________________ TestSpecialisedExplanations.test_eq_list_long ____________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -175,7 +177,7 @@ get on the terminal - we are working on that):
E Use -v to get the full diff E Use -v to get the full diff
failure_demo.py:71: AssertionError failure_demo.py:71: AssertionError
_________________ TestSpecialisedExplanations.test_eq_dict _________________ ______________________ TestSpecialisedExplanations.test_eq_dict ______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -193,7 +195,7 @@ get on the terminal - we are working on that):
E ...Full output truncated (2 lines hidden), use '-vv' to show E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:74: AssertionError failure_demo.py:74: AssertionError
_________________ TestSpecialisedExplanations.test_eq_set __________________ ______________________ TestSpecialisedExplanations.test_eq_set _______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -211,7 +213,7 @@ get on the terminal - we are working on that):
E ...Full output truncated (2 lines hidden), use '-vv' to show E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:77: AssertionError failure_demo.py:77: AssertionError
_____________ TestSpecialisedExplanations.test_eq_longer_list ______________ __________________ TestSpecialisedExplanations.test_eq_longer_list ___________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -222,7 +224,7 @@ get on the terminal - we are working on that):
E Use -v to get the full diff E Use -v to get the full diff
failure_demo.py:80: AssertionError failure_demo.py:80: AssertionError
_________________ TestSpecialisedExplanations.test_in_list _________________ ______________________ TestSpecialisedExplanations.test_in_list ______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -231,7 +233,7 @@ get on the terminal - we are working on that):
E assert 1 in [0, 2, 3, 4, 5] E assert 1 in [0, 2, 3, 4, 5]
failure_demo.py:83: AssertionError failure_demo.py:83: AssertionError
__________ TestSpecialisedExplanations.test_not_in_text_multiline __________ _______________ TestSpecialisedExplanations.test_not_in_text_multiline _______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -250,7 +252,7 @@ get on the terminal - we are working on that):
E ...Full output truncated (2 lines hidden), use '-vv' to show E ...Full output truncated (2 lines hidden), use '-vv' to show
failure_demo.py:87: AssertionError failure_demo.py:87: AssertionError
___________ TestSpecialisedExplanations.test_not_in_text_single ____________ ________________ TestSpecialisedExplanations.test_not_in_text_single _________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -263,7 +265,7 @@ get on the terminal - we are working on that):
E ? +++ E ? +++
failure_demo.py:91: AssertionError failure_demo.py:91: AssertionError
_________ TestSpecialisedExplanations.test_not_in_text_single_long _________ ______________ TestSpecialisedExplanations.test_not_in_text_single_long ______________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -276,7 +278,7 @@ get on the terminal - we are working on that):
E ? +++ E ? +++
failure_demo.py:95: AssertionError failure_demo.py:95: AssertionError
______ TestSpecialisedExplanations.test_not_in_text_single_long_term _______ ___________ TestSpecialisedExplanations.test_not_in_text_single_long_term ____________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -289,16 +291,28 @@ get on the terminal - we are working on that):
E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ E ? ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
failure_demo.py:99: AssertionError failure_demo.py:99: AssertionError
______________ TestSpecialisedExplanations.test_eq_dataclass _______________ ___________________ TestSpecialisedExplanations.test_eq_dataclass ____________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
def test_eq_dataclass(self): def test_eq_dataclass(self):
> from dataclasses import dataclass from dataclasses import dataclass
E ModuleNotFoundError: No module named 'dataclasses'
failure_demo.py:102: ModuleNotFoundError @dataclass
________________ TestSpecialisedExplanations.test_eq_attrs _________________ class Foo(object):
a: int
b: str
left = Foo(1, "b")
right = Foo(1, "c")
> assert left == right
E AssertionError: assert TestSpecialis...oo(a=1, b='b') == TestSpecialise...oo(a=1, b='c')
E Omitting 1 identical items, use -vv to show
E Differing attributes:
E b: 'b' != 'c'
failure_demo.py:111: AssertionError
_____________________ TestSpecialisedExplanations.test_eq_attrs ______________________
self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef> self = <failure_demo.TestSpecialisedExplanations object at 0xdeadbeef>
@ -319,7 +333,7 @@ get on the terminal - we are working on that):
E b: 'b' != 'c' E b: 'b' != 'c'
failure_demo.py:123: AssertionError failure_demo.py:123: AssertionError
______________________________ test_attribute ______________________________ ___________________________________ test_attribute ___________________________________
def test_attribute(): def test_attribute():
class Foo(object): class Foo(object):
@ -331,7 +345,7 @@ get on the terminal - we are working on that):
E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef>.b E + where 1 = <failure_demo.test_attribute.<locals>.Foo object at 0xdeadbeef>.b
failure_demo.py:131: AssertionError failure_demo.py:131: AssertionError
_________________________ test_attribute_instance __________________________ ______________________________ test_attribute_instance _______________________________
def test_attribute_instance(): def test_attribute_instance():
class Foo(object): class Foo(object):
@ -343,7 +357,7 @@ get on the terminal - we are working on that):
E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>() E + where <failure_demo.test_attribute_instance.<locals>.Foo object at 0xdeadbeef> = <class 'failure_demo.test_attribute_instance.<locals>.Foo'>()
failure_demo.py:138: AssertionError failure_demo.py:138: AssertionError
__________________________ test_attribute_failure __________________________ _______________________________ test_attribute_failure _______________________________
def test_attribute_failure(): def test_attribute_failure():
class Foo(object): class Foo(object):
@ -356,7 +370,7 @@ get on the terminal - we are working on that):
> assert i.b == 2 > assert i.b == 2
failure_demo.py:149: failure_demo.py:149:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <failure_demo.test_attribute_failure.<locals>.Foo object at 0xdeadbeef> self = <failure_demo.test_attribute_failure.<locals>.Foo object at 0xdeadbeef>
@ -365,7 +379,7 @@ get on the terminal - we are working on that):
E Exception: Failed to get attrib E Exception: Failed to get attrib
failure_demo.py:144: Exception failure_demo.py:144: Exception
_________________________ test_attribute_multiple __________________________ ______________________________ test_attribute_multiple _______________________________
def test_attribute_multiple(): def test_attribute_multiple():
class Foo(object): class Foo(object):
@ -382,7 +396,7 @@ get on the terminal - we are working on that):
E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>() E + where <failure_demo.test_attribute_multiple.<locals>.Bar object at 0xdeadbeef> = <class 'failure_demo.test_attribute_multiple.<locals>.Bar'>()
failure_demo.py:159: AssertionError failure_demo.py:159: AssertionError
__________________________ TestRaises.test_raises __________________________ _______________________________ TestRaises.test_raises _______________________________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -392,7 +406,7 @@ get on the terminal - we are working on that):
E ValueError: invalid literal for int() with base 10: 'qwe' E ValueError: invalid literal for int() with base 10: 'qwe'
failure_demo.py:169: ValueError failure_demo.py:169: ValueError
______________________ TestRaises.test_raises_doesnt _______________________ ___________________________ TestRaises.test_raises_doesnt ____________________________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -401,7 +415,7 @@ get on the terminal - we are working on that):
E Failed: DID NOT RAISE <class 'OSError'> E Failed: DID NOT RAISE <class 'OSError'>
failure_demo.py:172: Failed failure_demo.py:172: Failed
__________________________ TestRaises.test_raise ___________________________ _______________________________ TestRaises.test_raise ________________________________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -410,7 +424,7 @@ get on the terminal - we are working on that):
E ValueError: demo error E ValueError: demo error
failure_demo.py:175: ValueError failure_demo.py:175: ValueError
________________________ TestRaises.test_tupleerror ________________________ _____________________________ TestRaises.test_tupleerror _____________________________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -419,7 +433,7 @@ get on the terminal - we are working on that):
E ValueError: not enough values to unpack (expected 2, got 1) E ValueError: not enough values to unpack (expected 2, got 1)
failure_demo.py:178: ValueError failure_demo.py:178: ValueError
______ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ______ ___________ TestRaises.test_reinterpret_fails_with_print_for_the_fun_of_it ___________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -430,9 +444,9 @@ get on the terminal - we are working on that):
E TypeError: 'int' object is not iterable E TypeError: 'int' object is not iterable
failure_demo.py:183: TypeError failure_demo.py:183: TypeError
--------------------------- Captured stdout call --------------------------- -------------------------------- Captured stdout call --------------------------------
items is [1, 2, 3] items is [1, 2, 3]
________________________ TestRaises.test_some_error ________________________ _____________________________ TestRaises.test_some_error _____________________________
self = <failure_demo.TestRaises object at 0xdeadbeef> self = <failure_demo.TestRaises object at 0xdeadbeef>
@ -441,7 +455,7 @@ get on the terminal - we are working on that):
E NameError: name 'namenotexi' is not defined E NameError: name 'namenotexi' is not defined
failure_demo.py:186: NameError failure_demo.py:186: NameError
____________________ test_dynamic_compile_shows_nicely _____________________ _________________________ test_dynamic_compile_shows_nicely __________________________
def test_dynamic_compile_shows_nicely(): def test_dynamic_compile_shows_nicely():
import imp import imp
@ -456,14 +470,14 @@ get on the terminal - we are working on that):
> module.foo() > module.foo()
failure_demo.py:204: failure_demo.py:204:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def foo(): def foo():
> assert 1 == 0 > assert 1 == 0
E AssertionError E AssertionError
<0-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:201>:2: AssertionError <0-codegen 'abc-123' $REGENDOC_TMPDIR/assertion/failure_demo.py:201>:2: AssertionError
____________________ TestMoreErrors.test_complex_error _____________________ _________________________ TestMoreErrors.test_complex_error __________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -477,10 +491,10 @@ get on the terminal - we are working on that):
> somefunc(f(), g()) > somefunc(f(), g())
failure_demo.py:215: failure_demo.py:215:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
failure_demo.py:13: in somefunc failure_demo.py:13: in somefunc
otherfunc(x, y) otherfunc(x, y)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
a = 44, b = 43 a = 44, b = 43
@ -489,7 +503,7 @@ get on the terminal - we are working on that):
E assert 44 == 43 E assert 44 == 43
failure_demo.py:9: AssertionError failure_demo.py:9: AssertionError
___________________ TestMoreErrors.test_z1_unpack_error ____________________ ________________________ TestMoreErrors.test_z1_unpack_error _________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -499,7 +513,7 @@ get on the terminal - we are working on that):
E ValueError: not enough values to unpack (expected 2, got 0) E ValueError: not enough values to unpack (expected 2, got 0)
failure_demo.py:219: ValueError failure_demo.py:219: ValueError
____________________ TestMoreErrors.test_z2_type_error _____________________ _________________________ TestMoreErrors.test_z2_type_error __________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -509,7 +523,7 @@ get on the terminal - we are working on that):
E TypeError: 'int' object is not iterable E TypeError: 'int' object is not iterable
failure_demo.py:223: TypeError failure_demo.py:223: TypeError
______________________ TestMoreErrors.test_startswith ______________________ ___________________________ TestMoreErrors.test_startswith ___________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -522,7 +536,7 @@ get on the terminal - we are working on that):
E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith E + where <built-in method startswith of str object at 0xdeadbeef> = '123'.startswith
failure_demo.py:228: AssertionError failure_demo.py:228: AssertionError
__________________ TestMoreErrors.test_startswith_nested ___________________ _______________________ TestMoreErrors.test_startswith_nested ________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -541,7 +555,7 @@ get on the terminal - we are working on that):
E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef>() E + and '456' = <function TestMoreErrors.test_startswith_nested.<locals>.g at 0xdeadbeef>()
failure_demo.py:237: AssertionError failure_demo.py:237: AssertionError
_____________________ TestMoreErrors.test_global_func ______________________ __________________________ TestMoreErrors.test_global_func ___________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -552,7 +566,7 @@ get on the terminal - we are working on that):
E + where 43 = globf(42) E + where 43 = globf(42)
failure_demo.py:240: AssertionError failure_demo.py:240: AssertionError
_______________________ TestMoreErrors.test_instance _______________________ ____________________________ TestMoreErrors.test_instance ____________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -563,7 +577,7 @@ get on the terminal - we are working on that):
E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef>.x E + where 42 = <failure_demo.TestMoreErrors object at 0xdeadbeef>.x
failure_demo.py:244: AssertionError failure_demo.py:244: AssertionError
_______________________ TestMoreErrors.test_compare ________________________ ____________________________ TestMoreErrors.test_compare _____________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -573,7 +587,7 @@ get on the terminal - we are working on that):
E + where 11 = globf(10) E + where 11 = globf(10)
failure_demo.py:247: AssertionError failure_demo.py:247: AssertionError
_____________________ TestMoreErrors.test_try_finally ______________________ __________________________ TestMoreErrors.test_try_finally ___________________________
self = <failure_demo.TestMoreErrors object at 0xdeadbeef> self = <failure_demo.TestMoreErrors object at 0xdeadbeef>
@ -584,7 +598,7 @@ get on the terminal - we are working on that):
E assert 1 == 0 E assert 1 == 0
failure_demo.py:252: AssertionError failure_demo.py:252: AssertionError
___________________ TestCustomAssertMsg.test_single_line ___________________ ________________________ TestCustomAssertMsg.test_single_line ________________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef> self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@ -599,7 +613,7 @@ get on the terminal - we are working on that):
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_single_line.<locals>.A'>.a E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_single_line.<locals>.A'>.a
failure_demo.py:263: AssertionError failure_demo.py:263: AssertionError
____________________ TestCustomAssertMsg.test_multiline ____________________ _________________________ TestCustomAssertMsg.test_multiline _________________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef> self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@ -618,7 +632,7 @@ get on the terminal - we are working on that):
E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_multiline.<locals>.A'>.a E + where 1 = <class 'failure_demo.TestCustomAssertMsg.test_multiline.<locals>.A'>.a
failure_demo.py:270: AssertionError failure_demo.py:270: AssertionError
___________________ TestCustomAssertMsg.test_custom_repr ___________________ ________________________ TestCustomAssertMsg.test_custom_repr ________________________
self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef> self = <failure_demo.TestCustomAssertMsg object at 0xdeadbeef>
@ -640,4 +654,4 @@ get on the terminal - we are working on that):
E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a E + where 1 = This is JSON\n{\n 'foo': 'bar'\n}.a
failure_demo.py:283: AssertionError failure_demo.py:283: AssertionError
======================== 44 failed in 0.12 seconds ========================= ============================= 44 failed in 0.12 seconds ==============================

144
doc/en/example/simple.rst
View File

@ -48,9 +48,9 @@ Let's run this without supplying our new option:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_sample.py $ pytest -q test_sample.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_answer ________________________________ ____________________________________ test_answer _____________________________________
cmdopt = 'type1' cmdopt = 'type1'
@ -63,7 +63,7 @@ Let's run this without supplying our new option:
E assert 0 E assert 0
test_sample.py:6: AssertionError test_sample.py:6: AssertionError
--------------------------- Captured stdout call --------------------------- -------------------------------- Captured stdout call --------------------------------
first first
1 failed in 0.12 seconds 1 failed in 0.12 seconds
@ -72,9 +72,9 @@ And now with supplying a command line option:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q --cmdopt=type2 $ pytest -q --cmdopt=type2
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_answer ________________________________ ____________________________________ test_answer _____________________________________
cmdopt = 'type2' cmdopt = 'type2'
@ -87,7 +87,7 @@ And now with supplying a command line option:
E assert 0 E assert 0
test_sample.py:6: AssertionError test_sample.py:6: AssertionError
--------------------------- Captured stdout call --------------------------- -------------------------------- Captured stdout call --------------------------------
second second
1 failed in 0.12 seconds 1 failed in 0.12 seconds
@ -126,12 +126,14 @@ directory with the above conftest.py:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. _`excontrolskip`: .. _`excontrolskip`:
@ -186,30 +188,34 @@ and when running it will see a skipped "slow" test:
.. code-block:: pytest .. code-block:: pytest
$ pytest -rs # "-rs" means report details on the little 's' $ pytest -rs # "-rs" means report details on the little 's'
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py .s [100%] test_module.py .s [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
SKIP [1] test_module.py:8: need --runslow option to run SKIP [1] test_module.py:8: need --runslow option to run
=================== 1 passed, 1 skipped in 0.12 seconds ==================== ======================== 1 passed, 1 skipped in 0.12 seconds =========================
Or run it including the ``slow`` marked test: Or run it including the ``slow`` marked test:
.. code-block:: pytest .. code-block:: pytest
$ pytest --runslow $ pytest --runslow
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py .. [100%] test_module.py .. [100%]
========================= 2 passed in 0.12 seconds ========================= ============================== 2 passed in 0.12 seconds ==============================
Writing well integrated assertion helpers Writing well integrated assertion helpers
-------------------------------------------------- --------------------------------------------------
@ -245,9 +251,9 @@ Let's run our little function:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_checkconfig.py $ pytest -q test_checkconfig.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ test_something ______________________________ ___________________________________ test_something ___________________________________
def test_something(): def test_something():
> checkconfig(42) > checkconfig(42)
@ -344,13 +350,15 @@ which will add the string to the test header accordingly:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
project deps: mylib-1.1 project deps: mylib-1.1
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. regendoc:wipe .. regendoc:wipe
@ -372,27 +380,31 @@ which will add info only when run with "--v":
.. code-block:: pytest .. code-block:: pytest
$ pytest -v $ pytest -v
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
info1: did you know that ... info1: did you know that ...
did you? did you?
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 0 items collecting ... collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
and nothing when run plainly: and nothing when run plainly:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
profiling test duration profiling test duration
-------------------------- --------------------------
@ -426,18 +438,20 @@ Now we can profile which test functions execute the slowest:
.. code-block:: pytest .. code-block:: pytest
$ pytest --durations=3 $ pytest --durations=3
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 3 items collected 3 items
test_some_are_slow.py ... [100%] test_some_are_slow.py ... [100%]
========================= slowest 3 test durations ========================= ============================== slowest 3 test durations ==============================
0.30s call test_some_are_slow.py::test_funcslow2 0.30s call test_some_are_slow.py::test_funcslow2
0.20s call test_some_are_slow.py::test_funcslow1 0.20s call test_some_are_slow.py::test_funcslow1
0.10s call test_some_are_slow.py::test_funcfast 0.10s call test_some_are_slow.py::test_funcfast
========================= 3 passed in 0.12 seconds ========================= ============================== 3 passed in 0.12 seconds ==============================
incremental testing - test steps incremental testing - test steps
--------------------------------------------------- ---------------------------------------------------
@ -500,15 +514,17 @@ If we run this:
.. code-block:: pytest .. code-block:: pytest
$ pytest -rx $ pytest -rx
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 4 items collected 4 items
test_step.py .Fx. [100%] test_step.py .Fx. [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
____________________ TestUserHandling.test_modification ____________________ _________________________ TestUserHandling.test_modification _________________________
self = <test_step.TestUserHandling object at 0xdeadbeef> self = <test_step.TestUserHandling object at 0xdeadbeef>
@ -517,10 +533,10 @@ If we run this:
E assert 0 E assert 0
test_step.py:11: AssertionError test_step.py:11: AssertionError
========================= short test summary info ========================== ============================== short test summary info ===============================
XFAIL test_step.py::TestUserHandling::test_deletion XFAIL test_step.py::TestUserHandling::test_deletion
reason: previous test failed (test_modification) reason: previous test failed (test_modification)
============== 1 failed, 2 passed, 1 xfailed in 0.12 seconds =============== =================== 1 failed, 2 passed, 1 xfailed in 0.12 seconds ====================
We'll see that ``test_deletion`` was not executed because ``test_modification`` We'll see that ``test_deletion`` was not executed because ``test_modification``
failed. It is reported as an "expected failure". failed. It is reported as an "expected failure".
@ -583,18 +599,20 @@ We can run this:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 7 items collected 7 items
test_step.py .Fx. [ 57%] test_step.py .Fx. [ 57%]
a/test_db.py F [ 71%] a/test_db.py F [ 71%]
a/test_db2.py F [ 85%] a/test_db2.py F [ 85%]
b/test_error.py E [100%] b/test_error.py E [100%]
================================== ERRORS ================================== ======================================= ERRORS =======================================
_______________________ ERROR at setup of test_root ________________________ ____________________________ ERROR at setup of test_root _____________________________
file $REGENDOC_TMPDIR/b/test_error.py, line 1 file $REGENDOC_TMPDIR/b/test_error.py, line 1
def test_root(db): # no db here, will error out def test_root(db): # no db here, will error out
E fixture 'db' not found E fixture 'db' not found
@ -602,8 +620,8 @@ We can run this:
> use 'pytest --fixtures [testpath]' for help on them. > use 'pytest --fixtures [testpath]' for help on them.
$REGENDOC_TMPDIR/b/test_error.py:1 $REGENDOC_TMPDIR/b/test_error.py:1
================================= FAILURES ================================= ====================================== FAILURES ======================================
____________________ TestUserHandling.test_modification ____________________ _________________________ TestUserHandling.test_modification _________________________
self = <test_step.TestUserHandling object at 0xdeadbeef> self = <test_step.TestUserHandling object at 0xdeadbeef>
@ -612,7 +630,7 @@ We can run this:
E assert 0 E assert 0
test_step.py:11: AssertionError test_step.py:11: AssertionError
_________________________________ test_a1 __________________________________ ______________________________________ test_a1 _______________________________________
db = <conftest.DB object at 0xdeadbeef> db = <conftest.DB object at 0xdeadbeef>
@ -622,7 +640,7 @@ We can run this:
E assert 0 E assert 0
a/test_db.py:2: AssertionError a/test_db.py:2: AssertionError
_________________________________ test_a2 __________________________________ ______________________________________ test_a2 _______________________________________
db = <conftest.DB object at 0xdeadbeef> db = <conftest.DB object at 0xdeadbeef>
@ -632,7 +650,7 @@ We can run this:
E assert 0 E assert 0
a/test_db2.py:2: AssertionError a/test_db2.py:2: AssertionError
========== 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ========== =============== 3 failed, 2 passed, 1 xfailed, 1 error in 0.12 seconds ===============
The two test modules in the ``a`` directory see the same ``db`` fixture instance The two test modules in the ``a`` directory see the same ``db`` fixture instance
while the one test in the sister-directory ``b`` doesn't see it. We could of course while the one test in the sister-directory ``b`` doesn't see it. We could of course
@ -696,15 +714,17 @@ and run them:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_module.py $ pytest test_module.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py FF [100%] test_module.py FF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________________ test_fail1 ________________________________ _____________________________________ test_fail1 _____________________________________
tmpdir = local('PYTEST_TMPDIR/test_fail10') tmpdir = local('PYTEST_TMPDIR/test_fail10')
@ -713,14 +733,14 @@ and run them:
E assert 0 E assert 0
test_module.py:2: AssertionError test_module.py:2: AssertionError
________________________________ test_fail2 ________________________________ _____________________________________ test_fail2 _____________________________________
def test_fail2(): def test_fail2():
> assert 0 > assert 0
E assert 0 E assert 0
test_module.py:6: AssertionError test_module.py:6: AssertionError
========================= 2 failed in 0.12 seconds ========================= ============================== 2 failed in 0.12 seconds ==============================
you will have a "failures" file which contains the failing test ids:: you will have a "failures" file which contains the failing test ids::
@ -797,17 +817,19 @@ and run it:
.. code-block:: pytest .. code-block:: pytest
$ pytest -s test_module.py $ pytest -s test_module.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 3 items collected 3 items
test_module.py Esetting up a test failed! test_module.py::test_setup_fails test_module.py Esetting up a test failed! test_module.py::test_setup_fails
Fexecuting test failed test_module.py::test_call_fails Fexecuting test failed test_module.py::test_call_fails
F F
================================== ERRORS ================================== ======================================= ERRORS =======================================
____________________ ERROR at setup of test_setup_fails ____________________ _________________________ ERROR at setup of test_setup_fails _________________________
@pytest.fixture @pytest.fixture
def other(): def other():
@ -815,8 +837,8 @@ and run it:
E assert 0 E assert 0
test_module.py:7: AssertionError test_module.py:7: AssertionError
================================= FAILURES ================================= ====================================== FAILURES ======================================
_____________________________ test_call_fails ______________________________ __________________________________ test_call_fails ___________________________________
something = None something = None
@ -825,14 +847,14 @@ and run it:
E assert 0 E assert 0
test_module.py:15: AssertionError test_module.py:15: AssertionError
________________________________ test_fail2 ________________________________ _____________________________________ test_fail2 _____________________________________
def test_fail2(): def test_fail2():
> assert 0 > assert 0
E assert 0 E assert 0
test_module.py:19: AssertionError test_module.py:19: AssertionError
==================== 2 failed, 1 error in 0.12 seconds ===================== ========================= 2 failed, 1 error in 0.12 seconds ==========================
You'll see that the fixture finalizers could use the precise reporting You'll see that the fixture finalizers could use the precise reporting
information. information.

88
doc/en/fixture.rst
View File

@ -71,15 +71,17 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_smtpsimple.py $ pytest test_smtpsimple.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_smtpsimple.py F [100%] test_smtpsimple.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________________ test_ehlo _________________________________ _____________________________________ test_ehlo ______________________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -90,7 +92,7 @@ marked ``smtp_connection`` fixture function. Running the test looks like this:
E assert 0 E assert 0
test_smtpsimple.py:11: AssertionError test_smtpsimple.py:11: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
In the failure traceback we see that the test function was called with a In the failure traceback we see that the test function was called with a
``smtp_connection`` argument, the ``smtplib.SMTP()`` instance created by the fixture ``smtp_connection`` argument, the ``smtplib.SMTP()`` instance created by the fixture
@ -211,15 +213,17 @@ inspect what is going on and can now run the tests:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_module.py $ pytest test_module.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_module.py FF [100%] test_module.py FF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________________ test_ehlo _________________________________ _____________________________________ test_ehlo ______________________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -231,7 +235,7 @@ inspect what is going on and can now run the tests:
E assert 0 E assert 0
test_module.py:6: AssertionError test_module.py:6: AssertionError
________________________________ test_noop _________________________________ _____________________________________ test_noop ______________________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -242,7 +246,7 @@ inspect what is going on and can now run the tests:
E assert 0 E assert 0
test_module.py:11: AssertionError test_module.py:11: AssertionError
========================= 2 failed in 0.12 seconds ========================= ============================== 2 failed in 0.12 seconds ==============================
You see the two ``assert 0`` failing and more importantly you can also see You see the two ``assert 0`` failing and more importantly you can also see
that the same (module-scoped) ``smtp_connection`` object was passed into the that the same (module-scoped) ``smtp_connection`` object was passed into the
@ -491,14 +495,14 @@ Running it:
.. code-block:: pytest .. code-block:: pytest
$ pytest -qq --tb=short test_anothersmtp.py $ pytest -qq --tb=short test_anothersmtp.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
______________________________ test_showhelo _______________________________ ___________________________________ test_showhelo ____________________________________
test_anothersmtp.py:5: in test_showhelo test_anothersmtp.py:5: in test_showhelo
assert 0, smtp_connection.helo() assert 0, smtp_connection.helo()
E AssertionError: (250, b'mail.python.org') E AssertionError: (250, b'mail.python.org')
E assert 0 E assert 0
------------------------- Captured stdout teardown ------------------------- ------------------------------ Captured stdout teardown ------------------------------
finalizing <smtplib.SMTP object at 0xdeadbeef> (mail.python.org) finalizing <smtplib.SMTP object at 0xdeadbeef> (mail.python.org)
voila! The ``smtp_connection`` fixture function picked up our mail server name voila! The ``smtp_connection`` fixture function picked up our mail server name
@ -595,9 +599,9 @@ So let's just do another run:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_module.py $ pytest -q test_module.py
FFFF [100%] FFFF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
________________________ test_ehlo[smtp.gmail.com] _________________________ _____________________________ test_ehlo[smtp.gmail.com] ______________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -609,7 +613,7 @@ So let's just do another run:
E assert 0 E assert 0
test_module.py:6: AssertionError test_module.py:6: AssertionError
________________________ test_noop[smtp.gmail.com] _________________________ _____________________________ test_noop[smtp.gmail.com] ______________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -620,7 +624,7 @@ So let's just do another run:
E assert 0 E assert 0
test_module.py:11: AssertionError test_module.py:11: AssertionError
________________________ test_ehlo[mail.python.org] ________________________ _____________________________ test_ehlo[mail.python.org] _____________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -631,9 +635,9 @@ So let's just do another run:
E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8\nCHUNKING' E AssertionError: assert b'smtp.gmail.com' in b'mail.python.org\nPIPELINING\nSIZE 51200000\nETRN\nSTARTTLS\nAUTH DIGEST-MD5 NTLM CRAM-MD5\nENHANCEDSTATUSCODES\n8BITMIME\nDSN\nSMTPUTF8\nCHUNKING'
test_module.py:5: AssertionError test_module.py:5: AssertionError
-------------------------- Captured stdout setup --------------------------- ------------------------------- Captured stdout setup --------------------------------
finalizing <smtplib.SMTP object at 0xdeadbeef> finalizing <smtplib.SMTP object at 0xdeadbeef>
________________________ test_noop[mail.python.org] ________________________ _____________________________ test_noop[mail.python.org] _____________________________
smtp_connection = <smtplib.SMTP object at 0xdeadbeef> smtp_connection = <smtplib.SMTP object at 0xdeadbeef>
@ -644,7 +648,7 @@ So let's just do another run:
E assert 0 E assert 0
test_module.py:11: AssertionError test_module.py:11: AssertionError
------------------------- Captured stdout teardown ------------------------- ------------------------------ Captured stdout teardown ------------------------------
finalizing <smtplib.SMTP object at 0xdeadbeef> finalizing <smtplib.SMTP object at 0xdeadbeef>
4 failed in 0.12 seconds 4 failed in 0.12 seconds
@ -699,9 +703,11 @@ Running the above tests results in the following test IDs being used:
.. code-block:: pytest .. code-block:: pytest
$ pytest --collect-only $ pytest --collect-only
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 10 items collected 10 items
<Module test_anothersmtp.py> <Module test_anothersmtp.py>
<Function test_showhelo[smtp.gmail.com]> <Function test_showhelo[smtp.gmail.com]>
@ -717,7 +723,7 @@ Running the above tests results in the following test IDs being used:
<Function test_ehlo[mail.python.org]> <Function test_ehlo[mail.python.org]>
<Function test_noop[mail.python.org]> <Function test_noop[mail.python.org]>
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. _`fixture-parametrize-marks`: .. _`fixture-parametrize-marks`:
@ -743,17 +749,19 @@ Running this test will *skip* the invocation of ``data_set`` with value ``2``:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_fixture_marks.py -v $ pytest test_fixture_marks.py -v
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 3 items collecting ... collected 3 items
test_fixture_marks.py::test_data[0] PASSED [ 33%] test_fixture_marks.py::test_data[0] PASSED [ 33%]
test_fixture_marks.py::test_data[1] PASSED [ 66%] test_fixture_marks.py::test_data[1] PASSED [ 66%]
test_fixture_marks.py::test_data[2] SKIPPED [100%] test_fixture_marks.py::test_data[2] SKIPPED [100%]
=================== 2 passed, 1 skipped in 0.12 seconds ==================== ======================== 2 passed, 1 skipped in 0.12 seconds =========================
.. _`interdependent fixtures`: .. _`interdependent fixtures`:
@ -788,16 +796,18 @@ Here we declare an ``app`` fixture which receives the previously defined
.. code-block:: pytest .. code-block:: pytest
$ pytest -v test_appsetup.py $ pytest -v test_appsetup.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 2 items collecting ... collected 2 items
test_appsetup.py::test_smtp_connection_exists[smtp.gmail.com] PASSED [ 50%] test_appsetup.py::test_smtp_connection_exists[smtp.gmail.com] PASSED [ 50%]
test_appsetup.py::test_smtp_connection_exists[mail.python.org] PASSED [100%] test_appsetup.py::test_smtp_connection_exists[mail.python.org] PASSED [100%]
========================= 2 passed in 0.12 seconds ========================= ============================== 2 passed in 0.12 seconds ==============================
Due to the parametrization of ``smtp_connection``, the test will run twice with two Due to the parametrization of ``smtp_connection``, the test will run twice with two
different ``App`` instances and respective smtp servers. There is no different ``App`` instances and respective smtp servers. There is no
@ -859,10 +869,12 @@ Let's run the tests in verbose mode and with looking at the print-output:
.. code-block:: pytest .. code-block:: pytest
$ pytest -v -s test_module.py $ pytest -v -s test_module.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6
cachedir: .pytest_cache cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collecting ... collected 8 items collecting ... collected 8 items
test_module.py::test_0[1] SETUP otherarg 1 test_module.py::test_0[1] SETUP otherarg 1
@ -898,7 +910,7 @@ Let's run the tests in verbose mode and with looking at the print-output:
TEARDOWN modarg mod2 TEARDOWN modarg mod2
========================= 8 passed in 0.12 seconds ========================= ============================== 8 passed in 0.12 seconds ==============================
You can see that the parametrized module-scoped ``modarg`` resource caused an You can see that the parametrized module-scoped ``modarg`` resource caused an
ordering of test execution that lead to the fewest possible "active" resources. ordering of test execution that lead to the fewest possible "active" resources.
@ -963,7 +975,7 @@ to verify our fixture is activated and the tests pass:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
.. [100%] .. [100%]
2 passed in 0.12 seconds 2 passed in 0.12 seconds
You can specify multiple fixtures like this: You can specify multiple fixtures like this:
@ -1064,7 +1076,7 @@ If we run it, we get two passing tests:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q $ pytest -q
.. [100%] .. [100%]
2 passed in 0.12 seconds 2 passed in 0.12 seconds
Here is how autouse fixtures work in other scopes: Here is how autouse fixtures work in other scopes:

30
doc/en/getting-started.rst
View File

@ -25,6 +25,8 @@ Install ``pytest``
$ pytest --version $ pytest --version
This is pytest version 4.x.y, imported from $PYTHON_PREFIX/lib/python3.6/site-packages/pytest.py This is pytest version 4.x.y, imported from $PYTHON_PREFIX/lib/python3.6/site-packages/pytest.py
setuptools registered plugins:
hypothesis-3.x.y at $PYTHON_PREFIX/lib/python3.6/site-packages/hypothesis/extra/pytestplugin.py
.. _`simpletest`: .. _`simpletest`:
@ -45,15 +47,17 @@ That’s it. You can now execute the test function:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_sample.py F [100%] test_sample.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_answer ________________________________ ____________________________________ test_answer _____________________________________
def test_answer(): def test_answer():
> assert func(3) == 5 > assert func(3) == 5
@ -61,7 +65,7 @@ That’s it. You can now execute the test function:
E + where 4 = func(3) E + where 4 = func(3)
test_sample.py:5: AssertionError test_sample.py:5: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
This test returns a failure report because ``func(3)`` does not return ``5``. This test returns a failure report because ``func(3)`` does not return ``5``.
@ -94,7 +98,7 @@ Execute the test function with “quiet” reporting mode:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_sysexit.py $ pytest -q test_sysexit.py
. [100%] . [100%]
1 passed in 0.12 seconds 1 passed in 0.12 seconds
Group multiple tests in a class Group multiple tests in a class
@ -117,9 +121,9 @@ Once you develop multiple tests, you may want to group them into a class. pytest
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_class.py $ pytest -q test_class.py
.F [100%] .F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
____________________________ TestClass.test_two ____________________________ _________________________________ TestClass.test_two _________________________________
self = <test_class.TestClass object at 0xdeadbeef> self = <test_class.TestClass object at 0xdeadbeef>
@ -149,9 +153,9 @@ List the name ``tmpdir`` in the test function signature and ``pytest`` will look
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_tmpdir.py $ pytest -q test_tmpdir.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_____________________________ test_needsfiles ______________________________ __________________________________ test_needsfiles ___________________________________
tmpdir = local('PYTEST_TMPDIR/test_needsfiles0') tmpdir = local('PYTEST_TMPDIR/test_needsfiles0')
@ -161,7 +165,7 @@ List the name ``tmpdir`` in the test function signature and ``pytest`` will look
E assert 0 E assert 0
test_tmpdir.py:3: AssertionError test_tmpdir.py:3: AssertionError
--------------------------- Captured stdout call --------------------------- -------------------------------- Captured stdout call --------------------------------
PYTEST_TMPDIR/test_needsfiles0 PYTEST_TMPDIR/test_needsfiles0
1 failed in 0.12 seconds 1 failed in 0.12 seconds

12
doc/en/index.rst
View File

@ -27,15 +27,17 @@ To execute it:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_sample.py F [100%] test_sample.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_______________________________ test_answer ________________________________ ____________________________________ test_answer _____________________________________
def test_answer(): def test_answer():
> assert inc(3) == 5 > assert inc(3) == 5
@ -43,7 +45,7 @@ To execute it:
E + where 4 = inc(3) E + where 4 = inc(3)
test_sample.py:6: AssertionError test_sample.py:6: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used.
See :ref:`Getting Started <getstarted>` for more examples. See :ref:`Getting Started <getstarted>` for more examples.

32
doc/en/parametrize.rst
View File

@ -55,15 +55,17 @@ them in turn:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 3 items collected 3 items
test_expectation.py ..F [100%] test_expectation.py ..F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
____________________________ test_eval[6*9-42] _____________________________ _________________________________ test_eval[6*9-42] __________________________________
test_input = '6*9', expected = 42 test_input = '6*9', expected = 42
@ -78,7 +80,7 @@ them in turn:
E + where 54 = eval('6*9') E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError test_expectation.py:8: AssertionError
==================== 1 failed, 2 passed in 0.12 seconds ==================== ========================= 1 failed, 2 passed in 0.12 seconds =========================
As designed in this example, only one pair of input/output values fails As designed in this example, only one pair of input/output values fails
the simple test function. And as usual with test function arguments, the simple test function. And as usual with test function arguments,
@ -106,14 +108,16 @@ Let's run this:
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 3 items collected 3 items
test_expectation.py ..x [100%] test_expectation.py ..x [100%]
=================== 2 passed, 1 xfailed in 0.12 seconds ==================== ======================== 2 passed, 1 xfailed in 0.12 seconds =========================
The one parameter set which caused a failure previously now The one parameter set which caused a failure previously now
shows up as an "xfailed (expected to fail)" test. shows up as an "xfailed (expected to fail)" test.
@ -173,7 +177,7 @@ command line option and the parametrization of our test function::
If we now pass two stringinput values, our test will run twice:: If we now pass two stringinput values, our test will run twice::
$ pytest -q --stringinput="hello" --stringinput="world" test_strings.py $ pytest -q --stringinput="hello" --stringinput="world" test_strings.py
.. [100%] .. [100%]
2 passed in 0.12 seconds 2 passed in 0.12 seconds
Let's also run with a stringinput that will lead to a failing test: Let's also run with a stringinput that will lead to a failing test:
@ -181,9 +185,9 @@ Let's also run with a stringinput that will lead to a failing test:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q --stringinput="!" test_strings.py $ pytest -q --stringinput="!" test_strings.py
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
___________________________ test_valid_string[!] ___________________________ ________________________________ test_valid_string[!] ________________________________
stringinput = '!' stringinput = '!'
@ -205,8 +209,8 @@ list:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q -rs test_strings.py $ pytest -q -rs test_strings.py
s [100%] s [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
SKIP [1] test_strings.py: got empty parameter set ['stringinput'], function test_valid_string at $REGENDOC_TMPDIR/test_strings.py:1 SKIP [1] test_strings.py: got empty parameter set ['stringinput'], function test_valid_string at $REGENDOC_TMPDIR/test_strings.py:1
1 skipped in 0.12 seconds 1 skipped in 0.12 seconds

10
doc/en/skipping.rst
View File

@ -328,13 +328,15 @@ Running it with the report-on-xfail option gives this output:
.. code-block:: pytest .. code-block:: pytest
example $ pytest -rx xfail_demo.py example $ pytest -rx xfail_demo.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/example/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR/example, inifile: rootdir: $REGENDOC_TMPDIR/example, inifile:
plugins: hypothesis-3.x.y
collected 7 items collected 7 items
xfail_demo.py xxxxxxx [100%] xfail_demo.py xxxxxxx [100%]
========================= short test summary info ========================== ============================== short test summary info ===============================
XFAIL xfail_demo.py::test_hello XFAIL xfail_demo.py::test_hello
XFAIL xfail_demo.py::test_hello2 XFAIL xfail_demo.py::test_hello2
reason: [NOTRUN] reason: [NOTRUN]
@ -348,7 +350,7 @@ Running it with the report-on-xfail option gives this output:
reason: reason reason: reason
XFAIL xfail_demo.py::test_hello7 XFAIL xfail_demo.py::test_hello7
======================== 7 xfailed in 0.12 seconds ========================= ============================= 7 xfailed in 0.12 seconds ==============================
.. _`skip/xfail with parametrize`: .. _`skip/xfail with parametrize`:

24
doc/en/tmpdir.rst
View File

@ -40,15 +40,17 @@ Running this would result in a passed test except for the last
.. code-block:: pytest .. code-block:: pytest
$ pytest test_tmp_path.py $ pytest test_tmp_path.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_tmp_path.py F [100%] test_tmp_path.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_____________________________ test_create_file _____________________________ __________________________________ test_create_file __________________________________
tmp_path = PosixPath('PYTEST_TMPDIR/test_create_file0') tmp_path = PosixPath('PYTEST_TMPDIR/test_create_file0')
@ -63,7 +65,7 @@ Running this would result in a passed test except for the last
E assert 0 E assert 0
test_tmp_path.py:13: AssertionError test_tmp_path.py:13: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
The ``tmp_path_factory`` fixture The ``tmp_path_factory`` fixture
-------------------------------- --------------------------------
@ -102,15 +104,17 @@ Running this would result in a passed test except for the last
.. code-block:: pytest .. code-block:: pytest
$ pytest test_tmpdir.py $ pytest test_tmpdir.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_tmpdir.py F [100%] test_tmpdir.py F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_____________________________ test_create_file _____________________________ __________________________________ test_create_file __________________________________
tmpdir = local('PYTEST_TMPDIR/test_create_file0') tmpdir = local('PYTEST_TMPDIR/test_create_file0')
@ -123,7 +127,7 @@ Running this would result in a passed test except for the last
E assert 0 E assert 0
test_tmpdir.py:7: AssertionError test_tmpdir.py:7: AssertionError
========================= 1 failed in 0.12 seconds ========================= ============================== 1 failed in 0.12 seconds ==============================
.. _`tmpdir factory example`: .. _`tmpdir factory example`:

16
doc/en/unittest.rst
View File

@ -127,15 +127,17 @@ the ``self.db`` values in the traceback:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_unittest_db.py $ pytest test_unittest_db.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_unittest_db.py FF [100%] test_unittest_db.py FF [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
___________________________ MyTest.test_method1 ____________________________ ________________________________ MyTest.test_method1 _________________________________
self = <test_unittest_db.MyTest testMethod=test_method1> self = <test_unittest_db.MyTest testMethod=test_method1>
@ -146,7 +148,7 @@ the ``self.db`` values in the traceback:
E assert 0 E assert 0
test_unittest_db.py:9: AssertionError test_unittest_db.py:9: AssertionError
___________________________ MyTest.test_method2 ____________________________ ________________________________ MyTest.test_method2 _________________________________
self = <test_unittest_db.MyTest testMethod=test_method2> self = <test_unittest_db.MyTest testMethod=test_method2>
@ -156,7 +158,7 @@ the ``self.db`` values in the traceback:
E assert 0 E assert 0
test_unittest_db.py:12: AssertionError test_unittest_db.py:12: AssertionError
========================= 2 failed in 0.12 seconds ========================= ============================== 2 failed in 0.12 seconds ==============================
This default pytest traceback shows that the two test methods This default pytest traceback shows that the two test methods
share the same ``self.db`` instance which was our intention share the same ``self.db`` instance which was our intention
@ -206,7 +208,7 @@ Running this test module ...:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_unittest_cleandir.py $ pytest -q test_unittest_cleandir.py
. [100%] . [100%]
1 passed in 0.12 seconds 1 passed in 0.12 seconds
... gives us one passed test because the ``initdir`` fixture function ... gives us one passed test because the ``initdir`` fixture function

20
doc/en/usage.rst
View File

@ -155,12 +155,14 @@ Example:
.. code-block:: pytest .. code-block:: pytest
$ pytest -ra $ pytest -ra
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
The ``-r`` options accepts a number of characters after it, with ``a`` used above meaning "all except passes". The ``-r`` options accepts a number of characters after it, with ``a`` used above meaning "all except passes".
@ -180,12 +182,14 @@ More than one character can be used, so for example to only see failed and skipp
.. code-block:: pytest .. code-block:: pytest
$ pytest -rfs $ pytest -rfs
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
Using ``p`` lists the passing tests, whilst ``P`` adds an extra section "PASSES" with those tests that passed but had Using ``p`` lists the passing tests, whilst ``P`` adds an extra section "PASSES" with those tests that passed but had
captured output: captured output:
@ -193,12 +197,14 @@ captured output:
.. code-block:: pytest .. code-block:: pytest
$ pytest -rpP $ pytest -rpP
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 0 items collected 0 items
======================= no tests ran in 0.12 seconds ======================= ============================ no tests ran in 0.12 seconds ============================
.. _pdb-option: .. _pdb-option:
@ -584,7 +590,7 @@ Running it will show that ``MyPlugin`` was added and its
hook was invoked:: hook was invoked::
$ python myinvoke.py $ python myinvoke.py
. [100%]*** test run reporting finishing . [100%]*** test run reporting finishing
.. note:: .. note::

20
doc/en/warnings.rst
View File

@ -23,20 +23,22 @@ Running pytest now produces this output:
.. code-block:: pytest .. code-block:: pytest
$ pytest test_show_warnings.py $ pytest test_show_warnings.py
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: rootdir: $REGENDOC_TMPDIR, inifile:
plugins: hypothesis-3.x.y
collected 1 item collected 1 item
test_show_warnings.py . [100%] test_show_warnings.py . [100%]
============================= warnings summary ============================= ================================== warnings summary ==================================
test_show_warnings.py::test_one test_show_warnings.py::test_one
$REGENDOC_TMPDIR/test_show_warnings.py:4: UserWarning: api v1, should use functions from v2 $REGENDOC_TMPDIR/test_show_warnings.py:4: UserWarning: api v1, should use functions from v2
warnings.warn(UserWarning("api v1, should use functions from v2")) warnings.warn(UserWarning("api v1, should use functions from v2"))
-- Docs: https://docs.pytest.org/en/latest/warnings.html -- Docs: https://docs.pytest.org/en/latest/warnings.html
=================== 1 passed, 1 warnings in 0.12 seconds =================== ======================== 1 passed, 1 warnings in 0.12 seconds ========================
The ``-W`` flag can be passed to control which warnings will be displayed or even turn The ``-W`` flag can be passed to control which warnings will be displayed or even turn
them into errors: them into errors:
@ -44,15 +46,15 @@ them into errors:
.. code-block:: pytest .. code-block:: pytest
$ pytest -q test_show_warnings.py -W error::UserWarning $ pytest -q test_show_warnings.py -W error::UserWarning
F [100%] F [100%]
================================= FAILURES ================================= ====================================== FAILURES ======================================
_________________________________ test_one _________________________________ ______________________________________ test_one ______________________________________
def test_one(): def test_one():
> assert api_v1() == 1 > assert api_v1() == 1
test_show_warnings.py:8: test_show_warnings.py:8:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def api_v1(): def api_v1():
> warnings.warn(UserWarning("api v1, should use functions from v2")) > warnings.warn(UserWarning("api v1, should use functions from v2"))
@ -355,7 +357,7 @@ defines an ``__init__`` constructor, as this prevents the class from being insta
$ pytest test_pytest_warnings.py -q $ pytest test_pytest_warnings.py -q
============================= warnings summary ============================= ================================== warnings summary ==================================
test_pytest_warnings.py:1 test_pytest_warnings.py:1
$REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestWarning: cannot collect test class 'Test' because it has a __init__ constructor $REGENDOC_TMPDIR/test_pytest_warnings.py:1: PytestWarning: cannot collect test class 'Test' because it has a __init__ constructor
class Test: class Test:

10
doc/en/writing_plugins.rst
View File

@ -411,20 +411,22 @@ additionally it is possible to copy examples for an example folder before runnin
.. code-block:: pytest .. code-block:: pytest
$ pytest $ pytest
=========================== test session starts ============================ ================================ test session starts =================================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('$REGENDOC_TMPDIR/.hypothesis/examples')
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
plugins: hypothesis-3.x.y
collected 2 items collected 2 items
test_example.py .. [100%] test_example.py .. [100%]
============================= warnings summary ============================= ================================== warnings summary ==================================
test_example.py::test_plugin test_example.py::test_plugin
$REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time $REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time
testdir.copy_example("test_example.py") testdir.copy_example("test_example.py")
-- Docs: https://docs.pytest.org/en/latest/warnings.html -- Docs: https://docs.pytest.org/en/latest/warnings.html
=================== 2 passed, 1 warnings in 0.12 seconds =================== ======================== 2 passed, 1 warnings in 0.12 seconds ========================
For more information about the result object that ``runpytest()`` returns, and For more information about the result object that ``runpytest()`` returns, and
the methods that it provides please check out the :py:class:`RunResult the methods that it provides please check out the :py:class:`RunResult