diff --git a/doc/en/assert.rst b/doc/en/assert.rst index 01767225f..43fedebed 100644 --- a/doc/en/assert.rst +++ b/doc/en/assert.rst @@ -22,7 +22,9 @@ following:: assert f() == 4 to assert that your function returns a certain value. If this assertion fails -you will see the return value of the function call:: +you will see the return value of the function call: + +.. code-block:: pytest $ pytest test_assert1.py =========================== test session starts ============================ @@ -165,7 +167,9 @@ when it encounters comparisons. For example:: set2 = set("8035") assert set1 == set2 -if you run this module:: +if you run this module: + +.. code-block:: pytest $ pytest test_assert2.py =========================== test session starts ============================ @@ -235,7 +239,9 @@ now, given this test module:: assert f1 == f2 you can run the test module and get the custom output defined in -the conftest file:: +the conftest file: + +.. code-block:: pytest $ pytest -q test_foocompare.py F [100%] diff --git a/doc/en/builtin.rst b/doc/en/builtin.rst index 60681bc9b..1e376f0d3 100644 --- a/doc/en/builtin.rst +++ b/doc/en/builtin.rst @@ -12,7 +12,9 @@ For information on plugin hooks and objects, see :ref:`plugins`. For information on the ``pytest.mark`` mechanism, see :ref:`mark`. -For information about fixtures, see :ref:`fixtures`. To see a complete list of available fixtures (add ``-v`` to also see fixtures with leading ``_``), type :: +For information about fixtures, see :ref:`fixtures`. To see a complete list of available fixtures (add ``-v`` to also see fixtures with leading ``_``), type : + +.. code-block:: pytest $ pytest -q --fixtures cache diff --git a/doc/en/cache.rst b/doc/en/cache.rst index 4a493959a..914c36fed 100644 --- a/doc/en/cache.rst +++ b/doc/en/cache.rst @@ -43,7 +43,9 @@ First, let's create 50 test invocation of which only 2 fail:: if i in (17, 25): pytest.fail("bad luck") -If you run this for the first time you will see two failures:: +If you run this for the first time you will see two failures: + +.. code-block:: pytest $ pytest -q .................F.......F........................ [100%] @@ -72,7 +74,9 @@ If you run this for the first time you will see two failures:: test_50.py:6: Failed 2 failed, 48 passed in 0.12 seconds -If you then run it with ``--lf``:: +If you then run it with ``--lf``: + +.. code-block:: pytest $ pytest --lf =========================== test session starts ============================ @@ -113,7 +117,9 @@ not been run ("deselected"). Now, if you run with the ``--ff`` option, all tests will be run but the first previous failures will be executed first (as can be seen from the series -of ``FF`` and dots):: +of ``FF`` and dots): + +.. code-block:: pytest $ pytest --ff =========================== test session starts ============================ @@ -192,7 +198,9 @@ across pytest invocations:: assert mydata == 23 If you run this command once, it will take a while because -of the sleep:: +of the sleep: + +.. code-block:: pytest $ pytest -q F [100%] @@ -209,7 +217,9 @@ of the sleep:: 1 failed in 0.12 seconds If you run it a second time the value will be retrieved from -the cache and this will be quick:: +the cache and this will be quick: + +.. code-block:: pytest $ pytest -q F [100%] @@ -232,7 +242,9 @@ Inspecting Cache content ------------------------------- You can always peek at the content of the cache using the -``--cache-show`` command line option:: +``--cache-show`` command line option: + +.. code-block:: pytest $ pytest --cache-show =========================== test session starts ============================ diff --git a/doc/en/capture.rst b/doc/en/capture.rst index a3b7d4d5e..488b2b874 100644 --- a/doc/en/capture.rst +++ b/doc/en/capture.rst @@ -61,7 +61,9 @@ is that you can use print statements for debugging:: assert False and running this module will show you precisely the output -of the failing function and hide the other one:: +of the failing function and hide the other one: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ diff --git a/doc/en/conf.py b/doc/en/conf.py index 686a05ba4..445449185 100644 --- a/doc/en/conf.py +++ b/doc/en/conf.py @@ -40,6 +40,7 @@ todo_include_todos = 1 # Add any Sphinx extension module names here, as strings. They can be extensions # coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ + "pygments_pytest", "sphinx.ext.autodoc", "sphinx.ext.todo", "sphinx.ext.autosummary", diff --git a/doc/en/doctest.rst b/doc/en/doctest.rst index 0030d8df8..52edd4cf2 100644 --- a/doc/en/doctest.rst +++ b/doc/en/doctest.rst @@ -58,7 +58,9 @@ and another like this:: """ return 42 -then you can just invoke ``pytest`` without command line options:: +then you can just invoke ``pytest`` without command line options: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ diff --git a/doc/en/example/markers.rst b/doc/en/example/markers.rst index bcd42616b..8ab885f0f 100644 --- a/doc/en/example/markers.rst +++ b/doc/en/example/markers.rst @@ -27,7 +27,9 @@ You can "mark" a test function with custom metadata like this:: .. versionadded:: 2.2 -You can then restrict a test run to only run tests marked with ``webtest``:: +You can then restrict a test run to only run tests marked with ``webtest``: + +.. code-block:: pytest $ pytest -v -m webtest =========================== test session starts ============================ @@ -40,7 +42,9 @@ You can then restrict a test run to only run tests marked with ``webtest``:: ================== 1 passed, 3 deselected in 0.12 seconds ================== -Or the inverse, running all tests except the webtest ones:: +Or the inverse, running all tests except the webtest ones: + +.. code-block:: pytest $ pytest -v -m "not webtest" =========================== test session starts ============================ @@ -60,7 +64,9 @@ Selecting tests based on their node ID You can provide one or more :ref:`node IDs ` as positional arguments to select only specified tests. This makes it easy to select -tests based on their module, class, method, or function name:: +tests based on their module, class, method, or function name: + +.. code-block:: pytest $ pytest -v test_server.py::TestClass::test_method =========================== test session starts ============================ @@ -73,7 +79,9 @@ tests based on their module, class, method, or function name:: ========================= 1 passed in 0.12 seconds ========================= -You can also select on the class:: +You can also select on the class: + +.. code-block:: pytest $ pytest -v test_server.py::TestClass =========================== test session starts ============================ @@ -86,19 +94,21 @@ You can also select on the class:: ========================= 1 passed in 0.12 seconds ========================= -Or select multiple nodes:: +Or select multiple nodes: - $ pytest -v test_server.py::TestClass test_server.py::test_send_http - =========================== test session starts ============================ - platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 - cachedir: .pytest_cache - rootdir: $REGENDOC_TMPDIR, inifile: - collecting ... collected 2 items +.. code-block:: pytest - test_server.py::TestClass::test_method PASSED [ 50%] - test_server.py::test_send_http PASSED [100%] + $ pytest -v test_server.py::TestClass test_server.py::test_send_http + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python3.6 + cachedir: .pytest_cache + rootdir: $REGENDOC_TMPDIR, inifile: + collecting ... collected 2 items - ========================= 2 passed in 0.12 seconds ========================= + test_server.py::TestClass::test_method PASSED [ 50%] + test_server.py::test_send_http PASSED [100%] + + ========================= 2 passed in 0.12 seconds ========================= .. _node-id: @@ -124,7 +134,9 @@ Using ``-k expr`` to select tests based on their name You can use the ``-k`` command line option to specify an expression which implements a substring match on the test names instead of the exact match on markers that ``-m`` provides. This makes it easy to -select tests based on their names:: +select tests based on their names: + +.. code-block:: pytest $ pytest -v -k http # running with the above defined example module =========================== test session starts ============================ @@ -137,7 +149,9 @@ select tests based on their names:: ================== 1 passed, 3 deselected in 0.12 seconds ================== -And you can also run all tests except the ones that match the keyword:: +And you can also run all tests except the ones that match the keyword: + +.. code-block:: pytest $ pytest -k "not send_http" -v =========================== test session starts ============================ @@ -152,7 +166,9 @@ And you can also run all tests except the ones that match the keyword:: ================== 3 passed, 1 deselected in 0.12 seconds ================== -Or to select "http" and "quick" tests:: +Or to select "http" and "quick" tests: + +.. code-block:: pytest $ pytest -k "http or quick" -v =========================== test session starts ============================ @@ -351,7 +367,9 @@ A test file using this local plugin:: pass and an example invocations specifying a different environment than what -the test needs:: +the test needs: + +.. code-block:: pytest $ pytest -E stage2 =========================== test session starts ============================ @@ -363,7 +381,9 @@ the test needs:: ======================== 1 skipped in 0.12 seconds ========================= -and here is one that specifies exactly the environment needed:: +and here is one that specifies exactly the environment needed: + +.. code-block:: pytest $ pytest -E stage1 =========================== test session starts ============================ @@ -428,7 +448,9 @@ However, if there is a callable as the single positional argument with no keywor def test_with_args(): pass -The output is as follows:: +The output is as follows: + +.. code-block:: pytest $ pytest -q -s Mark(name='my_marker', args=(,), kwargs={}) @@ -469,7 +491,9 @@ test function. From a conftest file we can read it like this:: print("glob args=%s kwargs=%s" % (mark.args, mark.kwargs)) sys.stdout.flush() -Let's run this without capturing output and see what we get:: +Let's run this without capturing output and see what we get: + +.. code-block:: pytest $ pytest -q -s glob args=('function',) kwargs={'x': 3} @@ -524,7 +548,9 @@ Let's do a little test file to show how this looks like:: def test_runs_everywhere(): pass -then you will see two tests skipped and two executed tests as expected:: +then you will see two tests skipped and two executed tests as expected: + +.. code-block:: pytest $ pytest -rs # this option reports skip reasons =========================== test session starts ============================ @@ -538,7 +564,9 @@ then you will see two tests skipped and two executed tests as expected:: =================== 2 passed, 2 skipped in 0.12 seconds ==================== -Note that if you specify a platform via the marker-command line option like this:: +Note that if you specify a platform via the marker-command line option like this: + +.. code-block:: pytest $ pytest -m linux =========================== test session starts ============================ @@ -589,48 +617,52 @@ We want to dynamically define two markers and can do it in a elif "event" in item.nodeid: item.add_marker(pytest.mark.event) -We can now use the ``-m option`` to select one set:: +We can now use the ``-m option`` to select one set: - $ pytest -m interface --tb=short - =========================== test session starts ============================ - platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y - rootdir: $REGENDOC_TMPDIR, inifile: - collected 4 items / 2 deselected +.. code-block:: pytest - test_module.py FF [100%] + $ pytest -m interface --tb=short + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 4 items / 2 deselected - ================================= FAILURES ================================= - __________________________ test_interface_simple ___________________________ - test_module.py:3: in test_interface_simple - assert 0 - E assert 0 - __________________________ test_interface_complex __________________________ - test_module.py:6: in test_interface_complex - assert 0 - E assert 0 - ================== 2 failed, 2 deselected in 0.12 seconds ================== + test_module.py FF [100%] -or to select both "event" and "interface" tests:: + ================================= FAILURES ================================= + __________________________ test_interface_simple ___________________________ + test_module.py:3: in test_interface_simple + assert 0 + E assert 0 + __________________________ test_interface_complex __________________________ + test_module.py:6: in test_interface_complex + assert 0 + E assert 0 + ================== 2 failed, 2 deselected in 0.12 seconds ================== - $ pytest -m "interface or event" --tb=short - =========================== test session starts ============================ - platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y - rootdir: $REGENDOC_TMPDIR, inifile: - collected 4 items / 1 deselected +or to select both "event" and "interface" tests: - test_module.py FFF [100%] +.. code-block:: pytest - ================================= FAILURES ================================= - __________________________ test_interface_simple ___________________________ - test_module.py:3: in test_interface_simple - assert 0 - E assert 0 - __________________________ test_interface_complex __________________________ - test_module.py:6: in test_interface_complex - assert 0 - E assert 0 - ____________________________ test_event_simple _____________________________ - test_module.py:9: in test_event_simple - assert 0 - E assert 0 - ================== 3 failed, 1 deselected in 0.12 seconds ================== + $ pytest -m "interface or event" --tb=short + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y + rootdir: $REGENDOC_TMPDIR, inifile: + collected 4 items / 1 deselected + + test_module.py FFF [100%] + + ================================= FAILURES ================================= + __________________________ test_interface_simple ___________________________ + test_module.py:3: in test_interface_simple + assert 0 + E assert 0 + __________________________ test_interface_complex __________________________ + test_module.py:6: in test_interface_complex + assert 0 + E assert 0 + ____________________________ test_event_simple _____________________________ + test_module.py:9: in test_event_simple + assert 0 + E assert 0 + ================== 3 failed, 1 deselected in 0.12 seconds ================== diff --git a/doc/en/example/nonpython.rst b/doc/en/example/nonpython.rst index bab89f7b8..5b96a33bf 100644 --- a/doc/en/example/nonpython.rst +++ b/doc/en/example/nonpython.rst @@ -23,7 +23,9 @@ You can create a simple example file: :literal: and if you installed `PyYAML`_ or a compatible YAML-parser you can -now execute the test specification:: +now execute the test specification: + +.. code-block:: pytest nonpython $ pytest test_simple.yml =========================== test session starts ============================ @@ -55,7 +57,9 @@ your own domain specific testing language this way. will be reported as a (red) string. ``reportinfo()`` is used for representing the test location and is also -consulted when reporting in ``verbose`` mode:: +consulted when reporting in ``verbose`` mode: + +.. code-block:: pytest nonpython $ pytest -v =========================== test session starts ============================ @@ -77,7 +81,9 @@ consulted when reporting in ``verbose`` mode:: .. regendoc:wipe While developing your custom test collection and execution it's also -interesting to just look at the collection tree:: +interesting to just look at the collection tree: + +.. code-block:: pytest nonpython $ pytest --collect-only =========================== test session starts ============================ diff --git a/doc/en/example/parametrize.rst b/doc/en/example/parametrize.rst index b983b52b5..488f6e310 100644 --- a/doc/en/example/parametrize.rst +++ b/doc/en/example/parametrize.rst @@ -42,14 +42,18 @@ Now we add a test configuration like this:: end = 2 metafunc.parametrize("param1", range(end)) -This means that we only run 2 tests if we do not pass ``--all``:: +This means that we only run 2 tests if we do not pass ``--all``: + +.. code-block:: pytest $ pytest -q test_compute.py .. [100%] 2 passed in 0.12 seconds We run only two computations, so we see two dots. -let's run the full monty:: +let's run the full monty: + +.. code-block:: pytest $ pytest -q --all ....F [100%] @@ -134,8 +138,9 @@ used as the test IDs. These are succinct, but can be a pain to maintain. In ``test_timedistance_v2``, we specified ``ids`` as a function that can generate a string representation to make part of the test ID. So our ``datetime`` values use the label generated by ``idfn``, but because we didn't generate a label for ``timedelta`` -objects, they are still using the default pytest representation:: +objects, they are still using the default pytest representation: +.. code-block:: pytest $ pytest test_time.py --collect-only =========================== test session starts ============================ @@ -191,7 +196,9 @@ only have to work a bit to construct the correct arguments for pytest's def test_demo2(self, attribute): assert isinstance(attribute, str) -this is a fully self-contained example which you can run with:: +this is a fully self-contained example which you can run with: + +.. code-block:: pytest $ pytest test_scenarios.py =========================== test session starts ============================ @@ -203,8 +210,9 @@ this is a fully self-contained example which you can run with:: ========================= 4 passed in 0.12 seconds ========================= -If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function:: +If you just collect tests you'll also nicely see 'advanced' and 'basic' as variants for the test function: +.. code-block:: pytest $ pytest --collect-only test_scenarios.py =========================== test session starts ============================ @@ -268,7 +276,9 @@ creates a database object for the actual test invocations:: else: raise ValueError("invalid internal test config") -Let's first see how it looks like at collection time:: +Let's first see how it looks like at collection time: + +.. code-block:: pytest $ pytest test_backends.py --collect-only =========================== test session starts ============================ @@ -281,7 +291,9 @@ Let's first see how it looks like at collection time:: ======================= no tests ran in 0.12 seconds ======================= -And then when we run the test:: +And then when we run the test: + +.. code-block:: pytest $ pytest -q test_backends.py .F [100%] @@ -329,7 +341,9 @@ will be passed to respective fixture function:: assert x == 'aaa' assert y == 'b' -The result of this test will be successful:: +The result of this test will be successful: + +.. code-block:: pytest $ pytest test_indirect_list.py --collect-only =========================== test session starts ============================ @@ -377,7 +391,9 @@ parametrizer`_ but in a lot less code:: pytest.raises(ZeroDivisionError, "a/b") Our test generator looks up a class-level definition which specifies which -argument sets to use for each test function. Let's run it:: +argument sets to use for each test function. Let's run it: + +.. code-block:: pytest $ pytest -q F.. [100%] @@ -407,7 +423,9 @@ is to be run with different sets of arguments for its three arguments: .. literalinclude:: multipython.py -Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize):: +Running it results in some skips if we don't have all the python interpreters installed and otherwise runs all combinations (5 interpreters times 5 interpreters times 3 objects to serialize/deserialize): + +.. code-block:: pytest . $ pytest -rs -q multipython.py ...sss...sssssssss...sss... [100%] @@ -456,7 +474,9 @@ And finally a little test module:: assert round(basemod.func1(), 3) == round(optmod.func1(), 3) -If you run this with reporting for skips enabled:: +If you run this with reporting for skips enabled: + +.. code-block:: pytest $ pytest -rs test_module.py =========================== test session starts ============================ @@ -511,21 +531,22 @@ we mark the rest three parametrized tests with the custom marker ``basic``, and for the fourth test we also use the built-in mark ``xfail`` to indicate this test is expected to fail. For explicitness, we set test ids for some tests. -Then run ``pytest`` with verbose mode and with only the ``basic`` marker:: +Then run ``pytest`` with verbose mode and with only the ``basic`` marker: - pytest -v -m basic - ============================================ test session starts ============================================= - platform linux -- Python 3.x.y, pytest-3.x.y, py-1.x.y, pluggy-0.x.y +.. code-block:: pytest + + $ pytest -v -m basic + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python + cachedir: .pytest_cache rootdir: $REGENDOC_TMPDIR, inifile: - collected 4 items + collecting ... collected 17 items / 14 deselected - test_pytest_param_example.py::test_eval[1+7-8] PASSED - test_pytest_param_example.py::test_eval[basic_2+4] PASSED - test_pytest_param_example.py::test_eval[basic_6*9] xfail - ========================================== short test summary info =========================================== - XFAIL test_pytest_param_example.py::test_eval[basic_6*9] + test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%] + test_pytest_param_example.py::test_eval[basic_2+4] PASSED [ 66%] + test_pytest_param_example.py::test_eval[basic_6*9] xfail [100%] - ============================================= 1 tests deselected ============================================= + ============ 2 passed, 14 deselected, 1 xfailed in 0.12 seconds ============ As the result: diff --git a/doc/en/example/pythoncollection.rst b/doc/en/example/pythoncollection.rst index 0c1a2540b..3f1dd68ee 100644 --- a/doc/en/example/pythoncollection.rst +++ b/doc/en/example/pythoncollection.rst @@ -24,20 +24,22 @@ by passing the ``--ignore=path`` option on the cli. ``pytest`` allows multiple '-- test_world_03.py Now if you invoke ``pytest`` with ``--ignore=tests/foobar/test_foobar_03.py --ignore=tests/hello/``, -you will see that ``pytest`` only collects test-modules, which do not match the patterns specified:: +you will see that ``pytest`` only collects test-modules, which do not match the patterns specified: - ========= test session starts ========== - platform darwin -- Python 2.7.10, pytest-2.8.2, py-1.4.30, pluggy-0.3.1 +.. code-block:: pytest + + =========================== test session starts ============================ + platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y rootdir: $REGENDOC_TMPDIR, inifile: collected 5 items - tests/example/test_example_01.py . - tests/example/test_example_02.py . - tests/example/test_example_03.py . - tests/foobar/test_foobar_01.py . - tests/foobar/test_foobar_02.py . + tests/example/test_example_01.py . [ 20%] + tests/example/test_example_02.py . [ 40%] + tests/example/test_example_03.py . [ 60%] + tests/foobar/test_foobar_01.py . [ 80%] + tests/foobar/test_foobar_02.py . [100%] - ======= 5 passed in 0.02 seconds ======= + ========================= 5 passed in 0.02 seconds ========================= Deselect tests during test collection ------------------------------------- @@ -123,7 +125,9 @@ that match ``*_check``. For example, if we have:: def complex_check(self): pass -The test collection would look like this:: +The test collection would look like this: + +.. code-block:: pytest $ pytest --collect-only =========================== test session starts ============================ @@ -176,7 +180,9 @@ treat it as a filesystem path. Finding out what is collected ----------------------------------------------- -You can always peek at the collection tree without running tests like this:: +You can always peek at the collection tree without running tests like this: + +.. code-block:: pytest . $ pytest --collect-only pythoncollection.py =========================== test session starts ============================ @@ -231,7 +237,9 @@ and a ``setup.py`` dummy file like this:: 0/0 # will raise exception if imported If you run with a Python 2 interpreter then you will find the one test and will -leave out the ``setup.py`` file:: +leave out the ``setup.py`` file: + +.. code-block:: pytest #$ pytest --collect-only ====== test session starts ====== @@ -244,7 +252,9 @@ leave out the ``setup.py`` file:: ====== no tests ran in 0.04 seconds ====== If you run with a Python 3 interpreter both the one test and the ``setup.py`` -file will be left out:: +file will be left out: + +.. code-block:: pytest $ pytest --collect-only =========================== test session starts ============================ diff --git a/doc/en/example/reportingdemo.rst b/doc/en/example/reportingdemo.rst index fceb83dfc..ffccdf77f 100644 --- a/doc/en/example/reportingdemo.rst +++ b/doc/en/example/reportingdemo.rst @@ -7,7 +7,9 @@ Demo of Python failure reports with pytest Here is a nice run of several tens of failures and how ``pytest`` presents things (unfortunately not showing the nice colors here in the HTML that you -get on the terminal - we are working on that):: +get on the terminal - we are working on that): + +.. code-block:: pytest assertion $ pytest failure_demo.py =========================== test session starts ============================ @@ -364,7 +366,7 @@ get on the terminal - we are working on that):: > int(s) E ValueError: invalid literal for int() with base 10: 'qwe' - <0-codegen $PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/python_api.py:682>:1: ValueError + <0-codegen $REGENDOC_TMPDIR/assertion/failure_demo.py:145>:1: ValueError ______________________ TestRaises.test_raises_doesnt _______________________ self = diff --git a/doc/en/example/simple.rst b/doc/en/example/simple.rst index fa93a77f3..95c0e6365 100644 --- a/doc/en/example/simple.rst +++ b/doc/en/example/simple.rst @@ -43,7 +43,9 @@ provide the ``cmdopt`` through a :ref:`fixture function `: def cmdopt(request): return request.config.getoption("--cmdopt") -Let's run this without supplying our new option:: +Let's run this without supplying our new option: + +.. code-block:: pytest $ pytest -q test_sample.py F [100%] @@ -65,7 +67,9 @@ Let's run this without supplying our new option:: first 1 failed in 0.12 seconds -And now with supplying a command line option:: +And now with supplying a command line option: + +.. code-block:: pytest $ pytest -q --cmdopt=type2 F [100%] @@ -117,7 +121,9 @@ the command line arguments before they get processed: If you have the `xdist plugin `_ installed you will now always perform test runs using a number of subprocesses close to your CPU. Running in an empty -directory with the above conftest.py:: +directory with the above conftest.py: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -175,7 +181,9 @@ We can now write a test module like this: def test_func_slow(): pass -and when running it will see a skipped "slow" test:: +and when running it will see a skipped "slow" test: + +.. code-block:: pytest $ pytest -rs # "-rs" means report details on the little 's' =========================== test session starts ============================ @@ -189,7 +197,9 @@ and when running it will see a skipped "slow" test:: =================== 1 passed, 1 skipped in 0.12 seconds ==================== -Or run it including the ``slow`` marked test:: +Or run it including the ``slow`` marked test: + +.. code-block:: pytest $ pytest --runslow =========================== test session starts ============================ @@ -230,7 +240,9 @@ Example: The ``__tracebackhide__`` setting influences ``pytest`` showing of tracebacks: the ``checkconfig`` function will not be shown unless the ``--full-trace`` command line option is specified. -Let's run our little function:: +Let's run our little function: + +.. code-block:: pytest $ pytest -q test_checkconfig.py F [100%] @@ -327,7 +339,9 @@ It's easy to present extra information in a ``pytest`` run: def pytest_report_header(config): return "project deps: mylib-1.1" -which will add the string to the test header accordingly:: +which will add the string to the test header accordingly: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -353,7 +367,9 @@ display more information if applicable: if config.getoption("verbose") > 0: return ["info1: did you know that ...", "did you?"] -which will add info only when run with "--v":: +which will add info only when run with "--v": + +.. code-block:: pytest $ pytest -v =========================== test session starts ============================ @@ -366,7 +382,9 @@ which will add info only when run with "--v":: ======================= no tests ran in 0.12 seconds ======================= -and nothing when run plainly:: +and nothing when run plainly: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -403,7 +421,9 @@ out which tests are the slowest. Let's make an artificial test suite: def test_funcslow2(): time.sleep(0.3) -Now we can profile which test functions execute the slowest:: +Now we can profile which test functions execute the slowest: + +.. code-block:: pytest $ pytest --durations=3 =========================== test session starts ============================ @@ -475,7 +495,9 @@ tests in a class. Here is a test module example: def test_normal(): pass -If we run this:: +If we run this: + +.. code-block:: pytest $ pytest -rx =========================== test session starts ============================ @@ -556,7 +578,9 @@ the ``db`` fixture: def test_root(db): # no db here, will error out pass -We can run this:: +We can run this: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -667,7 +691,9 @@ if you then have failing tests: def test_fail2(): assert 0 -and run them:: +and run them: + +.. code-block:: pytest $ pytest test_module.py =========================== test session starts ============================ @@ -766,7 +792,9 @@ if you then have failing tests: def test_fail2(): assert 0 -and run it:: +and run it: + +.. code-block:: pytest $ pytest -s test_module.py =========================== test session starts ============================ diff --git a/doc/en/example/special.rst b/doc/en/example/special.rst index 524ae7883..800cb8e90 100644 --- a/doc/en/example/special.rst +++ b/doc/en/example/special.rst @@ -57,7 +57,9 @@ will be called ahead of running any tests:: def test_unit1(self): print("test_unit1 method called") -If you run this without output capturing:: +If you run this without output capturing: + +.. code-block:: pytest $ pytest -q -s test_module.py callattr_ahead_of_alltests called diff --git a/doc/en/fixture.rst b/doc/en/fixture.rst index 2fb3d4b56..e70787a62 100644 --- a/doc/en/fixture.rst +++ b/doc/en/fixture.rst @@ -66,7 +66,9 @@ using it:: Here, the ``test_ehlo`` needs the ``smtp_connection`` fixture value. pytest will discover and call the :py:func:`@pytest.fixture <_pytest.python.fixture>` -marked ``smtp_connection`` fixture function. Running the test looks like this:: +marked ``smtp_connection`` fixture function. Running the test looks like this: + +.. code-block:: pytest $ pytest test_smtpsimple.py =========================== test session starts ============================ @@ -204,7 +206,9 @@ located):: assert 0 # for demo purposes We deliberately insert failing ``assert 0`` statements in order to -inspect what is going on and can now run the tests:: +inspect what is going on and can now run the tests: + +.. code-block:: pytest $ pytest test_module.py =========================== test session starts ============================ @@ -482,7 +486,9 @@ server URL in its module namespace:: def test_showhelo(smtp_connection): assert 0, smtp_connection.helo() -Running it:: +Running it: + +.. code-block:: pytest $ pytest -qq --tb=short test_anothersmtp.py F [100%] @@ -584,7 +590,9 @@ The main change is the declaration of ``params`` with :py:func:`@pytest.fixture <_pytest.python.fixture>`, a list of values for each of which the fixture function will execute and can access a value via ``request.param``. No test function code needs to change. -So let's just do another run:: +So let's just do another run: + +.. code-block:: pytest $ pytest -q test_module.py FFFF [100%] @@ -686,7 +694,9 @@ a function which will be called with the fixture value and then has to return a string to use. In the latter case if the function return ``None`` then pytest's auto-generated ID will be used. -Running the above tests results in the following test IDs being used:: +Running the above tests results in the following test IDs being used: + +.. code-block:: pytest $ pytest --collect-only =========================== test session starts ============================ @@ -728,7 +738,9 @@ Example:: def test_data(data_set): pass -Running this test will *skip* the invocation of ``data_set`` with value ``2``:: +Running this test will *skip* the invocation of ``data_set`` with value ``2``: + +.. code-block:: pytest $ pytest test_fixture_marks.py -v =========================== test session starts ============================ @@ -771,7 +783,9 @@ and instantiate an object ``app`` where we stick the already defined assert app.smtp_connection Here we declare an ``app`` fixture which receives the previously defined -``smtp_connection`` fixture and instantiates an ``App`` object with it. Let's run it:: +``smtp_connection`` fixture and instantiates an ``App`` object with it. Let's run it: + +.. code-block:: pytest $ pytest -v test_appsetup.py =========================== test session starts ============================ @@ -840,7 +854,9 @@ to show the setup/teardown flow:: print(" RUN test2 with otherarg %s and modarg %s" % (otherarg, modarg)) -Let's run the tests in verbose mode and with looking at the print-output:: +Let's run the tests in verbose mode and with looking at the print-output: + +.. code-block:: pytest $ pytest -v -s test_module.py =========================== test session starts ============================ @@ -942,7 +958,9 @@ and declare its use in a test module via a ``usefixtures`` marker:: Due to the ``usefixtures`` marker, the ``cleandir`` fixture will be required for the execution of each test method, just as if you specified a "cleandir" function argument to each of them. Let's run it -to verify our fixture is activated and the tests pass:: +to verify our fixture is activated and the tests pass: + +.. code-block:: pytest $ pytest -q .. [100%] @@ -1041,7 +1059,9 @@ which implies that all test methods in the class will use this fixture without a need to state it in the test function signature or with a class-level ``usefixtures`` decorator. -If we run it, we get two passing tests:: +If we run it, we get two passing tests: + +.. code-block:: pytest $ pytest -q .. [100%] diff --git a/doc/en/getting-started.rst b/doc/en/getting-started.rst index 97ae2abdc..22ffe7bd4 100644 --- a/doc/en/getting-started.rst +++ b/doc/en/getting-started.rst @@ -43,7 +43,9 @@ Create a simple test function with just four lines of code:: def test_answer(): assert func(3) == 5 -That’s it. You can now execute the test function:: +That’s it. You can now execute the test function: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -90,7 +92,9 @@ Use the ``raises`` helper to assert that some code raises an exception:: with pytest.raises(SystemExit): f() -Execute the test function with “quiet” reporting mode:: +Execute the test function with “quiet” reporting mode: + +.. code-block:: pytest $ pytest -q test_sysexit.py . [100%] @@ -111,7 +115,9 @@ Once you develop multiple tests, you may want to group them into a class. pytest x = "hello" assert hasattr(x, 'check') -``pytest`` discovers all tests following its :ref:`Conventions for Python test discovery `, so it finds both ``test_`` prefixed functions. There is no need to subclass anything. We can simply run the module by passing its filename:: +``pytest`` discovers all tests following its :ref:`Conventions for Python test discovery `, so it finds both ``test_`` prefixed functions. There is no need to subclass anything. We can simply run the module by passing its filename: + +.. code-block:: pytest $ pytest -q test_class.py .F [100%] @@ -141,7 +147,9 @@ Request a unique temporary directory for functional tests print(tmpdir) assert 0 -List the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory:: +List the name ``tmpdir`` in the test function signature and ``pytest`` will lookup and call a fixture factory to create the resource before performing the test function call. Before the test runs, ``pytest`` creates a unique-per-test-invocation temporary directory: + +.. code-block:: pytest $ pytest -q test_tmpdir.py F [100%] diff --git a/doc/en/index.rst b/doc/en/index.rst index 510df4a09..7c201fbd7 100644 --- a/doc/en/index.rst +++ b/doc/en/index.rst @@ -22,7 +22,9 @@ An example of a simple test: assert inc(3) == 5 -To execute it:: +To execute it: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ diff --git a/doc/en/parametrize.rst b/doc/en/parametrize.rst index d3bce1b58..099b531c2 100644 --- a/doc/en/parametrize.rst +++ b/doc/en/parametrize.rst @@ -50,7 +50,9 @@ to an expected output:: Here, the ``@parametrize`` decorator defines three different ``(test_input,expected)`` tuples so that the ``test_eval`` function will run three times using -them in turn:: +them in turn: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -99,7 +101,9 @@ for example with the builtin ``mark.xfail``:: def test_eval(test_input, expected): assert eval(test_input) == expected -Let's run this:: +Let's run this: + +.. code-block:: pytest $ pytest =========================== test session starts ============================ @@ -172,7 +176,9 @@ If we now pass two stringinput values, our test will run twice:: .. [100%] 2 passed in 0.12 seconds -Let's also run with a stringinput that will lead to a failing test:: +Let's also run with a stringinput that will lead to a failing test: + +.. code-block:: pytest $ pytest -q --stringinput="!" test_strings.py F [100%] @@ -194,7 +200,9 @@ As expected our test function fails. If you don't specify a stringinput it will be skipped because ``metafunc.parametrize()`` will be called with an empty parameter -list:: +list: + +.. code-block:: pytest $ pytest -q -rs test_strings.py s [100%] diff --git a/doc/en/requirements.txt b/doc/en/requirements.txt index e3cc47ed5..b9fdcb8f6 100644 --- a/doc/en/requirements.txt +++ b/doc/en/requirements.txt @@ -1,3 +1,4 @@ +pygments-pytest>=1.0.4 # pinning sphinx to 1.4.* due to search issues with rtd: # https://github.com/rtfd/readthedocs-sphinx-ext/issues/25 sphinx ==1.4.* diff --git a/doc/en/skipping.rst b/doc/en/skipping.rst index 075886a96..ae1dc7149 100644 --- a/doc/en/skipping.rst +++ b/doc/en/skipping.rst @@ -323,7 +323,9 @@ Here is a simple test file with the several usages: .. literalinclude:: example/xfail_demo.py -Running it with the report-on-xfail option gives this output:: +Running it with the report-on-xfail option gives this output: + +.. code-block:: pytest example $ pytest -rx xfail_demo.py =========================== test session starts ============================ diff --git a/doc/en/tmpdir.rst b/doc/en/tmpdir.rst index 12a3e5641..8c21e17e5 100644 --- a/doc/en/tmpdir.rst +++ b/doc/en/tmpdir.rst @@ -35,7 +35,9 @@ created in the `base temporary directory`_. assert 0 Running this would result in a passed test except for the last -``assert 0`` line which we use to look at values:: +``assert 0`` line which we use to look at values: + +.. code-block:: pytest $ pytest test_tmp_path.py =========================== test session starts ============================ @@ -95,7 +97,9 @@ and more. Here is an example test usage:: assert 0 Running this would result in a passed test except for the last -``assert 0`` line which we use to look at values:: +``assert 0`` line which we use to look at values: + +.. code-block:: pytest $ pytest test_tmpdir.py =========================== test session starts ============================ diff --git a/doc/en/unittest.rst b/doc/en/unittest.rst index 5a6561d4b..34c8a35db 100644 --- a/doc/en/unittest.rst +++ b/doc/en/unittest.rst @@ -122,7 +122,9 @@ fixture definition:: The ``@pytest.mark.usefixtures("db_class")`` class-decorator makes sure that the pytest fixture function ``db_class`` is called once per class. Due to the deliberately failing assert statements, we can take a look at -the ``self.db`` values in the traceback:: +the ``self.db`` values in the traceback: + +.. code-block:: pytest $ pytest test_unittest_db.py =========================== test session starts ============================ @@ -199,7 +201,9 @@ used for all methods of the class where it is defined. This is a shortcut for using a ``@pytest.mark.usefixtures("initdir")`` marker on the class like in the previous example. -Running this test module ...:: +Running this test module ...: + +.. code-block:: pytest $ pytest -q test_unittest_cleandir.py . [100%] diff --git a/doc/en/usage.rst b/doc/en/usage.rst index 43f20394f..6c42cd0ec 100644 --- a/doc/en/usage.rst +++ b/doc/en/usage.rst @@ -150,7 +150,9 @@ Detailed summary report The ``-r`` flag can be used to display test results summary at the end of the test session, making it easy in large test suites to get a clear picture of all failures, skips, xfails, etc. -Example:: +Example: + +.. code-block:: pytest $ pytest -ra =========================== test session starts ============================ @@ -173,7 +175,9 @@ Here is the full list of available characters that can be used: - ``P`` - passed with output - ``a`` - all except ``pP`` -More than one character can be used, so for example to only see failed and skipped tests, you can execute:: +More than one character can be used, so for example to only see failed and skipped tests, you can execute: + +.. code-block:: pytest $ pytest -rfs =========================== test session starts ============================ diff --git a/doc/en/warnings.rst b/doc/en/warnings.rst index 182aae4fa..3e69d3480 100644 --- a/doc/en/warnings.rst +++ b/doc/en/warnings.rst @@ -18,7 +18,9 @@ and displays them at the end of the session:: def test_one(): assert api_v1() == 1 -Running pytest now produces this output:: +Running pytest now produces this output: + +.. code-block:: pytest $ pytest test_show_warnings.py =========================== test session starts ============================ @@ -37,7 +39,9 @@ Running pytest now produces this output:: =================== 1 passed, 1 warnings in 0.12 seconds =================== The ``-W`` flag can be passed to control which warnings will be displayed or even turn -them into errors:: +them into errors: + +.. code-block:: pytest $ pytest -q test_show_warnings.py -W error::UserWarning F [100%] @@ -347,7 +351,7 @@ defines an ``__init__`` constructor, as this prevents the class from being insta def test_foo(self): assert 1 == 1 -:: +.. code-block:: pytest $ pytest test_pytest_warnings.py -q diff --git a/doc/en/writing_plugins.rst b/doc/en/writing_plugins.rst index bee3d6102..70bf315aa 100644 --- a/doc/en/writing_plugins.rst +++ b/doc/en/writing_plugins.rst @@ -388,26 +388,27 @@ return a result object, with which we can assert the tests' outcomes. additionally it is possible to copy examples for an example folder before running pytest on it -.. code:: ini +.. code-block:: ini # content of pytest.ini [pytest] pytester_example_dir = . -.. code:: python +.. code-block:: python # content of test_example.py def test_plugin(testdir): - testdir.copy_example("test_example.py") - testdir.runpytest("-k", "test_example") + testdir.copy_example("test_example.py") + testdir.runpytest("-k", "test_example") + def test_example(): - pass + pass -.. code:: +.. code-block:: pytest $ pytest =========================== test session starts ============================ diff --git a/scripts/appveyor-retry.cmd b/scripts/appveyor-retry.cmd new file mode 100644 index 000000000..ac3836508 --- /dev/null +++ b/scripts/appveyor-retry.cmd @@ -0,0 +1,21 @@ +@echo off +rem Source: https://github.com/appveyor/ci/blob/master/scripts/appveyor-retry.cmd +rem initiate the retry number +set retryNumber=0 +set maxRetries=3 + +:RUN +%* +set LastErrorLevel=%ERRORLEVEL% +IF %LastErrorLevel% == 0 GOTO :EOF +set /a retryNumber=%retryNumber%+1 +IF %reTryNumber% == %maxRetries% (GOTO :FAILED) + +:RETRY +set /a retryNumberDisp=%retryNumber%+1 +@echo Command "%*" failed with exit code %LastErrorLevel%. Retrying %retryNumberDisp% of %maxRetries% +GOTO :RUN + +: FAILED +@echo Sorry, we tried running command for %maxRetries% times and all attempts were unsuccessful! +EXIT /B %LastErrorLevel% diff --git a/scripts/upload-coverage.bat b/scripts/upload-coverage.bat index 97553676c..8a8d94c98 100644 --- a/scripts/upload-coverage.bat +++ b/scripts/upload-coverage.bat @@ -5,7 +5,7 @@ if not defined PYTEST_NO_COVERAGE ( C:\Python36\Scripts\coverage combine C:\Python36\Scripts\coverage xml --ignore-errors C:\Python36\Scripts\coverage report -m --ignore-errors - C:\Python36\Scripts\codecov --required -X gcov pycov search -f coverage.xml --flags %TOXENV:-= % windows + scripts\appveyor-retry C:\Python36\Scripts\codecov --required -X gcov pycov search -f coverage.xml --flags %TOXENV:-= % windows ) else ( echo Skipping coverage upload, PYTEST_NO_COVERAGE is set ) diff --git a/tox.ini b/tox.ini index 4d2d910ab..65e40116a 100644 --- a/tox.ini +++ b/tox.ini @@ -124,10 +124,7 @@ setenv = {[testenv:py27-pluggymaster]setenv} skipsdist = True usedevelop = True changedir = doc/en -deps = - PyYAML - sphinx - sphinxcontrib-trio +deps = -r{toxinidir}/doc/en/requirements.txt commands = sphinx-build -W -b html . _build