From 53a8d20d88af3ff9f7fb6f4805be18489cce8335 Mon Sep 17 00:00:00 2001 From: Andy Freeland Date: Mon, 14 Apr 2014 12:27:55 -0400 Subject: [PATCH 1/2] fix issue499: document selecting tests by node ID --HG-- branch : issue499 --- doc/en/example/markers.txt | 84 ++++++++++++++++++++++++++++++++------ doc/en/usage.txt | 3 ++ 2 files changed, 74 insertions(+), 13 deletions(-) diff --git a/doc/en/example/markers.txt b/doc/en/example/markers.txt index 52322a6fa..9a9ef621a 100644 --- a/doc/en/example/markers.txt +++ b/doc/en/example/markers.txt @@ -21,6 +21,9 @@ You can "mark" a test function with custom metadata like this:: pass def test_another(): pass + class TestClass: + def test_method(self): + pass .. versionadded:: 2.2 @@ -29,25 +32,79 @@ You can then restrict a test run to only run tests marked with ``webtest``:: $ py.test -v -m webtest =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python - collecting ... collected 3 items + collecting ... collected 4 items test_server.py:3: test_send_http PASSED - =================== 2 tests deselected by "-m 'webtest'" =================== - ================== 1 passed, 2 deselected in 0.01 seconds ================== + =================== 3 tests deselected by "-m 'webtest'" =================== + ================== 1 passed, 3 deselected in 0.01 seconds ================== Or the inverse, running all tests except the webtest ones:: $ py.test -v -m "not webtest" =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python - collecting ... collected 3 items + collecting ... collected 4 items test_server.py:6: test_something_quick PASSED test_server.py:8: test_another PASSED + test_server.py:11: TestClass.test_method PASSED ================= 1 tests deselected by "-m 'not webtest'" ================= - ================== 2 passed, 1 deselected in 0.01 seconds ================== + ================== 3 passed, 1 deselected in 0.01 seconds ================== + +Selecing tests based on their node ID +------------------------------------- + +You can provide one or more node IDs as positional arguments to select +only specified tests. This makes it easy to select tests based on +their module, class, method, or function name:: + + $ py.test -v test_server.py::TestClass::test_method + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python + collecting ... collected 4 items + + test_server.py:11: TestClass.test_method PASSED + + ========================= 1 passed in 0.01 seconds ========================= + +You can also select on the class:: + + $ py.test -v test_server.py::TestClass + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python + collecting ... collected 4 items + + test_server.py:11: TestClass.test_method PASSED + + ========================= 1 passed in 0.01 seconds ========================= + +Or select multiple nodes:: + + $ py.test -v test_server.py::TestClass test_server.py::test_send_http + =========================== test session starts ============================ + platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python + collecting ... collected 8 items + + test_server.py:11: TestClass.test_method PASSED + test_server.py:3: test_send_http PASSED + + ========================= 2 passed in 0.01 seconds ========================= + +.. note:: + + Node IDs are of the form ``module.py::class::method`` or + ``module.py::function``. Node IDs control which tests are + collected, so ``module.py::class`` will select all test methods + on the class. Nodes are also created for each parameter of a + parametrized fixture or test, so selecting a parametrized test + must include the parameter value, e.g. + ``module.py::function[param]``. + + Node IDs for failing tests are displayed in the test summary info + when running py.test with the ``-rf`` option. You can also + construct Node IDs from the output of ``py.test --collectonly``. Using ``-k expr`` to select tests based on their name ------------------------------------------------------- @@ -62,38 +119,39 @@ select tests based on their names:: $ py.test -v -k http # running with the above defined example module =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python - collecting ... collected 3 items + collecting ... collected 4 items test_server.py:3: test_send_http PASSED - ====================== 2 tests deselected by '-khttp' ====================== - ================== 1 passed, 2 deselected in 0.01 seconds ================== + ====================== 3 tests deselected by '-khttp' ====================== + ================== 1 passed, 3 deselected in 0.01 seconds ================== And you can also run all tests except the ones that match the keyword:: $ py.test -k "not send_http" -v =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python - collecting ... collected 3 items + collecting ... collected 4 items test_server.py:6: test_something_quick PASSED test_server.py:8: test_another PASSED + test_server.py:11: TestClass.test_method PASSED ================= 1 tests deselected by '-knot send_http' ================== - ================== 2 passed, 1 deselected in 0.01 seconds ================== + ================== 3 passed, 1 deselected in 0.01 seconds ================== Or to select "http" and "quick" tests:: $ py.test -k "http or quick" -v =========================== test session starts ============================ platform linux2 -- Python 2.7.3 -- py-1.4.20 -- pytest-2.5.2 -- /home/hpk/p/pytest/.tox/regen/bin/python - collecting ... collected 3 items + collecting ... collected 4 items test_server.py:3: test_send_http PASSED test_server.py:6: test_something_quick PASSED - ================= 1 tests deselected by '-khttp or quick' ================== - ================== 2 passed, 1 deselected in 0.01 seconds ================== + ================= 2 tests deselected by '-khttp or quick' ================== + ================== 2 passed, 2 deselected in 0.01 seconds ================== .. note:: diff --git a/doc/en/usage.txt b/doc/en/usage.txt index 761be48a7..b03b9b9d6 100644 --- a/doc/en/usage.txt +++ b/doc/en/usage.txt @@ -49,6 +49,9 @@ Several test run options:: # the "string expression", e.g. "MyClass and not method" # will select TestMyClass.test_something # but not TestMyClass.test_method_simple + py.test test_mod.py::test_func # only run tests that match the "node ID", + # e.g "test_mod.py::test_func" will select + # only test_func in test_mod.py Import 'pkg' and use its filesystem location to find and run tests:: From 1728798e8105661e0fa8aa372ec09ad7585264ad Mon Sep 17 00:00:00 2001 From: Andy Freeland Date: Mon, 14 Apr 2014 14:24:13 -0400 Subject: [PATCH 2/2] Interal link to node ID explanation --HG-- branch : issue499 --- doc/en/example/markers.txt | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/doc/en/example/markers.txt b/doc/en/example/markers.txt index 9a9ef621a..8b7aa067b 100644 --- a/doc/en/example/markers.txt +++ b/doc/en/example/markers.txt @@ -56,9 +56,9 @@ Or the inverse, running all tests except the webtest ones:: Selecing tests based on their node ID ------------------------------------- -You can provide one or more node IDs as positional arguments to select -only specified tests. This makes it easy to select tests based on -their module, class, method, or function name:: +You can provide one or more :ref:`node IDs ` as positional +arguments to select only specified tests. This makes it easy to select +tests based on their module, class, method, or function name:: $ py.test -v test_server.py::TestClass::test_method =========================== test session starts ============================ @@ -92,6 +92,9 @@ Or select multiple nodes:: ========================= 2 passed in 0.01 seconds ========================= + +.. _node-id: + .. note:: Node IDs are of the form ``module.py::class::method`` or