improve mark.txt document and add new regristration/markers features.

(welcome to documentation driven development)

doc/example/index.txt

@@ -18,5 +18,6 @@ need more examples or have questions. Also take a look at the :ref:`comprehensiv
    simple.txt
    mysetup.txt
    parametrize.txt
+   markers.txt
    pythoncollection.txt
    nonpython.txt

doc/example/markers.txt

@@ -0,0 +1,83 @@
+
+Working with custom markers
+=================================================
+
+
+Here are some example using the :ref:`mark` mechanism.
+
+.. _`adding a custom marker from a plugin`:
+
+custom marker and command line option to control test runs
+----------------------------------------------------------
+
+Plugins can provide custom markers and implement specific behaviour
+based on it. This is a self-contained example which adds a command
+line option and a parametrized test function marker to run tests
+specifies via named environments::
+
+    # content of conftest.py
+
+    import pytest
+    def pytest_addoption(parser):
+        parser.addoption("-E", dest="env", action="store", metavar="NAME",
+            help="only run tests matching the environment NAME.")
+
+    def pytest_configure(config):
+        # register an additional marker
+        config.addinivalue_line("markers", 
+            "env(name): mark test to run only on named environment")
+
+    def pytest_runtest_setup(item):
+        if not isinstance(item, item.Function):
+            return
+        if hasattr(item.obj, 'env'):
+            envmarker = getattr(item.obj, 'env')
+            envname = envmarker.args[0]
+            if envname != item.config.option.env:
+                pytest.skip("test requires env %r" % envname)
+
+A test file using this local plugin::
+
+    # content of test_someenv.py
+
+    import pytest
+    @pytest.mark.env("stage1")
+    def test_basic_db_operation():
+        pass
+
+and an example invocations specifying a different environment than what
+the test needs::
+
+    $ py.test -E stage2
+    ============================= test session starts ==============================
+    platform darwin -- Python 2.7.1 -- pytest-2.2.0.dev6
+    collecting ... collected 1 items
+    
+    test_someenv.py s
+    
+    ========================== 1 skipped in 0.02 seconds ===========================
+  
+and here is one that specifies exactly the environment needed::
+
+    $ py.test -E stage1
+    ============================= test session starts ==============================
+    platform darwin -- Python 2.7.1 -- pytest-2.2.0.dev6
+    collecting ... collected 1 items
+    
+    test_someenv.py .
+    
+    =========================== 1 passed in 0.02 seconds ===========================
+
+The ``--markers`` option always gives you a list of available markers::
+
+    $ py.test --markers
+    @pytest.mark.env(name): mark test to run only on named environment
+    
+    @pytest.mark.skipif(*conditions): skip the given test function if evaluation of all conditions has a True value.  Evaluation happens within the module global context. Example: skipif('sys.platform == "win32"') skips the test if we are on the win32 platform. 
+    
+    @pytest.mark.xfail(*conditions, reason=None, run=True): mark the the test function as an expected failure. Optionally specify a reason and run=False if you don't even want to execute the test function. Any positional condition strings will be evaluated (like with skipif) and if one is False the marker will not be applied.
+    
+    @pytest.mark.tryfirst: mark a hook implementation function such that the plugin machinery will try to call it first/as early as possible.
+    
+    @pytest.mark.trylast: mark a hook implementation function such that the plugin machinery will try to call it last/as late as possible.
+    

doc/mark.txt

@@ -6,37 +6,71 @@ Marking test functions with attributes
 
 .. currentmodule:: _pytest.mark
 
-By using the ``pytest.mark`` helper you can instantiate
+By using the ``pytest.mark`` helper you can easily set
-decorators that will set named metadata on test functions.
+metadata on your test functions. To begin with, there are
+some builtin markers, for example:
 
-Marking a single function
+* skipif - skip a test function if a certain condition is met
+* xfail - produce an "expected failure" outcome if a certain
+  condition is met
+
+It's also easy to create custom markers or to apply markers
+to whole test classes or modules.
+
+marking test functions and selecting them for a run
 ----------------------------------------------------
 
-You can "mark" a test function with metadata like this::
+You can "mark" a test function with custom metadata like this::
+
+    # content of test_server.py
 
     import pytest
     @pytest.mark.webtest
     def test_send_http():
-        ...
+        pass # perform some webtest test for your app
 
-This will set the function attribute ``webtest`` to a :py:class:`MarkInfo`
+.. versionadded:: 2.2
-instance.  You can also specify parametrized metadata like this::
 
-    # content of test_mark.py
+You can restrict a test run only tests marked with ``webtest`` like this::
 
-    import pytest
+    $ py.test -m webtest
-    @pytest.mark.webtest(firefox=30)
-    def test_receive():
-        pass
 
-    @pytest.mark.webtest("functional", firefox=30)
+Or the inverse, running all tests except the webtest ones::
-    def test_run_and_look():
+    
-        pass
+    $ py.test -m "not webtest"
 
-and access it from other places like this::
+Registering markers
+-------------------------------------
 
-    test_receive.webtest.kwargs['firefox'] == 30
+.. versionadded:: 2.2
-    test_run_and_look.webtest.args[0] == "functional"
+
+.. ini-syntax for custom markers:
+
+Registering markers for your test suite is simple::
+
+    # content of pytest.ini
+    [pytest]
+    markers = 
+        webtest: mark a test as a webtest. 
+
+You can ask which markers exist for your test suite::
+
+    $ py.test --markers
+
+For an example on how to add and work markers from a plugin, see 
+:ref:`adding a custom marker from a plugin`.
+
+.. note::
+
+    It is recommended to explicitely register markers so that:
+
+    * there is one place in your test suite defining your markers
+
+    * asking for existing markers via ``py.test --markers`` gives good output
+
+    * typos in function markers can be treated as an error if you use
+      the :ref:`--strict` option.  Later versions of py.test might treat
+      non-registered markers as an error by default.
 
 .. _`scoped-marking`:
 
@@ -58,7 +92,7 @@ with classes to apply markers to all of its test methods::
 This is equivalent to directly applying the decorator to the
 two test functions.
 
-To remain compatible with Python2.5 you can also set a
+To remain backward-compatible with Python2.4 you can also set a
 ``pytestmark`` attribute on a TestClass like this::
 
     import pytest

doc/plugins.txt

@@ -327,7 +327,6 @@ test execution:
 
 .. autofunction: pytest_runtest_logreport
 
-
 Reference of important objects involved in hooks
 ===========================================================