refine documentation, move setup to own "setup" page and provide

some more examples. move old setup_module/... to xunit_old page.

doc/en/index.txt

@@ -25,7 +25,9 @@ Welcome to pytest!
 
 - **supports functional testing and complex test setups**
 
- - (new in 2.3) :ref:`easy test resource management and generalized xUnit setup <resources>`
+ - (new in 2.3) :ref:`easy test resource management, scoping and
+   parametrization <resources>`
+ - (new in 2.3) :ref:`xunitsetup`.
  - (new in 2.2) :ref:`durations`
  - (much improved in 2.2) :ref:`marking and test selection <mark>`
  - (improved in 2.2) :ref:`parametrized test functions <parametrized test functions>`

doc/en/resources.txt

@@ -1,74 +1,66 @@
 
 .. _resources:
 
-test resource management and xUnit setup (on steroids)
+=======================================================
+test resource injection and parametrization
 =======================================================
 
 .. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection
 
 .. versionadded: 2.3
 
-pytest offers advanced resource parametrization and injection mechanisms
+pytest offers very flexible means for managing test resources and
-including a fully integrated generalization of the popular xUnit
+test parametrization.
-setup-style methods.  A resource is created by a ``@pytest.mark.factory`` 
-marked function and its name is the name of the function.  A resource
-is injected into test or setup functions if they use the name 
-in their signature.  Therefore and also for historic reasons, resources are
-sometimes called "funcargs" because they ultimately appear as 
-function arguments.
-
-The pytest resource management and setup features are exposed through
-three decorators:
-
-* a `@pytest.mark.factory`_ marker to define resource factories,
-  their scoping and parametrization.
-
-* a `@pytest.mark.setup`_ marker to define setup functions and their 
-  scoping.
-
-* a `@pytest.mark.parametrize`_ marker for executing test functions
-  multiple times with different parameter sets
-
-Generally, resource factories and setup functions: 
-
-- can be defined in test modules, test classes, conftest.py files or
-  in plugins.
-
-- can themselves receive resources through their function arguments, 
-  simplifying the setup and use of interdependent resources.
-
-- can use the special `testcontext`_ object for access to the 
-  context in which the factory/setup is called and for registering 
-  finalizers.
-
-This document showcases these features through some basic examples.
-
-Note that pytest also comes with some :ref:`builtinresources` which
-you can use without defining them yourself.
-
-Background and terms
----------------------------
 
 The pytest resource management mechanism is an example of `Dependency
-Injection`_ which helps to de-couple test code from resource
+Injection`_ because test and :ref:`setup functions <xunitsetup>` receive
-instantiation code required for them to execute.  At test writing time
+resources simply by stating them as an input argument.  Therefore and
-you typically do not need to care for the details of how your required
+also for historic reasons, they are often called **funcargs**.  At test
-resources are constructed, if they live through a function, class,
+writing time you typically do not need to care for the details of how
-module or session scope or if the test will be called multiple times
+your required resources are constructed, if they live through a
-with different resource instances.
+function, class, module or session scope or if the test will be called
+multiple times with different resource instances.
 
 To create a value with which to call a test function a resource factory 
 function is called which gets full access to the test context and can
 register finalizers which are to be run after the last test in that context
 finished. Resource factories can be implemented in same test class or 
-test module, in a per-directory ``conftest.py`` file or in an external plugin.  This allows total de-coupling of test and setup code.
+test module, in a per-directory ``conftest.py`` file or in an external
+plugin.  This allows **total de-coupling of test and setup code**, 
+lowering the cost of refactoring.
 
 A test function may be invoked multiple times in which case we
-speak of :ref:`parametrized testing <parametrizing-tests>`. This can be
+speak of parametrization.  You can parametrize resources or parametrize
-very useful if you want to test e.g. against different database backends
+test function arguments directly or even implement your own parametrization
-or with multiple numerical arguments sets and want to reuse the same set
+scheme through a plugin hook.
-of test functions.
 
+A resource has a **name** under which test and setup functions 
+can access it by listing it as an input argument.  Due to this and
+also for historic reasons, resources are often called **funcargs**.
+A resource is created by a factory which can be flagged with a **scope**
+to only create resources on a per-class/per-module/per-session basis
+instead of the default per-function scope.
+
+Concretely, there are three means of resource and parametrization management:
+
+* a `@pytest.mark.factory`_ marker to define resource factories,
+  their scoping and parametrization.   Factories can themselves 
+  receive resources through their function arguments, easing
+  the setup of interdependent resources.  They can also use
+  the special `testcontext`_ object to access details n which
+  the factory/setup is called and for registering finalizers.
+
+* a `@pytest.mark.parametrize`_ marker for executing test functions
+  multiple times with different parameter sets
+
+* a `pytest_generate_tests`_ plugin hook marker for implementing
+  your parametrization for a test function which may depend on 
+  command line options, class/module attributes etc.
+
+Finally, pytest comes with some :ref:`builtinresources` which
+you can use without defining them yourself. Moreover, third-party
+plugins offer their own resources so that after installation
+you can simply use them in your test and setup functions.
 
 .. _`@pytest.mark.factory`:
 
@@ -81,11 +73,12 @@ of test functions.
 
 The `@pytest.mark.factory`_ marker allows to
 
-* mark a function as a factory for resources used by test and setup functions 
+* mark a function as a factory for resources, useable by test and setup functions 
-* define parametrization to run tests multiple times with different
+* define parameters in order to run tests multiple times with different
   resource instances
-* set a scope which determines the level of caching. valid scopes
+* set a scope which determines the level of caching, i.e. how often
-  are ``session``, ``module``, ``class`` and ``function``.
+  the factory will be called. Valid scopes are ``session``, ``module``, 
+  ``class`` and ``function``.
 
 Here is a simple example of a factory creating a shared ``smtplib.SMTP``
 connection resource which test functions then may use across the whole
@@ -248,12 +241,12 @@ Note that pytest orders your test run by resource usage, minimizing
 the number of active resources at any given time.
 
 
-Accessing resources from a factory function
+Interdepdendent resources
 ----------------------------------------------------------
 
-You can directly use resources as funcargs in resource factories.  
+You can not only use resources in test functions but also in resource factories
-Extending the previous example we can instantiate an application
+themselves.  Extending the previous example we can instantiate an application
-object and stick the live ``smtp`` resource into it::
+object by sticking the ``smtp`` resource into it::
 
     # content of test_appsetup.py
    
@@ -289,114 +282,6 @@ run twice with two different ``App`` instances and respective smtp servers.
 There is no need for the ``app`` factory to be aware of the parametrization.
 
 
-.. _`new_setup`:
-.. _`@pytest.mark.setup`:
-
-``@pytest.mark.setup``: xUnit setup methods on steroids
------------------------------------------------------------------
-
-.. regendoc:wipe
-
-.. versionadded:: 2.3
-
-The ``@pytest.mark.setup`` marker allows 
-
-* to define setup-functions close to test code or in conftest.py files
-  or plugins.
-* to mark a function as a setup method; the function can itself
-  receive funcargs and will execute multiple times if the funcargs
-  are parametrized
-* to set a scope which influences when the setup function going to be
-  called.  valid scopes are ``session``, ``module``, ``class`` and ``function``.
-
-Here is a simple example.  First we define a global ``globdir`` resource::
-
-    # content of conftest.py
-    import pytest
-
-    @pytest.mark.factory(scope="module")
-    def globdir(testcontext, tmpdir):
-        def fin():
-            print "finalize", tmpdir
-        testcontext.addfinalizer(fin)
-        print "created resource", tmpdir
-        return tmpdir
-
-And then we write a test file containing a setup-marked function 
-taking this resource and setting it as a module global::
-
-    # content of test_module.py
-    import pytest
-
-    @pytest.mark.setup(scope="module")
-    def setresource(testcontext, globdir):
-        print "setupresource", globdir
-        testcontext.module.myresource = globdir
-
-    def test_1():
-        assert myresource
-        print "using myresource", myresource
-
-    def test_2():
-        assert myresource
-        print "using myresource", myresource
-
-Let's run this module::
-
-    $ py.test -qs
-    collecting ... collected 2 items
-    ..
-    2 passed in 0.26 seconds
-    created resource /home/hpk/tmp/pytest-4427/test_10
-    setupresource /home/hpk/tmp/pytest-4427/test_10
-    using myresource /home/hpk/tmp/pytest-4427/test_10
-    using myresource /home/hpk/tmp/pytest-4427/test_10
-    finalize /home/hpk/tmp/pytest-4427/test_10
-
-The two test functions in the module use the same global ``myresource``
-object because the ``setresource`` set it as a module attribute.
-
-The ``globdir`` factory can now become parametrized without any test
-or setup code needing to change::
-
-    # content of conftest.py
-    import pytest
-
-    @pytest.mark.factory(scope="module", params=["aaa", "bbb"])
-    def globdir(testcontext, tmpdir):
-        newtmp = tmpdir.join(testcontext.param)
-        def fin():
-            print "finalize", newtmp
-        testcontext.addfinalizer(fin)
-        print "created resource", newtmp
-        return newtmp
-
-Running the unchanged previous test files now runs four tests::
-
-    $ py.test -qs
-    collecting ... collected 4 items
-    ....
-    4 passed in 0.26 seconds
-    created resource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
-    setupresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
-    using myresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
-    using myresource /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
-    finalize /home/hpk/tmp/pytest-4428/test_1_aaa_0/aaa
-    created resource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-    setupresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-    using myresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-    using myresource /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-    finalize /home/hpk/tmp/pytest-4428/test_1_bbb_0/bbb
-
-Each parameter causes the creation of a respective resource and the
-unchanged test module uses it in its ``@setup`` decorated method.
-
-.. note::
-
-   Tests using a particular parametrized resource instance will
-   executed next to each other.  Any finalizers will be run before the 
-   next parametrized resource instance is being setup and tests
-   are rerun.
 
 Grouping tests by resource parameters
 ----------------------------------------------------------
@@ -540,7 +425,10 @@ As expected only one pair of input/output values fails the simple test function.
 Note that there are various ways how you can mark groups of functions,
 see :ref:`mark`.
 
-Generating parameters combinations, depending on command line
+
+.. _`pytest_generate_tests`:
+
+``pytest_generate_test``: implementing your own parametrization scheme
 ----------------------------------------------------------------------------
 
 .. regendoc:wipe

doc/en/setup.txt

@@ -0,0 +1,204 @@
+.. _xunitsetup:
+.. _setup:
+
+``@setup`` functions or: xunit on steroids
+========================================================
+
+.. versionadded:: 2.3
+
+.. _`funcargs`: funcargs.html
+.. _`test parametrization`: funcargs.html#parametrizing-tests
+.. _`unittest plugin`: plugin/unittest.html
+.. _`xUnit`: http://en.wikipedia.org/wiki/XUnit
+
+Python, Java and many other languages support a so called xUnit_ style 
+of resource setup.  This typically involves the call of a ``setup``
+("fixture") method before running a test function and ``teardown`` after
+it has finished.  Unlike :ref:`injected resources <resources>` setup
+functions work indirectly by causing global side effects or 
+setting test case attributes which test methods can then access.  
+
+pytest originally introduced in 2005 a fine-grained model of detecting
+setup and teardown functions on a per-module, class or function basis.
+The Python unittest module and nose have subsequently incorporated them.
+This model remains supported as :ref:`old-style xunit`.
+
+With pytest-2.3 a new ``pytest.setup()`` decorator is introduced
+to mark functions as setup functions which:
+
+- can receive resources through funcargs,
+- fully interoperate with parametrized resources,
+- can be defined in a plugin or conftest.py file and get called 
+  on a per-session, per-module, per-class or per-function basis,
+- can access the full :ref:`testcontext` for which the setup is called,
+- can precisely control teardown by registering one or multiple 
+  teardown functions as soon as they have performed some actions 
+  which need undoing, eliminating the no need for a separate 
+  "teardown" decorator.
+- allow to separate different setup concerns even if they
+  happen to work in the same scope
+
+All of these features are now demonstrated by little examples.
+
+.. _`new_setup`:
+.. _`@pytest.mark.setup`:
+
+basic per-function setup
+-------------------------------
+
+.. regendoc:wipe
+
+Suppose you want to have a clean directory with a single
+file entry for each test function in a module and have
+the test execute with this directory as current working dir::
+
+    # content of test_funcdir.py
+    import pytest
+    import os
+
+    @pytest.mark.setup()
+    def mydir(tmpdir):
+        tmpdir.join("myfile").write("example content")
+        old = tmpdir.chdir()
+
+    def test_function1():
+        assert os.path.exists("myfile")
+        f = open("anotherfile", "w")
+        f.write("")
+        f.close()
+
+    def test_function2():
+        assert os.path.exists("myfile")
+        assert not os.path.exists("anotherfile")
+
+Our ``mydir`` setup function is executed on a per-function basis,
+the default scope used by the ``pytest.mark.setup`` decorator.
+It accesses the ``tmpdir`` resource which provides a new empty
+directory path object.  The ``test_function2`` here checks that 
+it executes with a fresh directory and specifically
+does not see the previously created ``anotherfile``.  We can
+thus expect two passing tests::
+
+    $ py.test -v
+    =========================== test session starts ============================
+    platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python
+    cachedir: /home/hpk/tmp/doc-exec-410/.cache
+    plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
+    collecting ... collected 2 items
+    
+    test_funcdir.py:9: test_function1 PASSED
+    test_funcdir.py:15: test_function2 PASSED
+    
+    ========================= 2 passed in 0.26 seconds =========================
+
+per-function setup, for every function of a project
+------------------------------------------------------------
+
+If you want to define a setup per-function but want to apply
+it to every function in your project you don't need to duplicate
+the setup-definition into each test module.  Instead you can put
+it into a ``conftest.py`` file into the root of your project::
+
+    # content of conftest.py
+    import pytest
+    import os
+
+    @pytest.mark.setup()
+    def cleandir(tmpdir):
+        old = tmpdir.chdir()
+
+The ``cleandir`` setup function will be called for every test function
+below the directory tree where ``conftest.py`` resides.  In this
+case it just uses the builtin ``tmpdir`` resource to change to the
+empty directory ahead of running a test.
+
+test modules accessing a global resource
+-------------------------------------------------------
+
+If you want test modules to access a global resource,
+you can stick the resource to the module globals in
+a per-module setup function.  We use a :ref:`resource factory
+<@pytest.mark.factory>` to create our global resource::
+
+    # content of conftest.py
+    import pytest
+
+    class GlobalResource:
+        def __init__(self):
+            pass
+
+    @pytest.mark.factory(scope="session")
+    def globresource():
+        return GlobalResource()
+
+    @pytest.mark.setup(scope="module")
+    def setresource(testcontext, globresource):
+        testcontext.module.globresource = globresource
+       
+Now any test module can access ``globresource`` as a module global::
+
+    # content of test_glob.py
+
+    def test_1():
+        print ("test_1 %s" % globresource)
+    def test_2():
+        print ("test_2 %s" % globresource)
+
+Let's run this module without output-capturing::
+
+    $ py.test -qs test_glob.py
+    collecting ... collected 2 items
+    ..
+    2 passed in 0.02 seconds
+    test_1 <conftest.GlobalResource instance at 0x13197e8>
+    test_2 <conftest.GlobalResource instance at 0x13197e8>
+
+The two tests see the same global ``globresource`` object.
+
+Parametrizing the global resource
++++++++++++++++++++++++++++++++++++++++++++++++++
+
+We extend the previous example and add parametrization to the globresource 
+factory and also add a finalizer::
+
+    # content of conftest.py
+
+    import pytest
+
+    class GlobalResource:
+        def __init__(self, param):
+            self.param = param
+
+    @pytest.mark.factory(scope="session", params=[1,2])
+    def globresource(testcontext):
+        g = GlobalResource(testcontext.param)
+        def fin():
+            print "finalizing", g
+        testcontext.addfinalizer(fin)
+        return g
+
+    @pytest.mark.setup(scope="module")
+    def setresource(testcontext, globresource):
+        testcontext.module.globresource = globresource
+
+And then re-run our test module::
+
+    $ py.test -qs test_glob.py
+    collecting ... collected 4 items
+    ....
+    4 passed in 0.02 seconds
+    test_1 <conftest.GlobalResource instance at 0x1922e18>
+    test_2 <conftest.GlobalResource instance at 0x1922e18>
+    finalizing <conftest.GlobalResource instance at 0x1922e18>
+    test_1 <conftest.GlobalResource instance at 0x1925518>
+    test_2 <conftest.GlobalResource instance at 0x1925518>
+    finalizing <conftest.GlobalResource instance at 0x1925518>
+
+We are now running the two tests twice with two different global resource
+instances.  Note that the tests are ordered such that only
+one instance is active at any given time: the finalizer of 
+the first globresource instance is called before the second
+instance is created and sent to the setup functions.
+
+
+

doc/en/xunit_old.txt

@@ -1,23 +1,16 @@
-.. _xunitsetup:
 
-====================================
+.. _`old-style xunit`:
-Extended xUnit style setup fixtures
-====================================
 
-.. _`funcargs`: funcargs.html
+Old-style xunit-style setup
-.. _`test parametrization`: funcargs.html#parametrizing-tests
+========================================
-.. _`unittest plugin`: plugin/unittest.html
-.. _`xUnit`: http://en.wikipedia.org/wiki/XUnit
-
-Python, Java and many other languages support xUnit_ style testing.
-This typically involves the call of a ``setup`` ("fixture") method
-before running a test function and ``teardown`` after it has finished.
-``py.test`` supports a more fine-grained model of setup/teardown
-handling by optionally calling per-module and per-class hooks.
 
+This section describes the old way how you can implement setup and
+teardown on a per-module/class/function basis.  It remains fully
+supported but it is recommended to rather use :ref:`@setup functions <setup>` 
+or :ref:`injected resources <resources>` for implementing your setup needs.
 
 Module level setup/teardown
-=============================================
+--------------------------------------
 
 If you have multiple test functions and test classes in a single
 module you can optionally implement the following fixture methods
@@ -32,7 +25,7 @@ which will usually be called once for all the functions::
         """
 
 Class level setup/teardown
-=============================================
+----------------------------------
 
 Similarly, the following methods are called at class level before
 and after all test methods of the class are called::
@@ -50,7 +43,7 @@ and after all test methods of the class are called::
         """
 
 Method and function level setup/teardown
-=============================================
+-----------------------------------------------
 
 Similarly, the following methods are called around each method invocation::