rework docs to demonstrate and discuss current yield syntax in more depth.

doc/en/apiref.txt

@@ -11,6 +11,7 @@ py.test reference documentation
    customize.txt
    assert.txt
    fixture.txt
+   yieldfixture.txt
    parametrize.txt
    xunit_setup.txt
    capture.txt

doc/en/fixture.txt

@@ -79,7 +79,7 @@ marked ``smtp`` fixture function.  Running the test looks like this::
     ================================= FAILURES =================================
     ________________________________ test_ehlo _________________________________
     
-    smtp = <smtplib.SMTP instance at 0x1ac66c8>
+    smtp = <smtplib.SMTP instance at 0x22530e0>
     
         def test_ehlo(smtp):
             response, msg = smtp.ehlo()
@@ -198,7 +198,7 @@ inspect what is going on and can now run the tests::
     ================================= FAILURES =================================
     ________________________________ test_ehlo _________________________________
     
-    smtp = <smtplib.SMTP instance at 0x15b2d88>
+    smtp = <smtplib.SMTP instance at 0x165fa28>
     
         def test_ehlo(smtp):
             response = smtp.ehlo()
@@ -210,7 +210,7 @@ inspect what is going on and can now run the tests::
     test_module.py:6: AssertionError
     ________________________________ test_noop _________________________________
     
-    smtp = <smtplib.SMTP instance at 0x15b2d88>
+    smtp = <smtplib.SMTP instance at 0x165fa28>
     
         def test_noop(smtp):
             response = smtp.noop()
@@ -219,7 +219,7 @@ inspect what is going on and can now run the tests::
     E       assert 0
     
     test_module.py:11: AssertionError
-    ========================= 2 failed in 0.16 seconds =========================
+    ========================= 2 failed in 0.18 seconds =========================
 
 You see the two ``assert 0`` failing and more importantly you can also see 
 that the same (module-scoped) ``smtp`` object was passed into the two 
@@ -266,8 +266,7 @@ Let's execute it::
 
     $ py.test -s -q --tb=no
     FF
-    2 failed in 0.16 seconds
+    2 failed in 0.20 seconds
-    teardown smtp
 
 We see that the ``smtp`` instance is finalized after the two
 tests finished execution.  Note that if we decorated our fixture
@@ -276,8 +275,9 @@ occur around each single test.  In either case the test
 module itself does not need to change or know about these details
 of fixture setup.
 
-Note that pytest-2.4 introduced an alternative `yield-context <yieldctx>`_ 
+Note that pytest-2.4 introduced an experimental alternative
-mechanism which allows to interact nicely with context managers.
+:ref:`yield fixture mechanism <yieldctx>` for easier context manager integration
+and more linear writing of teardown code.
 
 .. _`request-context`:
 
@@ -310,8 +310,7 @@ again, nothing much has changed::
 
     $ py.test -s -q --tb=no
     FF
-    2 failed in 0.17 seconds
+    2 failed in 0.18 seconds
-    teardown smtp
 
 Let's quickly create another test module that actually sets the
 server URL in its module namespace::
@@ -331,7 +330,7 @@ Running it::
     ______________________________ test_showhelo _______________________________
     test_anothersmtp.py:5: in test_showhelo
     >       assert 0, smtp.helo()
-    E       AssertionError: (250, 'hq.merlinux.eu')
+    E       AssertionError: (250, 'mail.python.org')
 
 voila! The ``smtp`` fixture function picked up our mail server name
 from the module namespace.
@@ -378,7 +377,7 @@ So let's just do another run::
     ================================= FAILURES =================================
     __________________________ test_ehlo[merlinux.eu] __________________________
     
-    smtp = <smtplib.SMTP instance at 0x1d1b680>
+    smtp = <smtplib.SMTP instance at 0x28d13b0>
     
         def test_ehlo(smtp):
             response = smtp.ehlo()
@@ -390,7 +389,7 @@ So let's just do another run::
     test_module.py:6: AssertionError
     __________________________ test_noop[merlinux.eu] __________________________
     
-    smtp = <smtplib.SMTP instance at 0x1d1b680>
+    smtp = <smtplib.SMTP instance at 0x28d13b0>
     
         def test_noop(smtp):
             response = smtp.noop()
@@ -401,7 +400,7 @@ So let's just do another run::
     test_module.py:11: AssertionError
     ________________________ test_ehlo[mail.python.org] ________________________
     
-    smtp = <smtplib.SMTP instance at 0x1d237e8>
+    smtp = <smtplib.SMTP instance at 0x28d8440>
     
         def test_ehlo(smtp):
             response = smtp.ehlo()
@@ -412,7 +411,7 @@ So let's just do another run::
     test_module.py:5: AssertionError
     ________________________ test_noop[mail.python.org] ________________________
     
-    smtp = <smtplib.SMTP instance at 0x1d237e8>
+    smtp = <smtplib.SMTP instance at 0x28d8440>
     
         def test_noop(smtp):
             response = smtp.noop()
@@ -421,7 +420,7 @@ So let's just do another run::
     E       assert 0
     
     test_module.py:11: AssertionError
-    4 failed in 6.04 seconds
+    4 failed in 6.47 seconds
 
 We see that our two test functions each ran twice, against the different
 ``smtp`` instances.  Note also, that with the ``mail.python.org`` 
@@ -462,14 +461,14 @@ Here we declare an ``app`` fixture which receives the previously defined
     $ py.test -v test_appsetup.py
     =========================== test session starts ============================
     platform linux2 -- Python 2.7.3 -- pytest-2.4.0.dev12 -- /home/hpk/venv/0/bin/python
-    cachedir: /tmp/doc-exec-120/.cache
+    cachedir: /tmp/doc-exec-127/.cache
     plugins: xdist, pep8, cov, cache, capturelog, instafail
     collecting ... collected 2 items
     
     test_appsetup.py:12: test_smtp_exists[mail.python.org] PASSED
     test_appsetup.py:12: test_smtp_exists[merlinux.eu] PASSED
     
-    ========================= 2 passed in 6.98 seconds =========================
+    ========================= 2 passed in 6.07 seconds =========================
 
 Due to the parametrization of ``smtp`` the test will run twice with two
 different ``App`` instances and respective smtp servers.  There is no
@@ -528,30 +527,30 @@ Let's run the tests in verbose mode and with looking at the print-output::
     $ py.test -v -s test_module.py
     =========================== test session starts ============================
     platform linux2 -- Python 2.7.3 -- pytest-2.4.0.dev12 -- /home/hpk/venv/0/bin/python
-    cachedir: /tmp/doc-exec-120/.cache
+    cachedir: /tmp/doc-exec-127/.cache
     plugins: xdist, pep8, cov, cache, capturelog, instafail
     collecting ... collected 8 items
     
-    test_module.py:15: test_0[1] PASSED
+    test_module.py:15: test_0[1]   test0 1
-    test_module.py:15: test_0[2] PASSED
+    PASSED
-    test_module.py:17: test_1[mod1] PASSED
+    test_module.py:15: test_0[2]   test0 2
-    test_module.py:19: test_2[1-mod1] PASSED
+    PASSED
-    test_module.py:19: test_2[2-mod1] PASSED
+    test_module.py:17: test_1[mod1] create mod1
-    test_module.py:17: test_1[mod2] PASSED
+      test1 mod1
-    test_module.py:19: test_2[1-mod2] PASSED
+    PASSED
-    test_module.py:19: test_2[2-mod2] PASSED
+    test_module.py:19: test_2[1-mod1]   test2 1 mod1
+    PASSED
+    test_module.py:19: test_2[2-mod1]   test2 2 mod1
+    PASSED
+    test_module.py:17: test_1[mod2] create mod2
+      test1 mod2
+    PASSED
+    test_module.py:19: test_2[1-mod2]   test2 1 mod2
+    PASSED
+    test_module.py:19: test_2[2-mod2]   test2 2 mod2
+    PASSED
     
     ========================= 8 passed in 0.02 seconds =========================
-      test0 1
-      test0 2
-    create mod1
-      test1 mod1
-      test2 1 mod1
-      test2 2 mod1
-    create mod2
-      test1 mod2
-      test2 1 mod2
-      test2 2 mod2
 
 You can see that the parametrized module-scoped ``modarg`` resource caused
 an ordering of test execution that lead to the fewest possible "active" resources. The finalizer for the ``mod1`` parametrized resource was executed 
@@ -728,61 +727,3 @@ fixtures functions starts at test classes, then test modules, then
 ``conftest.py`` files and finally builtin and third party plugins.
 
 
-.. _yieldctx:
-
-Fixture functions using "yield" / context manager integration
----------------------------------------------------------------
-
-.. versionadded:: 2.4
-
-pytest-2.4 allows fixture functions to use a ``yield`` instead 
-of a ``return`` statement to provide a fixture value.  Let's
-look at a quick example before discussing advantages::
-
-    # content of conftest.py
-    
-    import smtplib
-    import pytest
-
-    @pytest.fixture(scope="module", yieldctx=True)
-    def smtp():
-        smtp = smtplib.SMTP("merlinux.eu")
-        yield smtp  # provide the fixture value
-        print ("teardown smtp after a yield")
-        smtp.close()
-
-In contrast to the `finalization`_ example, our fixture
-function uses a single ``yield`` to provide the ``smtp`` fixture
-value.  The code after the ``yield`` statement serves as the
-teardown code, avoiding the indirection of registering a 
-teardown function.   More importantly, it also allows to 
-seemlessly re-use existing context managers, for example::
-
-    @pytest.fixture(yieldctx=True)
-    def somefixture():
-        with open("somefile") as f:
-            yield f.readlines()
-
-The file ``f`` will be closed once ``somefixture`` goes out of scope.
-It is possible to achieve the same result by using a ``request.addfinalizer``
-call but it is more boilerplate and not very obvious unless
-you know about the exact ``__enter__|__exit__`` protocol of with-style
-context managers.
-
-For some background, here is the protocol pytest follows for when
-``yieldctx=True`` is specified in the fixture decorator:
-
-a) iterate once into the generator for producing the value
-b) iterate a second time for tearing the fixture down, expecting
-   a StopIteration (which is produced automatically from the Python 
-   runtime when the generator returns).
-
-The teardown will always execute, independently of the outcome of
-test functions.  You do **not need** to write the teardown code into a
-``try-finally`` clause like you would usually do with
-:py:func:`contextlib.contextmanager` decorated functions.
-
-If the fixture generator yields a second value pytest will report 
-an error.  Yielding cannot be used for parametrization, rather
-see `fixture-parametrize`_.
-

doc/en/yieldfixture.txt

@@ -0,0 +1,132 @@
+
+.. _yieldctx:
+
+Fixture functions using "yield" / context manager integration
+---------------------------------------------------------------
+
+.. versionadded:: 2.4
+
+.. regendoc:wipe
+
+pytest-2.4 allows fixture functions to seemlessly use a ``yield`` instead 
+of a ``return`` statement to provide a fixture value while otherwise
+fully supporting all other fixture features.  
+
+.. note::
+
+   "yielding" fixture values is an experimental feature and its exact
+   declaration may change later but earliest in a 2.5 release.  You can thus
+   safely use this feature in the 2.4 series but may need to adapt your
+   fixtures later. Test functions themselves will not need to change
+   (they can be completely ignorant of the return/yield modes of 
+   fixture functions).
+
+Let's look at a simple standalone-example using the new ``yield`` syntax::
+
+    # content of test_yield.py
+    
+    import pytest
+
+    @pytest.fixture(yieldctx=True)
+    def passwd():
+        print ("\nsetup before yield")
+        f = open("/etc/passwd")
+        yield f.readlines()
+        print ("teardown after yield")
+        f.close()
+
+    def test_has_lines(passwd):
+        print ("test called")
+        assert passwd
+
+In contrast to :ref:`finalization through registering callbacks
+<finalization>`, our fixture function used a ``yield``
+statement to provide the lines of the ``/etc/passwd`` file.  
+The code after the ``yield`` statement serves as the teardown code, 
+avoiding the indirection of registering a teardown callback function.   
+
+Let's run it with output capturing disabled::
+
+    $ py.test -q -s test_yield.py
+    
+    setup before yield
+    test called
+    .teardown after yield
+    
+    1 passed in 0.01 seconds
+
+We can also seemlessly use the new syntax with ``with`` statements.
+Let's simplify the above ``passwd`` fixture::
+
+    # content of test_yield2.py
+    
+    import pytest
+
+    @pytest.fixture(yieldctx=True)
+    def passwd():
+        with open("/etc/passwd") as f:
+            yield f.readlines()
+
+    def test_has_lines(passwd):
+        assert len(passwd) >= 1
+
+The file ``f`` will be closed after the test finished execution
+because the Python ``file`` object supports finalization when
+the ``with`` statement ends. 
+
+Note that the new syntax is fully integrated with using ``scope``, 
+``params`` and other fixture features.  Changing existing 
+fixture functions to use ``yield`` is thus straight forward.
+
+Discussion and future considerations / feedback
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+The yield-syntax has been discussed by pytest users extensively.
+In general, the advantages of the using a ``yield`` fixture syntax are:
+
+- easy provision of fixtures in conjunction with context managers. 
+
+- no need to register a callback, providing for more synchronous
+  control flow in the fixture function.  Also there is no need to accept
+  the ``request`` object into the fixture function just for providing
+  finalization code.
+
+However, there are also limitations or foreseeable irritations:
+
+- usually ``yield`` is typically used for producing multiple values.
+  But fixture functions can only yield exactly one value.
+  Yielding a second fixture value will get you an error.
+  It's possible we can evolve pytest to allow for producing
+  multiple values as an alternative to current parametrization.
+  For now, you can just use the normal
+  :ref:`fixture parametrization <fixture-parametrize>`
+  mechanisms together with ``yield``-style fixtures.
+
+- the ``yield`` syntax is similar to what
+  :py:func:`contextlib.contextmanager` decorated functions
+  provide.  With pytest fixture functions, the "after yield" part will  
+  always be invoked, independently from the exception status
+  of the test function which uses the fixture.   The pytest
+  behaviour makes sense if you consider that many different
+  test functions might use a module or session scoped fixture.
+  Some test functions might raise exceptions and others not,
+  so how could pytest re-raise a single exception at the
+  ``yield`` point in the fixture function?
+
+- lastly ``yield`` introduces more than one way to write
+  fixture functions, so what's the obvious way to a newcomer?
+  Newcomers reading the docs will see feature examples using the
+  ``return`` style so should use that, if in doubt.  
+  Others can start experimenting with writing yield-style fixtures
+  and possibly help evolving them further.
+
+Some developers also expressed their preference for
+rather introduce a new ``@pytest.yieldfixture`` decorator
+instead of a keyword argument, or for assuming the above 
+yield-semantics automatically by introspecting if a fixture 
+function is a generator.  Depending on more experiences and 
+feedback during the 2.4 cycle, we revisit theses issues.
+
+If you want to feedback or participate in the ongoing
+discussion, please join our :ref:`contact channels`.
+you are most welcome.