p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

majorly refine funcargs docs and rename "resources.txt" to "funcargs.txt" so that existing web links will eventually land at this new page when pytest is released. Also integrated the detailed reasoning and update setup function docs 

to reflect latest discussions and feedback gathered on py-dev mailing list.
This commit is contained in:
holger krekel 2012-08-03 19:08:27 +02:00
parent 46dc7eeacb
commit 6746a00cb8
10 changed files with 687 additions and 735 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
_pytest/python.py
View File

@ -907,12 +907,15 @@ class Function(FunctionMixin, pytest.Item):
class FuncargRequest:
""" A request for function arguments from a test function.
""" (old-style) A request for function arguments from a test function.
Note that there is an optional ``param`` attribute in case
there was an invocation to metafunc.addcall(param=...).
If no such call was done in a ``pytest_generate_tests``
hook, the attribute will not be present.
hook, the attribute will not be present. Note that
as of pytest-2.3 you probably rather want to use the
testcontext object and mark your factory with a ``@pytest.factory``
marker.
"""
def __init__(self, pyfuncitem):

2
doc/en/apiref.txt
View File

@ -11,6 +11,8 @@ py.test reference documentation
customize.txt
assert.txt
funcargs.txt
funcarg_compare.txt
setup.txt
xunit_setup.txt
capture.txt
monkeypatch.txt

2
doc/en/conf.py
View File

@ -17,7 +17,7 @@
#
# The full version, including alpha/beta/rc tags.
# The short X.Y version.
version = release = "2.3.0.dev8"
version = release = "2.3.0.dev9"
import sys, os

1
doc/en/example/index.txt
View File

@ -21,4 +21,3 @@ need more examples or have questions. Also take a look at the :ref:`comprehensiv
markers.txt
pythoncollection.txt
nonpython.txt
newexamples.txt

113
doc/en/funcarg_compare.txt
View File

@ -1,55 +1,23 @@
V5: changes to new resource/setup facilities
.. _`funcargcompare`:
=============================================================
pytest-2.3: reasoning for the new funcarg and setup functions
=============================================================
**Target audience**: Reading this document requires basic knowledge of
python testing, xUnit setup methods and the basic pytest funcarg mechanism,
see http://pytest.org/latest/funcargs.html
**Changes**: This V5 draft is based on incorporating and thinking about
feedback on previous versions provided by Floris Bruynooghe, Carl Meyer,
Ronny Pfannschmidt and Samuele Pedroni. I have also now implemented it
which triggered a number of refinements as well. The main changes are:
* Collapse funcarg factory decorators into a single "@resource" one.
You can specify scopes and params with it. When using the decorator
the "pytest_funcarg__" prefix is not allowed and the old-style
``request`` object cannot be received.
* funcarg resource factories can now use funcargs themselves
* Drop setup/directory scope from this draft
* introduce a new @setup decorator similar to the @funcarg one
except that setup-markers cannot define parametriation themselves.
Instead they can easily depend on a parametrized funcarg (which
must not be visible at test function signatures).
* drop consideration of setup_X support for funcargs because
it is less flexible and probably causes more implementation
troubles than the current @setup approach which can share
a lot of logic with the @funcarg one.
* tests are grouped by parametrized funcargs and according to scope
(sounds like a small thing but is a big deal)
* make the new-style funcargs/setup use a "testcontext" object
which offers test context info and addfinalizer() methods but no
getfuncargvalue()/cached_setup()/applymarker anymore. Reason
being that getfuncargvalue()/cached_setup breaks other features
such as sorting by resource-scope and parametrization
python testing, xUnit setup methods and the (previous) basic pytest
funcarg mechanism, see http://pytest.org/2.2.4/funcargs.html
.. currentmodule:: _pytest
Shortcomings of the previous pytest_funcarg__ mechanism
---------------------------------------------------------
===========================================================
The previous funcarg mechanism calls a factory each time a
funcarg for a test function is testcontexted. If a factory wants
t re-use a resource across different scopes, it often used
the ``testcontext.cached_setup()`` helper to manage caching of
The pre pytest-2.3 funcarg mechanism calls a factory each time a
funcarg for a test function is required. If a factory wants to
re-use a resource across different scopes, it often used
the ``request.cached_setup()`` helper to manage caching of
resources. Here is a basic example how we could implement
a per-session Database object::
@ -95,10 +63,9 @@ new facilities.
Direct scoping of funcarg factories
--------------------------------------------------------
Instead of calling cached_setup(), you can decorate your factory
to state its scope::
Instead of calling cached_setup(), you can use the :ref:`@pytest.factory <@pytest.factory>` decorator and directly state the scope::
@pytest.mark.resource(scope="session")
@pytest.factory(scope="session")
def db(testcontext):
# factory will only be invoked once per session -
db = DataBase()
@ -108,25 +75,21 @@ to state its scope::
This factory implementation does not need to call ``cached_setup()`` anymore
because it will only be invoked once per session. Moreover, the
``testcontext.addfinalizer()`` registers a finalizer according to the specified
resource scope on which the factory function is operating. With this new
scoping, the still existing ``cached_setup()`` should be much less used
but will remain for compatibility reasons and for the case where you
still want to have your factory get called on a per-item basis.
resource scope on which the factory function is operating.
Direct parametrization of funcarg resource factories
----------------------------------------------------------
.. note:: Implemented
Previously, funcarg factories could not directly cause parametrization.
You needed to specify a ``@parametrize`` or implement a ``pytest_generate_tests`` hook to perform parametrization, i.e. calling a test multiple times
with different value sets. pytest-2.X introduces a decorator for use
on the factory itself::
You needed to specify a ``@parametrize`` decorator on your test function
or implement a ``pytest_generate_tests`` hook to perform
parametrization, i.e. calling a test multiple times with different value
sets. pytest-2.3 introduces a decorator for use on the factory itself::
@pytest.mark.resource(params=["mysql", "pg"])
@pytest.factory(params=["mysql", "pg"])
def pytest_funcarg__db(testcontext):
...
... # use testcontext.param
Here the factory will be invoked twice (with the respective "mysql"
and "pg" values set as ``testcontext.param`` attributes) and and all of
@ -141,7 +104,7 @@ functions/classes were parametrized via
Of course it's perfectly fine to combine parametrization and scoping::
@pytest.mark.resource(scope="session", params=["mysql", "pg"])
@pytest.factory(scope="session", params=["mysql", "pg"])
def pytest_funcarg__db(testcontext):
if testcontext.param == "mysql":
db = MySQL()
@ -155,26 +118,23 @@ receiving the values created by the two respective invocations to the
factory function.
No ``pytest_funcarg__`` prefix when using @resource decorator
No ``pytest_funcarg__`` prefix when using @factory decorator
-------------------------------------------------------------------
.. note:: Implemented
When using the ``@funcarg`` decorator the name of the function
does not need to (and in fact cannot) use the ``pytest_funcarg__``
naming::
denotes the name under which the resource can be accessed as a function
argument::
@pytest.mark.resource
@pytest.factory()
def db(testcontext):
...
The name under which the funcarg resource can be requested is ``db``.
You can also use the "old" non-decorator way of specifying funcarg factories
You can still use the "old" non-decorator way of specifying funcarg factories
aka::
def pytest_funcarg__db(testcontext):
def pytest_funcarg__db(request):
...
It is recommended to use the resource decorator, however.
@ -183,8 +143,6 @@ It is recommended to use the resource decorator, however.
solving per-session setup / the new @setup marker
--------------------------------------------------------------
.. note:: Implemented, at least working for basic situations.
pytest for a long time offered a pytest_configure and a pytest_sessionstart
hook which are often used to setup global resources. This suffers from
several problems:
@ -201,7 +159,7 @@ several problems:
fact that this hook is actually used for reporting, in particular
the test-header with platform/custom information.
Moreover, it is today not easy to define a scoped setup from plugins or
Moreover, it was not easy to define a scoped setup from plugins or
conftest files other than to implement a ``pytest_runtest_setup()`` hook
and caring for scoping/caching yourself. And it's virtually impossible
to do this with parametrization as ``pytest_runtest_setup()`` is called
@ -209,20 +167,17 @@ during test execution and parametrization happens at collection time.
It follows that pytest_configure/session/runtest_setup are often not
appropriate for implementing common fixture needs. Therefore,
pytest-2.X introduces a new :ref:`@pytest.setup` marker which takes
an optional "scope" parameter.
pytest-2.3 introduces a new :ref:`@pytest.setup <setup>` marker
for setup functions and it accepts an optional "scope" parameter.
See :ref:`new_setup` for examples.
See :ref:`setup` for more explanation and examples.
funcarg and setup discovery now happens at collection time
---------------------------------------------------------------------
.. note::
Partially implemented - collectonly shows no extra information however.
pytest-2.X takes care to discover funcarg factories and @setup methods
pytest-2.3 takes care to discover funcarg factories and @setup methods
at collection time. This is more efficient especially for large test suites.
Moreover, a call to "py.test --collectonly" should be able to show
a lot of setup-information and thus presents a nice method to get an
Moreover, a call to "py.test --collectonly" should be able to in the future
show a lot of setup-information and thus presents a nice method to get an
overview of resource management in your project.

771
doc/en/funcargs.txt
View File

@ -1,75 +1,110 @@
==============================================================
Injecting objects into test functions (funcargs)
==============================================================
.. currentmodule:: _pytest.python
.. _resources:
.. _`funcargs`:
.. _`funcarg mechanism`:
Dependency injection through function arguments
=================================================
=======================================================
funcargs: resource injection and parametrization
=======================================================
.. note::
This section describes the pytest mechanisms prior
to the pytest-2.3 release. If you haven't used these
features yet, it makes more sense to stop here and read
:ref:`resources` instead.
py.test lets you inject objects into test invocations and precisely
control their life cycle in relation to the overall test execution. Moreover,
you can run a test function multiple times injecting different objects.
The basic mechanism for injecting objects is also called the
*funcarg mechanism* because objects are ultimately injected
by calling a test function with it as an argument. Unlike the
classical xUnit approach *funcargs* relate more to `Dependency Injection`_
because they help to de-couple test code from objects required for
them to execute. At test writing time you do not need to care for the
details of how your required resources are constructed or if they
live through a function, class, module or session scope.
pytest-2.3 introduces major refinements to the original funcarg
mechanism introduced to pytest-2.0. While the old way
remains fully supported, it is recommended to use the refined
mechanisms. See also the `compatibility notes`_ and the detailed
:ref:`reasoning for the new funcarg and setup functions <funcargcompare>`.
.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection
To create a value with which to call a test function a factory function
is called which gets full access to the test function context and can
register finalizers or invoke lifecycle-caching helpers. The factory
can be implemented in same test class or test module, in a
per-directory ``conftest.py`` file or in an external plugin. This
allows total de-coupling of test and setup code.
Introduction
====================
A test function may be invoked multiple times in which case we
speak of :ref:`parametrized testing <parametrizing-tests>`. This can be
very useful if you want to test e.g. against different database backends
or with multiple numerical arguments sets and want to reuse the same set
of test functions.
py.test supports the injection of objects into test and setup functions
and flexibly control their life cycle in relation to the overall test
execution. Moreover, you can run a test function multiple times
injecting different objects.
py.test comes with some :ref:`builtinresources` and there are some refined usages in the examples section.
The basic mechanism for injecting objects is called the *funcarg
mechanism* because objects are **injected** when a test or setup
function states it as an **argument**. The injected argument is
created by a call to a registered **funcarg factory**. This approach is
an example of `Dependency Injection`_ and helps to de-couple test code
from the setup of required objects: at test writing time you do not need
to care for the details of where and how your required resources are
constructed, if they are shared on a per-class, module or session basis,
or if your test function is invoked multiple times with differently
configured resource instances.
.. _funcarg:
When a test function is invoked multiple times with different arguments we
speak of **parametrized testing**. This is useful if you want to test
e.g. against different database backends or want to write a parametrized
test function, checking that certain inputs lead to certain outputs.
You can parametrize funcarg factories, parametrize test function
arguments or even implement your own parametrization scheme through a
plugin hook.
Basic injection example
--------------------------------
pytest additionally offers powerful xunit-style :ref:`setup functions <setup
functions>` for the cases where you need to create implicit test state
that is not passed explicitely to test functions.
Let's look at a simple self-contained test module::
Concretely, there are three main means of funcarg management:
* a `@pytest.factory`_ marker to define resource factories,
their scoping and parametrization. Factories can themselves
receive resources through their function arguments, easing
the setup of `interdependent resources`_. Factories can use
the special `testcontext`_ object to access details from where
the factory or setup function is called and for registering finalizers.
* a `@pytest.mark.parametrize`_ marker for executing test functions
multiple times with different argument sets,
* a `pytest_generate_tests`_ plugin hook marker for implementing
your parametrization for a test function which may depend on
command line options, class/module attributes etc.
Apart from making it easy to manage your own test resources
pytest also comes with some :ref:`builtinresources` which
you can use without defining them yourself. Third-party plugins
offer yet more domain-specific funcarg resources (for example the
`pytest-django plugin <http://pypi.python.org/pypi/pytest-django>`_) so
that after plugin installation you can simply use them in
your test and setup functions. This all contributes to high
re-useability of test resource management and goes far beyond what can
be done with the classical xUnit style approach which encodes resource
setup statically into the test source code, leading to duplicate and
hard-to change fixtures.
.. _`@pytest.factory`:
``@pytest.factory``: Creating parametrized, scoped resources
=====================================================================
Basic funcarg injection example
-----------------------------------------------------------
Let's look at a simple self-contained test module using a factory
and a funcarg::
# content of ./test_simplefactory.py
def pytest_funcarg__myfuncarg(request):
import pytest
@pytest.factory()
def myfuncarg():
return 42
def test_function(myfuncarg):
assert myfuncarg == 17
This test function needs an injected object named ``myfuncarg``.
py.test will automatically discover and call the ``pytest_funcarg__myfuncarg``
factory. Running the test looks like this::
Here, the ``test_function`` needs an object named ``myfuncarg`` and thus
py.test will discover and call the ``@pytest.factory`` marked ``myfuncarg``
factory function. Running the tests looks like this::
$ py.test test_simplefactory.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2
plugins: xdist, bugzilla, cache, oejskit, pep8, cov
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 1 items
test_simplefactory.py F
@ -83,7 +118,7 @@ factory. Running the test looks like this::
> assert myfuncarg == 17
E assert 42 == 17
test_simplefactory.py:5: AssertionError
test_simplefactory.py:8: AssertionError
========================= 1 failed in 0.02 seconds =========================
This shows that the test function was called with a ``myfuncarg``
@ -93,14 +128,12 @@ how py.test comes to call the test function this way:
1. py.test :ref:`finds <test discovery>` the ``test_function`` because
of the ``test_`` prefix. The test function needs a function argument
named ``myfuncarg``. A matching factory function is discovered by
looking for the name ``pytest_funcarg__myfuncarg``.
looking for a factory function named ``myfuncarg``.
2. ``pytest_funcarg__myfuncarg(request)`` is called and
returns the value for ``myfuncarg``.
2. ``myfuncarg()`` is called to create a value ``42``.
3. the test function can now be called: ``test_function(42)``.
This results in the above exception because of the assertion
mismatch.
3. ``test_function(42)`` is now called and results in the above
reported exception because of the assertion mismatch.
Note that if you misspell a function argument or want
to use one that isn't available, you'll see an error
@ -114,130 +147,517 @@ with a list of available function arguments.
to see available function arguments.
The request object passed to factories
-----------------------------------------
Each funcarg factory receives a :py:class:`~_pytest.python.FuncargRequest` object which
provides methods to manage caching and finalization in the context of the
test invocation as well as several attributes of the the underlying test item. In fact, as of version pytest-2.3, the request API is implemented on all Item
objects and therefore the request object has general :py:class:`Node attributes and methods <_pytest.main.Node>` attributes. This is a backward compatible
change so no changes are neccessary for pre-2.3 funcarg factories.
Location independency of funcarg factories
----------------------------------------------------
If during implementing your tests you realize that you
want to use a factory from multiple test files you can move it
to a :ref:`conftest.py <conftest.py>` file or even separately installable
:ref:`plugins <plugins>` without changing test code. The discovery of
funcarg factories starts at test classes, then test modules, then
``conftest.py`` files and finally builtin and 3-rd party plugins.
Creating and using a session-shared funcarg
-----------------------------------------------------------------
.. regendoc:wipe
.. versionadded:: 2.3
The `@pytest.factory`_ marker allows to
* mark a function as a factory for resources, useable by test and setup
functions
* define parameters in order to run tests multiple times with
different resource instances
* declare a scope which determines the level of caching, i.e. how often
the factory will be called. Valid scopes are ``session``, ``module``,
``class`` and ``function``.
Here is a simple example of a factory creating a shared ``smtplib.SMTP``
connection resource which test functions then may use across the whole
test session::
# content of conftest.py
import pytest
import smtplib
@pytest.factory(scope="session")
def smtp(testcontext):
return smtplib.SMTP("merlinux.eu")
The name of the factory is ``smtp`` (the factory function name)
and you can access its result by listing the name ``smtp`` as
an input parameter in any test or setup function::
# content of test_module.py
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
assert 0 # for demo purposes
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
assert 0 # for demo purposes
We deliberately insert failing ``assert 0`` statements in order to
inspect what is going on and can now run the tests::
$ py.test -q test_module.py
collecting ... collected 2 items
FF
================================= FAILURES =================================
________________________________ test_ehlo _________________________________
smtp = <smtplib.SMTP instance at 0x25e7b48>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
> assert 0 # for demo purposes
E assert 0
test_module.py:5: AssertionError
________________________________ test_noop _________________________________
smtp = <smtplib.SMTP instance at 0x25e7b48>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
2 failed in 0.20 seconds
you see the two ``assert 0`` failing and can also see that
the same (session-scoped) object was passed into the two test functions.
.. _`parametrizing tests, generalized`: http://tetamap.wordpress.com/2009/05/13/parametrizing-python-tests-generalized/
Parametrizing a session-shared funcarg resource
-----------------------------------------------------------------
.. _`blog post about the monkeypatch funcarg`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/
.. _`xUnit style`: xunit_setup.html
Extending the previous example, we can flag the factory to create
two ``smtp`` values which will cause all tests using it to
run twice with two different values. The factory function gets
access to each parameter through the special `testcontext`_ object::
# content of conftest.py
import pytest
import smtplib
@pytest.factory(scope="session",
params=["merlinux.eu", "mail.python.org"])
def smtp(testcontext):
return smtplib.SMTP(testcontext.param)
The main change is the definition of a ``params`` list in the
``factory``-marker and the ``testcontext.param`` access within the
factory function. No test function code needs to change.
So let's just do another run::
$ py.test -q
collecting ... collected 4 items
FFFF
================================= FAILURES =================================
__________________________ test_ehlo[merlinux.eu] __________________________
smtp = <smtplib.SMTP instance at 0x2960050>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
> assert 0 # for demo purposes
E assert 0
test_module.py:5: AssertionError
__________________________ test_noop[merlinux.eu] __________________________
smtp = <smtplib.SMTP instance at 0x2960050>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
________________________ test_ehlo[mail.python.org] ________________________
smtp = <smtplib.SMTP instance at 0x296a128>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
> assert "merlinux" in response[1]
E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
test_module.py:4: AssertionError
________________________ test_noop[mail.python.org] ________________________
smtp = <smtplib.SMTP instance at 0x296a128>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
4 failed in 6.00 seconds
We get four failures because we are running the two tests twice with
different ``smtp`` instantiations as defined on the factory.
Note that with the ``mail.python.org`` connection the second test
fails in ``test_ehlo`` because it expects a specific server string.
Adding a finalizer to the parametrized resource
--------------------------------------------------------
Further extending the ``smtp`` example, we now want to properly
close a smtp server connection after the last test using it
has been run. We can do this by calling the ``testcontext.addfinalizer()``
helper::
# content of conftest.py
import pytest
import smtplib
@pytest.factory(scope="session",
params=["merlinux.eu", "mail.python.org"])
def smtp(testcontext):
smtp = smtplib.SMTP(testcontext.param)
def fin():
print ("finalizing %s" % smtp)
smtp.close()
testcontext.addfinalizer(fin)
return smtp
We also add a print call and then run py.test without the default
output capturing and disabled traceback reporting::
$ py.test -s -q --tb=no
collecting ... collected 4 items
FFFF
4 failed in 5.62 seconds
finalizing <smtplib.SMTP instance at 0x2d60368>
finalizing <smtplib.SMTP instance at 0x2d68e60>
We see that the two ``smtp`` instances are finalized appropriately.
Looking at test collection without running tests
------------------------------------------------------
You can also look at what tests pytest collects without running them::
$ py.test --collectonly
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 4 items
<Module 'test_module.py'>
<Function 'test_ehlo[merlinux.eu]'>
<Function 'test_noop[merlinux.eu]'>
<Function 'test_ehlo[mail.python.org]'>
<Function 'test_noop[mail.python.org]'>
============================= in 0.02 seconds =============================
Note that pytest orders your test run by resource usage, minimizing
the number of active resources at any given time.
.. _`interdependent resources`:
Interdepdendent resources
----------------------------------------------------------
You can not only use funcargs in test functions but also in their factories
themselves. Extending the previous example we can instantiate another
object ``app`` and stick the ``smtp`` resource into it like this::
# content of test_appsetup.py
import pytest
class App:
def __init__(self, smtp):
self.smtp = smtp
@pytest.factory(scope="module")
def app(smtp):
return App(smtp)
def test_exists(app):
assert app.smtp
Here we define the factory local to the test module and make it access
the ``smtp`` resource by listing it as an input parameter. Let's run this::
$ py.test -v test_appsetup.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python
cachedir: /home/hpk/tmp/doc-exec-414/.cache
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 2 items
test_appsetup.py:12: test_exists[merlinux.eu] PASSED
test_appsetup.py:12: test_exists[mail.python.org] PASSED
========================= 2 passed in 5.37 seconds =========================
Due to the parametrization of ``smtp`` the test will run twice with two
different ``App`` instances and respective smtp servers. There is no
need for the ``app`` factory to be aware of the parametrization. Note
that the ``app`` factory has a scope of ``module`` whereas it uses the
session-scoped ``smtp`` object: it is fine for factories to use
"broader" scoped resources but not the other way round. A
session-scoped resource could not use a module-scoped resource in a
meaningful way.
.. _`automatic per-resource grouping`:
Grouping tests by resource parameters
----------------------------------------------------------
.. regendoc: wipe
pytest minimizes the number of active resources during test runs.
If you have a parametrized resource, then all the tests using one
resource instance will execute one after another. Then any finalizers
are called for that resource instance and then the next parametrized
resource instance is created and its tests are run. Among other things,
this eases testing of applications which create and use global state.
The following example uses two parametrized funcargs, one of which is
scoped on a per-module basis, and all the functions perform ``print`` call
it to document the flow of calls::
# content of test_module.py
import pytest
@pytest.factory(scope="module", params=["mod1", "mod2"])
def modarg(testcontext):
param = testcontext.param
print "create", param
def fin():
print "fin", param
testcontext.addfinalizer(fin)
return param
@pytest.factory(scope="function", params=[1,2])
def otherarg(testcontext):
return testcontext.param
def test_0(otherarg):
print " test0", otherarg
def test_1(modarg):
print " test1", modarg
def test_2(otherarg, modarg):
print " test2", otherarg, modarg
Let's run the tests in verbose mode and with looking at the print-output::
$ py.test -v -s test_module.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python
cachedir: /home/hpk/tmp/doc-exec-414/.cache
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 8 items
test_module.py:16: test_0[1] PASSED
test_module.py:16: test_0[2] PASSED
test_module.py:18: test_1[mod1] PASSED
test_module.py:20: test_2[1-mod1] PASSED
test_module.py:20: test_2[2-mod1] PASSED
test_module.py:18: test_1[mod2] PASSED
test_module.py:20: test_2[1-mod2] PASSED
test_module.py:20: test_2[2-mod2] PASSED
========================= 8 passed in 0.02 seconds =========================
test0 1
test0 2
create mod1
test1 mod1
test2 1 mod1
test2 2 mod1
fin mod1
create mod2
test1 mod2
test2 1 mod2
test2 2 mod2
fin mod2
You can see that the parametrized module-scoped ``modarg`` resource caused
a re-ordering of test execution. The finalizer for the ``mod1`` parametrized
resource was executed before the ``mod2`` resource was setup.
.. currentmodule:: _pytest.python
.. _`testcontext`:
``testcontext``: interacting with test context
---------------------------------------------------
The ``testcontext`` object may be used by `@pytest.factory`_ or
:ref:`@pytest.setup <setup>` marked functions. It contains
information relating to the test context within which the function
executes. Moreover, you can call
``testcontext.addfinalizer(myfinalizer)`` in order to trigger a call to
``myfinalizer`` after the last test in the test context has executed.
If passed to a parametrized factory ``testcontext.param`` will contain a
parameter (one value out of the ``params`` list specified with the
`@pytest.factory`_ marker).
.. autoclass:: _pytest.python.TestContext()
:members:
.. _`funcarg factory`:
.. _factory:
.. _`test generators`:
.. _`parametrizing-tests`:
.. _`parametrized test functions`:
Parametrizing multiple calls to a test function
===========================================================
Parametrizing test functions
==========================================================================
You can parametrize multiple runs of the same test
function by adding new test function calls with different
function argument values. Let's look at a simple self-contained
example:
While the `@pytest.factory`_ decorator allows to define parametrization
of funcarg resources at the factory-level, there are also means to
define parametrization at test functions directly:
Basic generated test example
----------------------------
* `@pytest.mark.parametrize`_ to provide multiple argument sets
for a particular test function or class.
Let's consider a test module which uses the ``pytest_generate_tests``
hook to generate several calls to the same test function::
* `pytest_generate_tests`_ to implement your own custom parametrization
scheme or extensions.
.. _`@pytest.mark.parametrize`:
``@pytest.mark.parametrize``: parametrizing test functions
---------------------------------------------------------------------
.. regendoc: wipe
.. versionadded:: 2.2
The builtin ``pytest.mark.parametrize`` decorator enables
parametrization of arguments for a test function. Here is a typical example
of a test function that wants check for expected output given a certain input::
# content of test_expectation.py
import pytest
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
assert eval(input) == expected
The ``@parametrize`` decorator defines three different argument sets for the
two ``(input, output)`` arguments of ``test_eval`` function so the latter
will be run three times::
$ py.test -q
collecting ... collected 13 items
....F........
================================= FAILURES =================================
____________________________ test_eval[6*9-42] _____________________________
input = '6*9', expected = 42
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
> assert eval(input) == expected
E assert 54 == 42
E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError
1 failed, 12 passed in 5.76 seconds
As expected only one pair of input/output values fails the simple test function.
As usual you can see the ``input`` and ``output`` values in the traceback.
Note that there are various ways how you can mark groups of functions,
see :ref:`mark`.
.. _`pytest_generate_tests`:
Basic ``pytest_generate_tests`` example
---------------------------------------------
The ``pytest_generate_tests`` hook is typically used if you want
to go beyond what ``@pytest.mark.parametrize`` offers. For example,
let's say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument. Let's first write a simple (do-nothing) computation test::
# content of test_compute.py
def test_compute(param1):
assert param1 < 4
Now we add a ``conftest.py`` file containing the addition of a
command line option and the generation of tests depending on
that option::
# content of conftest.py
def pytest_addoption(parser):
parser.addoption("--all", action="store_true",
help="run all combinations")
# content of test_example.py
def pytest_generate_tests(metafunc):
if "numiter" in metafunc.funcargnames:
metafunc.parametrize("numiter", range(10))
if 'param1' in metafunc.funcargnames:
if metafunc.config.option.all:
end = 5
else:
end = 2
metafunc.parametrize("param1", range(end))
def test_func(numiter):
assert numiter < 9
This means that we only run two tests if no option is passed::
Running this will generate ten invocations of ``test_func`` passing in each of the items in the list of ``range(10)``::
$ py.test -q test_compute.py
collecting ... collected 2 items
..
2 passed in 0.02 seconds
$ py.test test_example.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2
plugins: xdist, bugzilla, cache, oejskit, pep8, cov
collecting ... collected 10 items
test_example.py .........F
And we run five tests if we add the ``--all`` option::
$ py.test -q --all
collecting ... collected 5 items
....F
================================= FAILURES =================================
_______________________________ test_func[9] _______________________________
_____________________________ test_compute[4] ______________________________
numiter = 9
param1 = 4
def test_func(numiter):
> assert numiter < 9
E assert 9 < 9
def test_compute(param1):
> assert param1 < 4
E assert 4 < 4
test_example.py:6: AssertionError
==================== 1 failed, 9 passed in 0.03 seconds ====================
test_compute.py:3: AssertionError
1 failed, 4 passed in 0.03 seconds
Obviously, only when ``numiter`` has the value of ``9`` does the test fail. Note that the ``pytest_generate_tests(metafunc)`` hook is called during
the test collection phase which is separate from the actual test running.
Let's just look at what is collected::
$ py.test --collectonly test_example.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2
plugins: xdist, bugzilla, cache, oejskit, pep8, cov
collecting ... collected 10 items
<Module 'test_example.py'>
<Function 'test_func[0]'>
<Function 'test_func[1]'>
<Function 'test_func[2]'>
<Function 'test_func[3]'>
<Function 'test_func[4]'>
<Function 'test_func[5]'>
<Function 'test_func[6]'>
<Function 'test_func[7]'>
<Function 'test_func[8]'>
<Function 'test_func[9]'>
============================= in 0.02 seconds =============================
If you want to select only the run with the value ``7`` you could do::
$ py.test -v -k 7 test_example.py # or -k test_func[7]
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev2 -- /home/hpk/venv/1/bin/python
cachedir: /home/hpk/tmp/doc-exec-271/.cache
plugins: xdist, bugzilla, cache, oejskit, pep8, cov
collecting ... collected 10 items
test_example.py:5: test_func[0] PASSED
test_example.py:5: test_func[1] PASSED
test_example.py:5: test_func[2] PASSED
test_example.py:5: test_func[3] PASSED
test_example.py:5: test_func[4] PASSED
test_example.py:5: test_func[5] PASSED
test_example.py:5: test_func[6] PASSED
test_example.py:5: test_func[7] PASSED
test_example.py:5: test_func[8] PASSED
test_example.py:5: test_func[9] FAILED
================================= FAILURES =================================
_______________________________ test_func[9] _______________________________
numiter = 9
def test_func(numiter):
> assert numiter < 9
E assert 9 < 9
test_example.py:6: AssertionError
==================== 1 failed, 9 passed in 0.03 seconds ====================
As expected when running the full range of ``param1`` values
we'll get an error on the last one.
You might want to look at :ref:`more parametrization examples <paramexamples>`.
.. _`metafunc object`:
The **metafunc** object
@ -260,3 +680,52 @@ in the class or module where a test function is defined:
.. automethod:: Metafunc.parametrize
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)
.. regendoc:wipe
.. _`compatibility notes`:
.. _`funcargscompat`:
Compatibility notes
============================================================
**Funcargs** were originally introduced to pytest-2.0. In pytest-2.3
the mechanism were extended and refined. Here are some related notes:
* previously funcarg factories were specified with a special
``pytest_funcarg__NAME`` prefix instead of using the
``@pytest.factory`` decorator.
* Factories received a `request`_ object which managed caching through
``request.cached_setup()`` calls and allowed using other funcargs via
``request.getfuncargvalue()`` calls. These intricate APIs made it hard
to do proper parametrization and implement resource caching. The
new ``@pytest.factory`` decorator allows to simply declare the scope
and let pytest figure things out for you.
* if you used parametrization and funcarg factories which made use of
``request.cached_setup()`` it is recommeneded to invest a few minutes
and simplify your funcarg factory code to use the `@pytest.factory`_
decorator instead. This will also allow to take advantage of
the `automatic per-resource grouping`_ of tests.
.. note::
Throughout the pytest documents the ``pytest_funcarg__NAME`` way of
defining a funcarg factory is often termed "old-style". Their
use remains fully supported and existing code using it should run
unmodified.
.. _request:
The request object passed to old-style factories
-----------------------------------------------------------------
Old-style funcarg factory definitions can receive a :py:class:`~_pytest.python.FuncargRequest` object which
provides methods to manage caching and finalization in the context of the
test invocation as well as several attributes of the the underlying test item.

2
doc/en/index.txt
View File

@ -27,7 +27,7 @@ Welcome to pytest!
- (new in 2.3) :ref:`easy test resource management, scoping and
parametrization <resources>`
- (new in 2.3) :ref:`xunitsetup`.
- (new in 2.3) :ref:`setup functions`.
- (new in 2.2) :ref:`durations`
- (much improved in 2.2) :ref:`marking and test selection <mark>`
- (improved in 2.2) :ref:`parametrized test functions <parametrized test functions>`

487
doc/en/resources.txt
View File

@ -1,487 +0,0 @@
.. _resources:
=======================================================
test resource injection and parametrization
=======================================================
.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection
.. versionadded: 2.3
pytest offers very flexible means for managing test resources and
test parametrization.
The pytest resource management mechanism is an example of `Dependency
Injection`_ because test and :ref:`setup functions <xunitsetup>` receive
resources simply by stating them as an input argument. Therefore and
also for historic reasons, they are often called **funcargs**. At test
writing time you typically do not need to care for the details of how
your required resources are constructed, if they live through a
function, class, module or session scope or if the test will be called
multiple times with different resource instances.
To create a value with which to call a test function a resource factory
function is called which gets full access to the test context and can
register finalizers which are to be run after the last test in that context
finished. Resource factories can be implemented in same test class or
test module, in a per-directory ``conftest.py`` file or in an external
plugin. This allows **total de-coupling of test and setup code**,
lowering the cost of refactoring.
A test function may be invoked multiple times in which case we
speak of parametrization. You can parametrize resources or parametrize
test function arguments directly or even implement your own parametrization
scheme through a plugin hook.
A resource has a **name** under which test and setup functions
can access it by listing it as an input argument. Due to this and
also for historic reasons, resources are often called **funcargs**.
A resource is created by a factory which can be flagged with a **scope**
to only create resources on a per-class/per-module/per-session basis
instead of the default per-function scope.
Concretely, there are three means of resource and parametrization management:
* a `@pytest.factory`_ marker to define resource factories,
their scoping and parametrization. Factories can themselves
receive resources through their function arguments, easing
the setup of interdependent resources. They can also use
the special `testcontext`_ object to access details n which
the factory/setup is called and for registering finalizers.
* a `@pytest.mark.parametrize`_ marker for executing test functions
multiple times with different parameter sets
* a `pytest_generate_tests`_ plugin hook marker for implementing
your parametrization for a test function which may depend on
command line options, class/module attributes etc.
Finally, pytest comes with some :ref:`builtinresources` which
you can use without defining them yourself. Moreover, third-party
plugins offer their own resources so that after installation
you can simply use them in your test and setup functions.
.. _`@pytest.factory`:
``@pytest.factory``: Creating parametrized, scoped resources
-----------------------------------------------------------------
.. regendoc:wipe
.. versionadded:: 2.3
The `@pytest.factory`_ marker allows to
* mark a function as a factory for resources, useable by test and setup functions
* define parameters in order to run tests multiple times with different
resource instances
* set a scope which determines the level of caching, i.e. how often
the factory will be called. Valid scopes are ``session``, ``module``,
``class`` and ``function``.
Here is a simple example of a factory creating a shared ``smtplib.SMTP``
connection resource which test functions then may use across the whole
test session::
# content of conftest.py
import pytest
import smtplib
@pytest.factory(scope="session")
def smtp(testcontext):
smtp = smtplib.SMTP("merlinux.eu")
testcontext.addfinalizer(smtp.close)
return smtp
The name of the resource is ``smtp`` (the factory function name)
and you can now access the ``smtp`` resource by listing it as
an input parameter in any test function below the directory where
``conftest.py`` is located::
# content of test_module.py
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
assert 0 # for demo purposes
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
assert 0 # for demo purposes
If you run the tests::
$ py.test -q
collecting ... collected 2 items
FF
================================= FAILURES =================================
________________________________ test_ehlo _________________________________
smtp = <smtplib.SMTP instance at 0x1c7c638>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
> assert 0 # for demo purposes
E assert 0
test_module.py:5: AssertionError
________________________________ test_noop _________________________________
smtp = <smtplib.SMTP instance at 0x1c7c638>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
2 failed in 0.18 seconds
you will see the two ``assert 0`` failing and can see that
the same (session-scoped) object was passed into the two test functions.
If you now want to test multiple servers you can simply parametrize
the ``smtp`` factory::
# content of conftest.py
import pytest
import smtplib
@pytest.factory(scope="session",
params=["merlinux.eu", "mail.python.org"])
def smtp(testcontext):
smtp = smtplib.SMTP(testcontext.param)
def fin():
smtp.close()
testcontext.addfinalizer(fin)
return smtp
The main change is the definition of a ``params`` list in the
``factory``-marker and the ``testcontext.param`` access within the
factory function. No test code needs to change. So let's just do another
run::
$ py.test -q
collecting ... collected 4 items
FFFF
================================= FAILURES =================================
__________________________ test_ehlo[merlinux.eu] __________________________
smtp = <smtplib.SMTP instance at 0x1d162d8>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
assert "merlinux" in response[1]
> assert 0 # for demo purposes
E assert 0
test_module.py:5: AssertionError
__________________________ test_noop[merlinux.eu] __________________________
smtp = <smtplib.SMTP instance at 0x1d162d8>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
________________________ test_ehlo[mail.python.org] ________________________
smtp = <smtplib.SMTP instance at 0x1d1f098>
def test_ehlo(smtp):
response = smtp.ehlo()
assert response[0] == 250
> assert "merlinux" in response[1]
E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
test_module.py:4: AssertionError
________________________ test_noop[mail.python.org] ________________________
smtp = <smtplib.SMTP instance at 0x1d1f098>
def test_noop(smtp):
response = smtp.noop()
assert response[0] == 250
> assert 0 # for demo purposes
E assert 0
test_module.py:10: AssertionError
4 failed in 6.42 seconds
We get four failures because we are running the two tests twice with
different ``smtp`` instantiations as defined on the factory.
Note that with the ``mail.python.org`` connection the second test
fails in ``test_ehlo`` because it expects a specific server string.
You can also look at what tests pytest collects without running them::
$ py.test --collectonly
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 4 items
<Module 'test_module.py'>
<Function 'test_ehlo[merlinux.eu]'>
<Function 'test_noop[merlinux.eu]'>
<Function 'test_ehlo[mail.python.org]'>
<Function 'test_noop[mail.python.org]'>
============================= in 0.02 seconds =============================
Note that pytest orders your test run by resource usage, minimizing
the number of active resources at any given time.
Interdepdendent resources
----------------------------------------------------------
You can not only use resources in test functions but also in resource factories
themselves. Extending the previous example we can instantiate an application
object by sticking the ``smtp`` resource into it::
# content of test_appsetup.py
import pytest
class App:
def __init__(self, smtp):
self.smtp = smtp
@pytest.factory(scope="module")
def app(smtp):
return App(smtp)
def test_exists(app):
assert app.smtp
Let's run this::
$ py.test -v test_appsetup.py
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python
cachedir: /home/hpk/tmp/doc-exec-398/.cache
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 2 items
test_appsetup.py:12: test_exists[merlinux.eu] PASSED
test_appsetup.py:12: test_exists[mail.python.org] PASSED
========================= 2 passed in 5.96 seconds =========================
Due to the parametrization of ``smtp`` the test will
run twice with two different ``App`` instances and respective smtp servers.
There is no need for the ``app`` factory to be aware of the parametrization.
Grouping tests by resource parameters
----------------------------------------------------------
.. regendoc: wipe
pytest minimizes the number of active resources during test runs.
If you have a parametrized resource, then all the tests using one
resource instance will execute one after another. Then any finalizers
are called for that resource instance and then the next parametrized
resource instance is created and its tests are run. Among other things,
this eases testing of applications which create and use global state.
The following example uses two parametrized funcargs, one of which is
scoped on a per-module basis::
# content of test_module.py
import pytest
@pytest.factory(scope="module", params=["mod1", "mod2"])
def modarg(testcontext):
param = testcontext.param
print "create", param
def fin():
print "fin", param
testcontext.addfinalizer(fin)
return param
@pytest.factory(scope="function", params=[1,2])
def otherarg(testcontext):
return testcontext.param
def test_0(otherarg):
print " test0", otherarg
def test_1(modarg):
print " test1", modarg
def test_2(otherarg, modarg):
print " test2", otherarg, modarg
Let's run the tests in verbose mode and with looking at the print-output::
$ py.test -v -s
=========================== test session starts ============================
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev7 -- /home/hpk/venv/1/bin/python
cachedir: /home/hpk/tmp/doc-exec-398/.cache
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
collecting ... collected 8 items
test_module.py:16: test_0[1] PASSED
test_module.py:16: test_0[2] PASSED
test_module.py:18: test_1[mod1] PASSED
test_module.py:20: test_2[1-mod1] PASSED
test_module.py:20: test_2[2-mod1] PASSED
test_module.py:18: test_1[mod2] PASSED
test_module.py:20: test_2[1-mod2] PASSED
test_module.py:20: test_2[2-mod2] PASSED
========================= 8 passed in 0.03 seconds =========================
test0 1
test0 2
create mod1
test1 mod1
test2 1 mod1
test2 2 mod1
fin mod1
create mod2
test1 mod2
test2 1 mod2
test2 2 mod2
fin mod2
You can see that the parametrized module-scoped ``modarg`` resource caused
a re-ordering of test execution. The finalizer for the ``mod1`` parametrized
resource was executed before the ``mod2`` resource was setup.
.. currentmodule:: _pytest.python
.. _`testcontext`:
``testcontext``: interacting with test context
---------------------------------------------------
The ``testcontext`` object may be received by `@pytest.factory`_ or
`@pytest.setup`_ marked functions. It contains information relating
to the test context within which the function executes. Moreover, you
can call ``testcontext.addfinalizer(myfinalizer)`` in order to trigger
a call to ``myfinalizer`` after the last test in the test context has executed.
If passed to a parametrized factory ``testcontext.param`` will contain
a parameter (one value out of the ``params`` list specified with the
`@pytest.factory`_ marker).
.. autoclass:: _pytest.python.TestContext()
:members:
.. _`@pytest.mark.parametrize`:
``@pytest.mark.parametrize``: directly parametrizing test functions
----------------------------------------------------------------------------
.. versionadded:: 2.2
The builtin ``pytest.mark.parametrize`` decorator enables
parametrization of arguments for a test function. Here is an example
of a test function that wants check for expected output given a certain input::
# content of test_expectation.py
import pytest
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
assert eval(input) == expected
we parametrize two arguments of the test function so that the test
function is called three times. Let's run it::
$ py.test -q
collecting ... collected 11 items
..F........
================================= FAILURES =================================
____________________________ test_eval[6*9-42] _____________________________
input = '6*9', expected = 42
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
> assert eval(input) == expected
E assert 54 == 42
E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError
1 failed, 10 passed in 0.04 seconds
As expected only one pair of input/output values fails the simple test function.
Note that there are various ways how you can mark groups of functions,
see :ref:`mark`.
.. _`pytest_generate_tests`:
``pytest_generate_test``: implementing your own parametrization scheme
----------------------------------------------------------------------------
.. regendoc:wipe
Let's say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument. Let's first write a simple (do-nothing) computation test::
# content of test_compute.py
def test_compute(param1):
assert param1 < 4
Now we add a test configuration like this::
# content of conftest.py
def pytest_addoption(parser):
parser.addoption("--all", action="store_true",
help="run all combinations")
def pytest_generate_tests(metafunc):
if 'param1' in metafunc.funcargnames:
if metafunc.config.option.all:
end = 5
else:
end = 2
metafunc.parametrize("param1", range(end))
This means that we only run 2 tests if we do not pass ``--all``::
$ py.test -q test_compute.py
collecting ... collected 2 items
..
2 passed in 0.03 seconds
We run only two computations, so we see two dots.
let's run the full monty::
$ py.test -q --all
collecting ... collected 5 items
....F
================================= FAILURES =================================
_____________________________ test_compute[4] ______________________________
param1 = 4
def test_compute(param1):
> assert param1 < 4
E assert 4 < 4
test_compute.py:3: AssertionError
1 failed, 4 passed in 0.03 seconds
As expected when running the full range of ``param1`` values
we'll get an error on the last one.

26
doc/en/setup.txt
View File

@ -1,5 +1,7 @@
.. _xunitsetup:
.. _setup:
.. _`setup functions`:
.. _`@pytest.setup`:
``@setup`` functions or: xunit on steroids
========================================================
@ -18,19 +20,20 @@ it has finished. Unlike :ref:`injected resources <resources>` setup
functions work indirectly by causing global side effects or
setting test case attributes which test methods can then access.
pytest originally introduced in 2005 a fine-grained model of detecting
pytest originally introduced in 2005 a scope-specific model of detecting
setup and teardown functions on a per-module, class or function basis.
The Python unittest module and nose have subsequently incorporated them.
This model remains supported as :ref:`old-style xunit`.
The Python unittest package and nose have subsequently incorporated them.
This model remains supported by pytest as :ref:`old-style xunit`.
With pytest-2.3 a new ``pytest.setup()`` decorator is introduced
to mark functions as setup functions which:
Moreover, pytest-2.3 introduces a new ``pytest.setup()`` decorator
to mark functions as setup functions which allow to implement everything
you can do with the old-style and much more. Specifically setup functions:
- can receive resources through funcargs,
- can receive :ref:`resources through funcargs <resources>`,
- fully interoperate with parametrized resources,
- can be defined in a plugin or conftest.py file and get called
- can be defined in a plugin or :ref:`conftest.py` file and get called
on a per-session, per-module, per-class or per-function basis,
- can access the full :ref:`testcontext` for which the setup is called,
- can access the :ref:`testcontext <testcontext>` for which the setup is called,
- can precisely control teardown by registering one or multiple
teardown functions as soon as they have performed some actions
which need undoing, eliminating the no need for a separate
@ -75,7 +78,7 @@ Our ``mydir`` setup function is executed on a per-function basis,
the default scope used by the ``pytest.setup`` decorator.
It accesses the ``tmpdir`` resource which provides a new empty
directory path object. The ``test_function2`` here checks that
it executes with a fresh directory and specifically
it executes with a fresh directory and that it
does not see the previously created ``anotherfile``. We can
thus expect two passing tests::
@ -115,6 +118,11 @@ empty directory ahead of running a test.
test modules accessing a global resource
-------------------------------------------------------
.. note::
Relying on `global state is considered bad programming practise <http://en.wikipedia.org/wiki/Global_variable>`_ but when you work with an application
that relies on it you often have no choice.
If you want test modules to access a global resource,
you can stick the resource to the module globals in
a per-module setup function. We use a :ref:`resource factory

11
doc/en/xunit_old.txt → doc/en/xunit_setup.txt
View File

@ -4,10 +4,13 @@
Old-style xunit-style setup
========================================
This section describes the old way how you can implement setup and
teardown on a per-module/class/function basis. It remains fully
supported but it is recommended to rather use :ref:`@setup functions <setup>`
or :ref:`injected resources <resources>` for implementing your setup needs.
.. note::
This section describes the old way how you can implement setup and
teardown on a per-module/class/function basis. It remains fully
supported but it is recommended to rather use :ref:`@setup functions
<setup>` or :ref:`injected resources <resources>` for implementing your
setup needs.
Module level setup/teardown
--------------------------------------