761 lines
26 KiB
Plaintext
761 lines
26 KiB
Plaintext
|
|
.. _resources:
|
|
.. _`funcargs`:
|
|
.. _`funcarg mechanism`:
|
|
|
|
=======================================================
|
|
funcargs: resource injection and parametrization
|
|
=======================================================
|
|
|
|
.. note::
|
|
|
|
pytest-2.3 introduces major refinements to the original funcarg
|
|
mechanism introduced to pytest-2.0. While the old way
|
|
remains fully supported, it is recommended to use the refined
|
|
mechanisms. See also the `compatibility notes`_ and the detailed
|
|
:ref:`reasoning for the new funcarg and setup functions <funcargcompare>`.
|
|
|
|
.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection
|
|
|
|
Introduction
|
|
====================
|
|
|
|
py.test supports the injection of objects into test and setup functions
|
|
and flexibly control their life cycle in relation to the overall test
|
|
execution. Moreover, you can run a test function multiple times
|
|
injecting different objects.
|
|
|
|
The basic mechanism for injecting objects is called the *funcarg
|
|
mechanism* because objects are **injected** when a test or setup
|
|
function states it as an **argument**. The injected argument is
|
|
created by a call to a registered **funcarg factory**. This approach is
|
|
an example of `Dependency Injection`_ and helps to de-couple test code
|
|
from the setup of required objects: at test writing time you do not need
|
|
to care for the details of where and how your required resources are
|
|
constructed, if they are shared on a per-class, module or session basis,
|
|
or if your test function is invoked multiple times with differently
|
|
configured resource instances.
|
|
|
|
When a test function is invoked multiple times with different arguments we
|
|
speak of **parametrized testing**. This is useful if you want to test
|
|
e.g. against different database backends or want to write a parametrized
|
|
test function, checking that certain inputs lead to certain outputs.
|
|
You can parametrize funcarg factories, parametrize test function
|
|
arguments or even implement your own parametrization scheme through a
|
|
plugin hook.
|
|
|
|
pytest additionally offers powerful xunit-style :ref:`setup functions <setup
|
|
functions>` for the cases where you need to create implicit test state
|
|
that is not passed explicitely to test functions.
|
|
|
|
Concretely, there are three main means of funcarg management:
|
|
|
|
* a `@pytest.factory`_ marker to define resource factories,
|
|
their scoping and parametrization. Factories can themselves
|
|
receive resources through their function arguments, easing
|
|
the setup of `interdependent resources`_. Factories can use
|
|
the special `testcontext`_ object to access details from where
|
|
the factory or setup function is called and for registering finalizers.
|
|
|
|
* a `@pytest.mark.parametrize`_ marker for executing test functions
|
|
multiple times with different argument sets,
|
|
|
|
* a `pytest_generate_tests`_ plugin hook marker for implementing
|
|
your parametrization for a test function which may depend on
|
|
command line options, class/module attributes etc.
|
|
|
|
Apart from making it easy to manage your own test resources
|
|
pytest also comes with some :ref:`builtinresources` which
|
|
you can use without defining them yourself. Third-party plugins
|
|
offer yet more domain-specific funcarg resources (for example the
|
|
`pytest-django plugin <http://pypi.python.org/pypi/pytest-django>`_) so
|
|
that after plugin installation you can simply use them in
|
|
your test and setup functions. This all contributes to high
|
|
re-useability of test resource management and goes far beyond what can
|
|
be done with the classical xUnit style approach which encodes resource
|
|
setup statically into the test source code, leading to duplicate and
|
|
hard-to change fixtures.
|
|
|
|
.. _`@pytest.factory`:
|
|
|
|
``@pytest.factory``: Creating parametrized, scoped resources
|
|
=====================================================================
|
|
|
|
Basic funcarg injection example
|
|
-----------------------------------------------------------
|
|
|
|
Let's look at a simple self-contained test module using a factory
|
|
and a funcarg::
|
|
|
|
# content of ./test_simplefactory.py
|
|
import pytest
|
|
|
|
@pytest.factory()
|
|
def myfuncarg():
|
|
return 42
|
|
|
|
def test_function(myfuncarg):
|
|
assert myfuncarg == 17
|
|
|
|
Here, the ``test_function`` needs an object named ``myfuncarg`` and thus
|
|
py.test will discover and call the ``@pytest.factory`` marked ``myfuncarg``
|
|
factory function. Running the tests looks like this::
|
|
|
|
$ py.test test_simplefactory.py
|
|
=========================== test session starts ============================
|
|
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
|
|
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
|
|
collecting ... collected 1 items
|
|
|
|
test_simplefactory.py F
|
|
|
|
================================= FAILURES =================================
|
|
______________________________ test_function _______________________________
|
|
|
|
myfuncarg = 42
|
|
|
|
def test_function(myfuncarg):
|
|
> assert myfuncarg == 17
|
|
E assert 42 == 17
|
|
|
|
test_simplefactory.py:8: AssertionError
|
|
========================= 1 failed in 0.02 seconds =========================
|
|
|
|
This shows that the test function was called with a ``myfuncarg``
|
|
argument value of ``42`` and the assert fails as expected. Here is
|
|
how py.test comes to call the test function this way:
|
|
|
|
1. py.test :ref:`finds <test discovery>` the ``test_function`` because
|
|
of the ``test_`` prefix. The test function needs a function argument
|
|
named ``myfuncarg``. A matching factory function is discovered by
|
|
looking for a factory function named ``myfuncarg``.
|
|
|
|
2. ``myfuncarg()`` is called to create a value ``42``.
|
|
|
|
3. ``test_function(42)`` is now called and results in the above
|
|
reported exception because of the assertion mismatch.
|
|
|
|
Note that if you misspell a function argument or want
|
|
to use one that isn't available, you'll see an error
|
|
with a list of available function arguments.
|
|
|
|
.. Note::
|
|
|
|
You can always issue::
|
|
|
|
py.test --funcargs test_simplefactory.py
|
|
|
|
to see available function arguments.
|
|
|
|
|
|
Location independency of funcarg factories
|
|
----------------------------------------------------
|
|
|
|
If during implementing your tests you realize that you
|
|
want to use a factory from multiple test files you can move it
|
|
to a :ref:`conftest.py <conftest.py>` file or even separately installable
|
|
:ref:`plugins <plugins>` without changing test code. The discovery of
|
|
funcarg factories starts at test classes, then test modules, then
|
|
``conftest.py`` files and finally builtin and 3-rd party plugins.
|
|
|
|
Creating and using a session-shared funcarg
|
|
-----------------------------------------------------------------
|
|
|
|
.. regendoc:wipe
|
|
|
|
.. versionadded:: 2.3
|
|
|
|
The `@pytest.factory`_ marker allows to
|
|
|
|
* mark a function as a factory for resources, useable by test and setup
|
|
functions
|
|
|
|
* define parameters in order to run tests multiple times with
|
|
different resource instances
|
|
|
|
* declare a scope which determines the level of caching, i.e. how often
|
|
the factory will be called. Valid scopes are ``session``, ``module``,
|
|
``class`` and ``function``.
|
|
|
|
Here is a simple example of a factory creating a shared ``smtplib.SMTP``
|
|
connection resource which test functions then may use across the whole
|
|
test session::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
import smtplib
|
|
|
|
@pytest.factory(scope="session")
|
|
def smtp(testcontext):
|
|
return smtplib.SMTP("merlinux.eu")
|
|
|
|
The name of the factory is ``smtp`` (the factory function name)
|
|
and you can access its result by listing the name ``smtp`` as
|
|
an input parameter in any test or setup function::
|
|
|
|
# content of test_module.py
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
assert 0 # for demo purposes
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
assert 0 # for demo purposes
|
|
|
|
We deliberately insert failing ``assert 0`` statements in order to
|
|
inspect what is going on and can now run the tests::
|
|
|
|
$ py.test -q test_module.py
|
|
collecting ... collected 2 items
|
|
FF
|
|
================================= FAILURES =================================
|
|
________________________________ test_ehlo _________________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x25e7b48>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:5: AssertionError
|
|
________________________________ test_noop _________________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x25e7b48>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
2 failed in 0.20 seconds
|
|
|
|
you see the two ``assert 0`` failing and can also see that
|
|
the same (session-scoped) object was passed into the two test functions.
|
|
|
|
|
|
Parametrizing a session-shared funcarg resource
|
|
-----------------------------------------------------------------
|
|
|
|
Extending the previous example, we can flag the factory to create
|
|
two ``smtp`` values which will cause all tests using it to
|
|
run twice with two different values. The factory function gets
|
|
access to each parameter through the special `testcontext`_ object::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
import smtplib
|
|
|
|
@pytest.factory(scope="session",
|
|
params=["merlinux.eu", "mail.python.org"])
|
|
def smtp(testcontext):
|
|
return smtplib.SMTP(testcontext.param)
|
|
|
|
The main change is the definition of a ``params`` list in the
|
|
``factory``-marker and the ``testcontext.param`` access within the
|
|
factory function. No test function code needs to change.
|
|
So let's just do another run::
|
|
|
|
$ py.test -q
|
|
collecting ... collected 4 items
|
|
FFFF
|
|
================================= FAILURES =================================
|
|
__________________________ test_ehlo[merlinux.eu] __________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2960050>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
assert "merlinux" in response[1]
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:5: AssertionError
|
|
__________________________ test_noop[merlinux.eu] __________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x2960050>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
________________________ test_ehlo[mail.python.org] ________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x296a128>
|
|
|
|
def test_ehlo(smtp):
|
|
response = smtp.ehlo()
|
|
assert response[0] == 250
|
|
> assert "merlinux" in response[1]
|
|
E assert 'merlinux' in 'mail.python.org\nSIZE 10240000\nETRN\nSTARTTLS\nENHANCEDSTATUSCODES\n8BITMIME\nDSN'
|
|
|
|
test_module.py:4: AssertionError
|
|
________________________ test_noop[mail.python.org] ________________________
|
|
|
|
smtp = <smtplib.SMTP instance at 0x296a128>
|
|
|
|
def test_noop(smtp):
|
|
response = smtp.noop()
|
|
assert response[0] == 250
|
|
> assert 0 # for demo purposes
|
|
E assert 0
|
|
|
|
test_module.py:10: AssertionError
|
|
4 failed in 6.00 seconds
|
|
|
|
We get four failures because we are running the two tests twice with
|
|
different ``smtp`` instantiations as defined on the factory.
|
|
Note that with the ``mail.python.org`` connection the second test
|
|
fails in ``test_ehlo`` because it expects a specific server string.
|
|
|
|
Adding a finalizer to the parametrized resource
|
|
--------------------------------------------------------
|
|
|
|
Further extending the ``smtp`` example, we now want to properly
|
|
close a smtp server connection after the last test using it
|
|
has been run. We can do this by calling the ``testcontext.addfinalizer()``
|
|
helper::
|
|
|
|
# content of conftest.py
|
|
import pytest
|
|
import smtplib
|
|
|
|
@pytest.factory(scope="session",
|
|
params=["merlinux.eu", "mail.python.org"])
|
|
def smtp(testcontext):
|
|
smtp = smtplib.SMTP(testcontext.param)
|
|
def fin():
|
|
print ("finalizing %s" % smtp)
|
|
smtp.close()
|
|
testcontext.addfinalizer(fin)
|
|
return smtp
|
|
|
|
We also add a print call and then run py.test without the default
|
|
output capturing and disabled traceback reporting::
|
|
|
|
$ py.test -s -q --tb=no
|
|
collecting ... collected 4 items
|
|
FFFF
|
|
4 failed in 5.62 seconds
|
|
finalizing <smtplib.SMTP instance at 0x2d60368>
|
|
finalizing <smtplib.SMTP instance at 0x2d68e60>
|
|
|
|
We see that the two ``smtp`` instances are finalized appropriately.
|
|
|
|
Looking at test collection without running tests
|
|
------------------------------------------------------
|
|
|
|
You can also look at what tests pytest collects without running them::
|
|
|
|
$ py.test --collectonly
|
|
=========================== test session starts ============================
|
|
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8
|
|
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
|
|
collecting ... collected 4 items
|
|
<Module 'test_module.py'>
|
|
<Function 'test_ehlo[merlinux.eu]'>
|
|
<Function 'test_noop[merlinux.eu]'>
|
|
<Function 'test_ehlo[mail.python.org]'>
|
|
<Function 'test_noop[mail.python.org]'>
|
|
|
|
============================= in 0.02 seconds =============================
|
|
|
|
Note that pytest orders your test run by resource usage, minimizing
|
|
the number of active resources at any given time.
|
|
|
|
|
|
.. _`interdependent resources`:
|
|
|
|
Interdepdendent resources
|
|
----------------------------------------------------------
|
|
|
|
You can not only use funcargs in test functions but also in their factories
|
|
themselves. Extending the previous example we can instantiate another
|
|
object ``app`` and stick the ``smtp`` resource into it like this::
|
|
|
|
# content of test_appsetup.py
|
|
|
|
import pytest
|
|
|
|
class App:
|
|
def __init__(self, smtp):
|
|
self.smtp = smtp
|
|
|
|
@pytest.factory(scope="module")
|
|
def app(smtp):
|
|
return App(smtp)
|
|
|
|
def test_exists(app):
|
|
assert app.smtp
|
|
|
|
Here we define the factory local to the test module and make it access
|
|
the ``smtp`` resource by listing it as an input parameter. Let's run this::
|
|
|
|
$ py.test -v test_appsetup.py
|
|
=========================== test session starts ============================
|
|
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python
|
|
cachedir: /home/hpk/tmp/doc-exec-414/.cache
|
|
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
|
|
collecting ... collected 2 items
|
|
|
|
test_appsetup.py:12: test_exists[merlinux.eu] PASSED
|
|
test_appsetup.py:12: test_exists[mail.python.org] PASSED
|
|
|
|
========================= 2 passed in 5.37 seconds =========================
|
|
|
|
Due to the parametrization of ``smtp`` the test will run twice with two
|
|
different ``App`` instances and respective smtp servers. There is no
|
|
need for the ``app`` factory to be aware of the parametrization. Note
|
|
that the ``app`` factory has a scope of ``module`` whereas it uses the
|
|
session-scoped ``smtp`` object: it is fine for factories to use
|
|
"broader" scoped resources but not the other way round. A
|
|
session-scoped resource could not use a module-scoped resource in a
|
|
meaningful way.
|
|
|
|
.. _`automatic per-resource grouping`:
|
|
|
|
Grouping tests by resource parameters
|
|
----------------------------------------------------------
|
|
|
|
.. regendoc: wipe
|
|
|
|
pytest minimizes the number of active resources during test runs.
|
|
If you have a parametrized resource, then all the tests using one
|
|
resource instance will execute one after another. Then any finalizers
|
|
are called for that resource instance and then the next parametrized
|
|
resource instance is created and its tests are run. Among other things,
|
|
this eases testing of applications which create and use global state.
|
|
|
|
The following example uses two parametrized funcargs, one of which is
|
|
scoped on a per-module basis, and all the functions perform ``print`` call s
|
|
to show the flow of calls::
|
|
|
|
# content of test_module.py
|
|
import pytest
|
|
|
|
@pytest.factory(scope="module", params=["mod1", "mod2"])
|
|
def modarg(testcontext):
|
|
param = testcontext.param
|
|
print "create", param
|
|
def fin():
|
|
print "fin", param
|
|
testcontext.addfinalizer(fin)
|
|
return param
|
|
|
|
@pytest.factory(scope="function", params=[1,2])
|
|
def otherarg(testcontext):
|
|
return testcontext.param
|
|
|
|
def test_0(otherarg):
|
|
print " test0", otherarg
|
|
def test_1(modarg):
|
|
print " test1", modarg
|
|
def test_2(otherarg, modarg):
|
|
print " test2", otherarg, modarg
|
|
|
|
Let's run the tests in verbose mode and with looking at the print-output::
|
|
|
|
$ py.test -v -s test_module.py
|
|
=========================== test session starts ============================
|
|
platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev8 -- /home/hpk/venv/1/bin/python
|
|
cachedir: /home/hpk/tmp/doc-exec-414/.cache
|
|
plugins: xdist, bugzilla, cache, oejskit, cli, pep8, cov
|
|
collecting ... collected 8 items
|
|
|
|
test_module.py:16: test_0[1] PASSED
|
|
test_module.py:16: test_0[2] PASSED
|
|
test_module.py:18: test_1[mod1] PASSED
|
|
test_module.py:20: test_2[1-mod1] PASSED
|
|
test_module.py:20: test_2[2-mod1] PASSED
|
|
test_module.py:18: test_1[mod2] PASSED
|
|
test_module.py:20: test_2[1-mod2] PASSED
|
|
test_module.py:20: test_2[2-mod2] PASSED
|
|
|
|
========================= 8 passed in 0.02 seconds =========================
|
|
test0 1
|
|
test0 2
|
|
create mod1
|
|
test1 mod1
|
|
test2 1 mod1
|
|
test2 2 mod1
|
|
fin mod1
|
|
create mod2
|
|
test1 mod2
|
|
test2 1 mod2
|
|
test2 2 mod2
|
|
fin mod2
|
|
|
|
You can see that the parametrized module-scoped ``modarg`` resource caused
|
|
a re-ordering of test execution. The finalizer for the ``mod1`` parametrized
|
|
resource was executed before the ``mod2`` resource was setup.
|
|
|
|
.. currentmodule:: _pytest.python
|
|
.. _`testcontext`:
|
|
|
|
``testcontext``: interacting with test context
|
|
---------------------------------------------------
|
|
|
|
The ``testcontext`` object may be used by `@pytest.factory`_ or
|
|
:ref:`@pytest.setup <setup>` marked functions. It contains
|
|
information relating to the test context within which the marked
|
|
function executes. Moreover, you can call
|
|
``testcontext.addfinalizer(myfinalizer)`` in order to trigger a call to
|
|
``myfinalizer`` after the last test in the test context has executed.
|
|
If passed to a parametrized factory ``testcontext.param`` will contain a
|
|
parameter (one value out of the ``params`` list specified with the
|
|
`@pytest.factory`_ marker).
|
|
|
|
.. autoclass:: _pytest.python.TestContext()
|
|
:members:
|
|
|
|
|
|
|
|
.. _`test generators`:
|
|
.. _`parametrizing-tests`:
|
|
.. _`parametrized test functions`:
|
|
|
|
Parametrizing test functions
|
|
==========================================================================
|
|
|
|
While the `@pytest.factory`_ decorator allows to define parametrization
|
|
of funcarg resources at the factory-level, there are also means to
|
|
define parametrization at test functions directly:
|
|
|
|
* `@pytest.mark.parametrize`_ to provide multiple argument sets
|
|
for a particular test function or class.
|
|
|
|
* `pytest_generate_tests`_ to implement your own custom parametrization
|
|
scheme or extensions.
|
|
|
|
.. _`@pytest.mark.parametrize`:
|
|
|
|
``@pytest.mark.parametrize``: parametrizing test functions
|
|
---------------------------------------------------------------------
|
|
|
|
.. regendoc: wipe
|
|
|
|
.. versionadded:: 2.2
|
|
|
|
The builtin ``pytest.mark.parametrize`` decorator enables
|
|
parametrization of arguments for a test function. Here is a typical example
|
|
of a test function that wants check for expected output given a certain input::
|
|
|
|
# content of test_expectation.py
|
|
import pytest
|
|
@pytest.mark.parametrize(("input", "expected"), [
|
|
("3+5", 8),
|
|
("2+4", 6),
|
|
("6*9", 42),
|
|
])
|
|
def test_eval(input, expected):
|
|
assert eval(input) == expected
|
|
|
|
The ``@parametrize`` decorator defines three different argument sets for the
|
|
two ``(input, output)`` arguments of ``test_eval`` function so the latter
|
|
will be run three times::
|
|
|
|
$ py.test -q
|
|
|
|
collecting ... collected 13 items
|
|
....F........
|
|
================================= FAILURES =================================
|
|
____________________________ test_eval[6*9-42] _____________________________
|
|
|
|
input = '6*9', expected = 42
|
|
|
|
@pytest.mark.parametrize(("input", "expected"), [
|
|
("3+5", 8),
|
|
("2+4", 6),
|
|
("6*9", 42),
|
|
])
|
|
def test_eval(input, expected):
|
|
> assert eval(input) == expected
|
|
E assert 54 == 42
|
|
E + where 54 = eval('6*9')
|
|
|
|
test_expectation.py:8: AssertionError
|
|
1 failed, 12 passed in 5.76 seconds
|
|
|
|
As expected only one pair of input/output values fails the simple test function.
|
|
As usual you can see the ``input`` and ``output`` values in the traceback.
|
|
|
|
Note that there are various ways how you can mark groups of functions,
|
|
see :ref:`mark`.
|
|
|
|
|
|
.. _`pytest_generate_tests`:
|
|
|
|
Basic ``pytest_generate_tests`` example
|
|
---------------------------------------------
|
|
|
|
.. XXX
|
|
|
|
> line 598 "Basic ``pytest_generate_tests`` example" - I think this is
|
|
> not a very basic example! I think it is copied from parametrize.txt
|
|
> page, where it might make more sense. Here is what I would consider a
|
|
> basic example.
|
|
>
|
|
> # code
|
|
> def isSquare(n):
|
|
> n = n ** 0.5
|
|
> return int(n) == n
|
|
>
|
|
> # test file
|
|
> def pytest_generate_tests(metafunc):
|
|
> squares = [1, 4, 9, 16, 25, 36, 49]
|
|
> for n in range(1, 50):
|
|
> expected = n in squares
|
|
> if metafunc.function.__name__ == 'test_isSquare':
|
|
> metafunc.addcall(id=n, funcargs=dict(n=n,
|
|
> expected=expected))
|
|
>
|
|
>
|
|
> def test_isSquare(n, expected):
|
|
> assert isSquare(n) == expected
|
|
|
|
|
|
.. XXX
|
|
consider adding more examples, also mixed (factory-parametrized/test-function-parametrized, see mail from Brianna)
|
|
|
|
The ``pytest_generate_tests`` hook is typically used if you want
|
|
to go beyond what ``@pytest.mark.parametrize`` offers. For example,
|
|
let's say we want to execute a test with different computation
|
|
parameters and the parameter range shall be determined by a command
|
|
line argument. Let's first write a simple (do-nothing) computation test::
|
|
|
|
# content of test_compute.py
|
|
|
|
def test_compute(param1):
|
|
assert param1 < 4
|
|
|
|
Now we add a ``conftest.py`` file containing the addition of a
|
|
command line option and the generation of tests depending on
|
|
that option::
|
|
|
|
# content of conftest.py
|
|
|
|
def pytest_addoption(parser):
|
|
parser.addoption("--all", action="store_true",
|
|
help="run all combinations")
|
|
|
|
def pytest_generate_tests(metafunc):
|
|
if 'param1' in metafunc.funcargnames:
|
|
if metafunc.config.option.all:
|
|
end = 5
|
|
else:
|
|
end = 2
|
|
metafunc.parametrize("param1", range(end))
|
|
|
|
This means that we only run two tests if no option is passed::
|
|
|
|
$ py.test -q test_compute.py
|
|
collecting ... collected 2 items
|
|
..
|
|
2 passed in 0.02 seconds
|
|
|
|
And we run five tests if we add the ``--all`` option::
|
|
|
|
$ py.test -q --all
|
|
collecting ... collected 5 items
|
|
....F
|
|
================================= FAILURES =================================
|
|
_____________________________ test_compute[4] ______________________________
|
|
|
|
param1 = 4
|
|
|
|
def test_compute(param1):
|
|
> assert param1 < 4
|
|
E assert 4 < 4
|
|
|
|
test_compute.py:3: AssertionError
|
|
1 failed, 4 passed in 0.03 seconds
|
|
|
|
As expected when running the full range of ``param1`` values
|
|
we'll get an error on the last one.
|
|
|
|
You might want to look at :ref:`more parametrization examples <paramexamples>`.
|
|
|
|
|
|
.. _`metafunc object`:
|
|
|
|
The **metafunc** object
|
|
-------------------------------------------
|
|
|
|
metafunc objects are passed to the ``pytest_generate_tests`` hook.
|
|
They help to inspect a testfunction and to generate tests
|
|
according to test configuration or values specified
|
|
in the class or module where a test function is defined:
|
|
|
|
``metafunc.funcargnames``: set of required function arguments for given function
|
|
|
|
``metafunc.function``: underlying python test function
|
|
|
|
``metafunc.cls``: class object where the test function is defined in or None.
|
|
|
|
``metafunc.module``: the module object where the test function is defined in.
|
|
|
|
``metafunc.config``: access to command line opts and general config
|
|
|
|
.. automethod:: Metafunc.parametrize
|
|
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)
|
|
|
|
|
|
.. regendoc:wipe
|
|
|
|
|
|
.. _`compatibility notes`:
|
|
|
|
.. _`funcargscompat`:
|
|
|
|
Compatibility notes
|
|
============================================================
|
|
|
|
**Funcargs** were originally introduced to pytest-2.0. In pytest-2.3
|
|
the mechanism was extended and refined:
|
|
|
|
* previously funcarg factories were specified with a special
|
|
``pytest_funcarg__NAME`` prefix instead of using the
|
|
``@pytest.factory`` decorator.
|
|
|
|
* Factories received a `request`_ object which managed caching through
|
|
``request.cached_setup()`` calls and allowed using other funcargs via
|
|
``request.getfuncargvalue()`` calls. These intricate APIs made it hard
|
|
to do proper parametrization and implement resource caching. The
|
|
new ``@pytest.factory`` decorator allows to simply declare the scope
|
|
and let pytest figure things out for you.
|
|
|
|
* if you used parametrization and funcarg factories which made use of
|
|
``request.cached_setup()`` it is recommeneded to invest a few minutes
|
|
and simplify your funcarg factory code to use the `@pytest.factory`_
|
|
decorator instead. This will also allow to take advantage of
|
|
the `automatic per-resource grouping`_ of tests.
|
|
|
|
.. note::
|
|
|
|
Throughout the pytest documents the ``pytest_funcarg__NAME`` way of
|
|
defining a funcarg factory is often termed "old-style". Their
|
|
use remains fully supported and existing code using it should run
|
|
unmodified.
|
|
|
|
.. _request:
|
|
|
|
The request object passed to old-style factories
|
|
-----------------------------------------------------------------
|
|
|
|
Old-style funcarg factory definitions can receive a :py:class:`~_pytest.python.FuncargRequest` object which
|
|
provides methods to manage caching and finalization in the context of the
|
|
test invocation as well as several attributes of the the underlying test item.
|
|
|
|
|