2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
.. _`test generators`:
|
|
|
|
.. _`parametrizing-tests`:
|
|
|
|
.. _`parametrized test functions`:
|
|
|
|
.. _`parametrize`:
|
|
|
|
|
|
|
|
Parametrizing fixtures and test functions
|
|
|
|
==========================================================================
|
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
pytest supports test parametrization in several well-integrated ways:
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
- :ref:`@pytest.fixture` allows to define :ref:`parametrization
|
|
|
|
at the level of fixture functions <fixture-parametrize>`.
|
|
|
|
|
|
|
|
* `@pytest.mark.parametrize`_ allows to define parametrization at the
|
|
|
|
function or class level, provides multiple argument/fixture sets
|
2012-10-07 19:06:17 +08:00
|
|
|
for a particular test function or class.
|
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
* `pytest_generate_tests`_ enables implementing your own custom
|
|
|
|
dynamic parametrization scheme or extensions.
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
.. _`@pytest.mark.parametrize`:
|
|
|
|
|
|
|
|
``@pytest.mark.parametrize``: parametrizing test functions
|
|
|
|
---------------------------------------------------------------------
|
|
|
|
|
|
|
|
.. regendoc: wipe
|
|
|
|
|
|
|
|
.. versionadded:: 2.2
|
|
|
|
|
|
|
|
The builtin ``pytest.mark.parametrize`` decorator enables
|
|
|
|
parametrization of arguments for a test function. Here is a typical example
|
2012-10-08 19:19:31 +08:00
|
|
|
of a test function that implements checking that a certain input leads
|
|
|
|
to an expected output::
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
# content of test_expectation.py
|
|
|
|
import pytest
|
|
|
|
@pytest.mark.parametrize(("input", "expected"), [
|
|
|
|
("3+5", 8),
|
|
|
|
("2+4", 6),
|
|
|
|
("6*9", 42),
|
|
|
|
])
|
|
|
|
def test_eval(input, expected):
|
|
|
|
assert eval(input) == expected
|
|
|
|
|
|
|
|
The ``@parametrize`` decorator defines three different argument sets for the
|
|
|
|
two ``(input, output)`` arguments of ``test_eval`` function so the latter
|
|
|
|
will be run three times::
|
|
|
|
|
|
|
|
$ py.test -q
|
|
|
|
..F
|
|
|
|
================================= FAILURES =================================
|
|
|
|
____________________________ test_eval[6*9-42] _____________________________
|
|
|
|
|
|
|
|
input = '6*9', expected = 42
|
|
|
|
|
|
|
|
@pytest.mark.parametrize(("input", "expected"), [
|
|
|
|
("3+5", 8),
|
|
|
|
("2+4", 6),
|
|
|
|
("6*9", 42),
|
|
|
|
])
|
|
|
|
def test_eval(input, expected):
|
|
|
|
> assert eval(input) == expected
|
|
|
|
E assert 54 == 42
|
|
|
|
E + where 54 = eval('6*9')
|
|
|
|
|
|
|
|
test_expectation.py:8: AssertionError
|
|
|
|
|
|
|
|
As expected only one pair of input/output values fails the simple test function.
|
|
|
|
As usual you can see the ``input`` and ``output`` values in the traceback.
|
|
|
|
|
|
|
|
Note that there are various ways how you can mark groups of functions,
|
|
|
|
see :ref:`mark`.
|
|
|
|
|
|
|
|
|
|
|
|
.. _`pytest_generate_tests`:
|
|
|
|
|
|
|
|
Basic ``pytest_generate_tests`` example
|
|
|
|
---------------------------------------------
|
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
Sometimes you may want to implement your own parametrization scheme
|
|
|
|
or implement some dynamism for determining the parameters or scope
|
|
|
|
of a fixture. For this, you can use the ``pytest_generate_tests`` hook
|
|
|
|
which is called for every test function. Through the special `metafunc`
|
|
|
|
object you can inspect the requesting test context and, most importantly,
|
|
|
|
you can call ``metafunc.parametrize()`` to pass in parametrizatin.
|
|
|
|
For example, let's say we want to execute a test that takes some string
|
|
|
|
input and we want to pass that in with a command line option
|
|
|
|
``--stringinput=value``. Let's first write a simple test accepting
|
|
|
|
a ``stringinput`` fixture function argument::
|
|
|
|
|
|
|
|
# content of test_strings.py
|
|
|
|
|
|
|
|
def test_valid_string(stringinput):
|
|
|
|
assert stringinput.isalpha()
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
Now we add a ``conftest.py`` file containing the addition of a
|
2012-10-08 19:19:31 +08:00
|
|
|
command line option and the parametrization of our test function::
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
# content of conftest.py
|
|
|
|
|
|
|
|
def pytest_addoption(parser):
|
2012-10-08 19:19:31 +08:00
|
|
|
parser.addoption("--stringinput", action="append",
|
|
|
|
help="list of stringinputs to pass to test functions")
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
def pytest_generate_tests(metafunc):
|
2012-10-08 19:19:31 +08:00
|
|
|
if 'stringinput' in metafunc.fixturenames:
|
|
|
|
metafunc.parametrize("stringinput",
|
|
|
|
metafunc.config.option.stringinput)
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
If we now pass two stringinput values, our test will run twice::
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
$ py.test -q --stringinput="hello" --stringinput="world" test_strings.py
|
2012-10-07 19:06:17 +08:00
|
|
|
..
|
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
Let's run with a stringinput that will lead to an error::
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
$ py.test -q --stringinput="!" test_strings.py
|
|
|
|
F
|
2012-10-07 19:06:17 +08:00
|
|
|
================================= FAILURES =================================
|
2012-10-08 19:19:31 +08:00
|
|
|
___________________________ test_valid_string[!] ___________________________
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
stringinput = '!'
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
def test_valid_string(stringinput):
|
|
|
|
> assert stringinput.isalpha()
|
|
|
|
E assert <built-in method isalpha of str object at 0x7fd0b71f2fd0>()
|
|
|
|
E + where <built-in method isalpha of str object at 0x7fd0b71f2fd0> = '!'.isalpha
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
test_strings.py:3: AssertionError
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
As expected our test function will error out.
|
2012-10-07 19:06:17 +08:00
|
|
|
|
2012-10-08 19:19:31 +08:00
|
|
|
For further examples, you might want to look at :ref:`more
|
|
|
|
parametrization examples <paramexamples>`.
|
2012-10-07 19:06:17 +08:00
|
|
|
|
|
|
|
.. _`metafunc object`:
|
|
|
|
|
|
|
|
The **metafunc** object
|
|
|
|
-------------------------------------------
|
|
|
|
|
|
|
|
.. currentmodule:: _pytest.python
|
|
|
|
|
|
|
|
metafunc objects are passed to the ``pytest_generate_tests`` hook.
|
|
|
|
They help to inspect a testfunction and to generate tests
|
|
|
|
according to test configuration or values specified
|
|
|
|
in the class or module where a test function is defined:
|
|
|
|
|
|
|
|
``metafunc.fixturenames``: set of required function arguments for given function
|
|
|
|
|
|
|
|
``metafunc.function``: underlying python test function
|
|
|
|
|
|
|
|
``metafunc.cls``: class object where the test function is defined in or None.
|
|
|
|
|
|
|
|
``metafunc.module``: the module object where the test function is defined in.
|
|
|
|
|
|
|
|
``metafunc.config``: access to command line opts and general config
|
|
|
|
|
|
|
|
``metafunc.funcargnames``: alias for ``fixturenames``, for pre-2.3 compatibility
|
|
|
|
|
|
|
|
.. automethod:: Metafunc.parametrize
|
|
|
|
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)
|
|
|
|
|
|
|
|
|