test_ok1/doc/en/parametrize.txt

185 lines
6.3 KiB
Plaintext
Raw Normal View History

.. _`test generators`:
.. _`parametrizing-tests`:
.. _`parametrized test functions`:
.. _`parametrize`:
2012-10-18 18:24:50 +08:00
.. _`parametrize-basics`:
Parametrizing fixtures and test functions
==========================================================================
pytest supports test parametrization in several well-integrated ways:
- :py:func:`pytest.fixture` allows to define :ref:`parametrization
at the level of fixture functions <fixture-parametrize>`.
* `@pytest.mark.parametrize`_ allows to define parametrization at the
function or class level, provides multiple argument/fixture sets
for a particular test function or class.
* `pytest_generate_tests`_ enables implementing your own custom
dynamic parametrization scheme or extensions.
2012-10-18 18:24:50 +08:00
.. _parametrizemark:
.. _`@pytest.mark.parametrize`:
``@pytest.mark.parametrize``: parametrizing test functions
---------------------------------------------------------------------
.. regendoc: wipe
.. versionadded:: 2.2
The builtin ``pytest.mark.parametrize`` decorator enables
parametrization of arguments for a test function. Here is a typical example
of a test function that implements checking that a certain input leads
to an expected output::
# content of test_expectation.py
import pytest
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
assert eval(input) == expected
Here, the ``@parametrize`` decorator defines three different argument
sets for the two ``(input, output)`` arguments of the ``test_eval`` function
which will thus run three times::
$ py.test
=========================== test session starts ============================
2013-04-30 18:26:30 +08:00
platform linux2 -- Python 2.7.3 -- pytest-2.3.5
collected 3 items
test_expectation.py ..F
================================= FAILURES =================================
____________________________ test_eval[6*9-42] _____________________________
input = '6*9', expected = 42
@pytest.mark.parametrize(("input", "expected"), [
("3+5", 8),
("2+4", 6),
("6*9", 42),
])
def test_eval(input, expected):
> assert eval(input) == expected
E assert 54 == 42
E + where 54 = eval('6*9')
test_expectation.py:8: AssertionError
==================== 1 failed, 2 passed in 0.01 seconds ====================
As expected only one pair of input/output values fails the simple test function.
And as usual with test function arguments, you can see the ``input`` and ``output`` values in the traceback.
Note that there ways how you can mark a class or a module,
see :ref:`mark`.
.. _`pytest_generate_tests`:
Basic ``pytest_generate_tests`` example
---------------------------------------------
Sometimes you may want to implement your own parametrization scheme
or implement some dynamism for determining the parameters or scope
of a fixture. For this, you can use the ``pytest_generate_tests`` hook
which is called when collecting a test function. Through the passed in
`metafunc` object you can inspect the requesting test context and, most
importantly, you can call ``metafunc.parametrize()`` to cause
parametrization.
For example, let's say we want to run a test taking string inputs which
we want to set via a new py.test command line option. Let's first write
a simple test accepting a ``stringinput`` fixture function argument::
# content of test_strings.py
def test_valid_string(stringinput):
assert stringinput.isalpha()
Now we add a ``conftest.py`` file containing the addition of a
command line option and the parametrization of our test function::
# content of conftest.py
def pytest_addoption(parser):
parser.addoption("--stringinput", action="append", default=[],
help="list of stringinputs to pass to test functions")
def pytest_generate_tests(metafunc):
if 'stringinput' in metafunc.fixturenames:
metafunc.parametrize("stringinput",
metafunc.config.option.stringinput)
If we now pass two stringinput values, our test will run twice::
$ py.test -q --stringinput="hello" --stringinput="world" test_strings.py
..
Let's also run with a stringinput that will lead to a failing test::
$ py.test -q --stringinput="!" test_strings.py
F
================================= FAILURES =================================
___________________________ test_valid_string[!] ___________________________
stringinput = '!'
def test_valid_string(stringinput):
> assert stringinput.isalpha()
2013-04-30 18:26:30 +08:00
E assert <built-in method isalpha of str object at 0x2ba729dab300>()
E + where <built-in method isalpha of str object at 0x2ba729dab300> = '!'.isalpha
test_strings.py:3: AssertionError
As expected our test function fails.
If you don't specify a stringinput it will be skipped because
``metafunc.parametrize()`` will be called with an empty parameter
listlist::
$ py.test -q -rs test_strings.py
s
========================= short test summary info ==========================
2013-04-30 18:26:30 +08:00
SKIP [1] /home/hpk/p/pytest/.tox/regen/local/lib/python2.7/site-packages/_pytest/python.py:974: got empty parameter set, function test_valid_string at /tmp/doc-exec-240/test_strings.py:1
For further examples, you might want to look at :ref:`more
parametrization examples <paramexamples>`.
.. _`metafunc object`:
The **metafunc** object
-------------------------------------------
.. currentmodule:: _pytest.python
metafunc objects are passed to the ``pytest_generate_tests`` hook.
They help to inspect a testfunction and to generate tests
according to test configuration or values specified
in the class or module where a test function is defined:
``metafunc.fixturenames``: set of required function arguments for given function
``metafunc.function``: underlying python test function
``metafunc.cls``: class object where the test function is defined in or None.
``metafunc.module``: the module object where the test function is defined in.
``metafunc.config``: access to command line opts and general config
``metafunc.funcargnames``: alias for ``fixturenames``, for pre-2.3 compatibility
.. automethod:: Metafunc.parametrize
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)