test_ok1/doc/en/funcargs.txt

.. _resources:
.. _`funcargs`:
.. _`funcarg mechanism`:

=======================================================
funcargs: resource injection and parametrization
=======================================================

.. note::

    pytest-2.3 introduces major refinements to the test setup and funcarg
    mechanisms introduced to pytest-2.0.  All pre-2.3 usages remain
    supported and several use cases, among them scoping and parametrization
    of funcarg resources, are now easier to accomplish.  For more background, 
    see `compatibility notes`_ and the detailed :ref:`reasoning for the new 
    funcarg and setup functions <funcargcompare>`.

.. _`Dependency injection`: http://en.wikipedia.org/wiki/Dependency_injection

Introduction
====================

pytest supports the injection of test resources into test and setup functions 
and flexibly control their life cycle in relation to the overall test
execution.  Moreover, tests can get executed multiple times if you have
different variants of test resources to test with.  

The basic mechanism for injecting objects is called the *funcarg
mechanism* because objects are injected when a test or setup
**function** states it as an **argument**. The injected argument 
is created by a call to a registered **fixture function** for each argument
name.  This mechanism is an example of `Dependency Injection`_ 
and helps to de-couple test code from the setup of required
objects: at test writing time you do not need to care for the details of
where and how your required test resources are constructed, if they are
shared on a per-class, module or session basis, or if your test function
is invoked multiple times with differently configured resource
instances.

Fixture dependency injection allows to organise test resources 
in a modular explicit way so that test functions state their needs
in their signature.  pytest additionally offers powerful xunit-style 
:ref:`setup functions <setup functions>` for the cases where you need 
to create implicit test state that is not passed explicitely to test functions.

When a test function is invoked multiple times with different arguments we
speak of **parametrized testing**.  You can use it e. g. to repeatedly run test
functions against different database backends or to check that certain
inputs lead to certain outputs.

Concretely, there are three main means of funcarg management:

* a `@pytest.fixture`_ marker to define resource factories,
  their scoping and parametrization.   Factories can themselves 
  receive resources through their function arguments, easing
  the setup of `interdependent resources`_.  Factories can use
  the special `request`_ object to access details from where
  the factory or setup function is called and for registering finalizers.

* a `@pytest.mark.parametrize`_ marker for executing test functions
  multiple times with different argument sets,

* a `pytest_generate_tests`_ plugin hook marker for implementing
  your parametrization for a test function which may depend on 
  command line options, class/module attributes etc.

Apart from making it easy to manage your own test resources
pytest also comes with some :ref:`builtinresources` which
you can use without defining them yourself.  Third-party plugins 
offer yet more domain-specific funcarg resources (for example the
`pytest-django plugin <http://pypi.python.org/pypi/pytest-django>`_) so
that after plugin installation you can simply use them in
your test and setup functions.  This all contributes to high
re-useability of test resource management and goes far beyond what can
be done with the classical xUnit style approach which encodes resource
setup statically into the test source code, leading to duplicate and
hard-to change fixtures.

.. _`@pytest.fixture`:

``@pytest.fixture``: Creating parametrized, scoped resources
=====================================================================

Basic funcarg injection example
-----------------------------------------------------------

Let's look at a simple self-contained test module using a factory
and a funcarg::

    # content of ./test_simplefactory.py
    import pytest

    @pytest.fixture()
    def myfuncarg():
        return 42

    def test_function(myfuncarg):
        assert myfuncarg == 17

Here, the ``test_function`` needs an object named ``myfuncarg`` and thus
py.test will discover and call the ``@pytest.fixture`` marked ``myfuncarg`` 
factory function.  Running the tests looks like this::

    $ py.test test_simplefactory.py
    =========================== test session starts ============================
    platform linux2 -- Python 2.7.3 -- pytest-2.3.0.dev11
    plugins: xdist, bugzilla, cache, oejskit, cli, timeout, pep8, cov
    collecting ... collected 1 items
    
    test_simplefactory.py F
    
    ================================= FAILURES =================================
    ______________________________ test_function _______________________________
    
    myfuncarg = 42
    
        def test_function(myfuncarg):
    >       assert myfuncarg == 17
    E       assert 42 == 17
    
    test_simplefactory.py:8: AssertionError
    ========================= 1 failed in 0.01 seconds =========================

This shows that the test function was called with a ``myfuncarg``
argument value of ``42`` and the assert fails as expected.  Here is 
how py.test comes to call the test function this way:

1. py.test :ref:`finds <test discovery>` the ``test_function`` because
   of the ``test_`` prefix.  The test function needs a function argument
   named ``myfuncarg``.  A matching factory function is discovered by
   looking for a factory function named ``myfuncarg``.

2. ``myfuncarg()`` is called to create a value ``42``.

3. ``test_function(42)`` is now called and results in the above
   reported exception because of the assertion mismatch.

Note that if you misspell a function argument or want
to use one that isn't available, you'll see an error
with a list of available function arguments.

.. Note::

    You can always issue::

        py.test --fixtures test_simplefactory.py

    to see available function arguments.


Location independency of funcarg factories
----------------------------------------------------

If during implementing your tests you realize that you
want to use a factory from multiple test files you can move it
to a :ref:`conftest.py <conftest.py>` file or even separately installable
:ref:`plugins <plugins>` without changing test code.  The discovery of
funcarg factories starts at test classes, then test modules, then
``conftest.py`` files and finally builtin and 3-rd party plugins.



.. _`test generators`:
.. _`parametrizing-tests`:
.. _`parametrized test functions`:

Parametrizing test functions
==========================================================================

While the `@pytest.fixture`_ decorator allows to define parametrization
of funcarg resources at the factory-level, there are also means to
define parametrization at test functions directly:

* `@pytest.mark.parametrize`_ to provide multiple argument sets 
  for a particular test function or class.

* `pytest_generate_tests`_ to implement your own custom parametrization 
  scheme or extensions.

.. _`@pytest.mark.parametrize`:

``@pytest.mark.parametrize``: parametrizing test functions
---------------------------------------------------------------------

.. regendoc: wipe

.. versionadded:: 2.2

The builtin ``pytest.mark.parametrize`` decorator enables
parametrization of arguments for a test function.  Here is a typical example
of a test function that wants check for expected output given a certain input::

    # content of test_expectation.py
    import pytest
    @pytest.mark.parametrize(("input", "expected"), [
        ("3+5", 8),
        ("2+4", 6),
        ("6*9", 42),
    ])
    def test_eval(input, expected):
        assert eval(input) == expected

The ``@parametrize`` decorator defines three different argument sets for the 
two ``(input, output)`` arguments of ``test_eval`` function so the latter
will be run three times::

    $ py.test -q
    collecting ... collected 13 items
    ....F........
    ================================= FAILURES =================================
    ____________________________ test_eval[6*9-42] _____________________________
    
    input = '6*9', expected = 42
    
        @pytest.mark.parametrize(("input", "expected"), [
            ("3+5", 8),
            ("2+4", 6),
            ("6*9", 42),
        ])
        def test_eval(input, expected):
    >       assert eval(input) == expected
    E       assert 54 == 42
    E        +  where 54 = eval('6*9')
    
    test_expectation.py:8: AssertionError
    1 failed, 12 passed in 6.41 seconds

As expected only one pair of input/output values fails the simple test function.
As usual you can see the ``input`` and ``output`` values in the traceback.

Note that there are various ways how you can mark groups of functions,
see :ref:`mark`.


.. _`pytest_generate_tests`:

Basic ``pytest_generate_tests`` example
---------------------------------------------

..  XXX

    > line 598 "Basic ``pytest_generate_tests`` example" - I think this is
    > not a very basic example! I think it is copied from parametrize.txt
    > page, where it might make more sense. Here is what I would consider a
    > basic example.
    > 
    > # code
    > def isSquare(n):
    >     n = n ** 0.5
    >     return int(n) == n
    > 
    > # test file
    > def pytest_generate_tests(metafunc):
    >     squares = [1, 4, 9, 16, 25, 36, 49]
    >     for n in range(1, 50):
    >         expected = n in squares
    >         if metafunc.function.__name__ == 'test_isSquare':
    >             metafunc.addcall(id=n, funcargs=dict(n=n,
    >             expected=expected))
    > 
    > 
    > def test_isSquare(n, expected):
    >     assert isSquare(n) == expected


.. XXX
    consider adding more examples, also mixed (factory-parametrized/test-function-parametrized, see mail from Brianna)

The ``pytest_generate_tests`` hook is typically used if you want
to go beyond what ``@pytest.mark.parametrize`` offers.  For example,
let's say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument.  Let's first write a simple (do-nothing) computation test::

    # content of test_compute.py

    def test_compute(param1):
        assert param1 < 4

Now we add a ``conftest.py`` file containing the addition of a 
command line option and the generation of tests depending on
that option::

    # content of conftest.py

    def pytest_addoption(parser):
        parser.addoption("--all", action="store_true",
            help="run all combinations")

    def pytest_generate_tests(metafunc):
        if 'param1' in metafunc.fixturenames:
            if metafunc.config.option.all:
                end = 5
            else:
                end = 2
            metafunc.parametrize("param1", range(end))

This means that we only run two tests if no option is passed::

    $ py.test -q test_compute.py
    collecting ... collected 2 items
    ..
    2 passed in 0.01 seconds

And we run five tests if we add the ``--all`` option::

    $ py.test -q --all test_compute.py
    collecting ... collected 5 items
    ....F
    ================================= FAILURES =================================
    _____________________________ test_compute[4] ______________________________
    
    param1 = 4
    
        def test_compute(param1):
    >       assert param1 < 4
    E       assert 4 < 4
    
    test_compute.py:3: AssertionError
    1 failed, 4 passed in 0.02 seconds

As expected when running the full range of ``param1`` values
we'll get an error on the last one.

You might want to look at :ref:`more parametrization examples <paramexamples>`.


.. _`metafunc object`:

The **metafunc** object
-------------------------------------------

metafunc objects are passed to the ``pytest_generate_tests`` hook.
They help to inspect a testfunction and to generate tests
according to test configuration or values specified
in the class or module where a test function is defined:

``metafunc.fixturenames``: set of required function arguments for given function

``metafunc.function``: underlying python test function

``metafunc.cls``: class object where the test function is defined in or None.

``metafunc.module``: the module object where the test function is defined in.

``metafunc.config``: access to command line opts and general config

``metafunc.funcargnames``: alias for ``fixturenames``, for pre-2.3 compatibility

.. automethod:: Metafunc.parametrize
.. automethod:: Metafunc.addcall(funcargs=None,id=_notexists,param=_notexists)


.. regendoc:wipe


.. _`compatibility notes`:

.. _`funcargscompat`:

Compatibility notes
============================================================

**Fixtures** were originally introduced to pytest-2.0.  In pytest-2.3 
the mechanism was extended and refined:

* previously funcarg factories were specified with a special 
  ``pytest_funcarg__NAME`` prefix instead of using the 
  ``@pytest.fixture`` decorator.

* Factories received a `request`_ object which managed caching through
  ``request.cached_setup()`` calls and allowed using other funcargs via 
  ``request.getfuncargvalue()`` calls.  These intricate APIs made it hard 
  to do proper parametrization and implement resource caching. The
  new ``@pytest.fixture`` decorator allows to simply declare the scope
  and let pytest figure things out for you.

* if you used parametrization and funcarg factories which made use of
  ``request.cached_setup()`` it is recommeneded to invest a few minutes
  and simplify your fixture function code to use the `@pytest.fixture`_ 
  decorator instead.  This will also allow to take advantage of 
  the `automatic per-resource grouping`_ of tests.

.. note::

    Throughout the pytest documents the ``pytest_funcarg__NAME`` way of
    defining a fixture function is often termed "old-style".  Their
    use remains fully supported and existing code using it should run
    unmodified.