test_ok1/doc/test/funcargs.txt

======================================================
**funcargs**: powerful test setup and parametrization
======================================================

Since version 1.0 it is possible to provide arguments to test functions,
often called "funcargs".  The funcarg mechanisms were developed with
these goals in mind: 

* **no boilerplate**: cleanly encapsulate test setup and fixtures
* **flexibility**: easily setup test state depending on command line options or environment
* **readability**: write simple to read and debug test functions 
* **parametrizing tests**: run a test function multiple times with different parameters


.. contents:: Contents:
    :depth: 2

Basic mechanisms by example
=============================================

providing single function arguments as needed 
---------------------------------------------------------

Let's look at a simple example of using funcargs within a test module:

.. sourcecode:: python

    def pytest_funcarg__myfuncarg(request):
        return 42

    def test_function(myfuncarg):
        assert myfuncarg == 42

1. To setup the running of the ``test_function()`` call, py.test 
   looks up a provider for the ``myfuncarg`` argument. 
   The provider method is recognized by its ``pytest_funcarg__`` prefix 
   followed by the requested function argument name. 
   The `request object`_ gives access to test context. 

2. A ``test_function(42)`` call is executed.  If the test fails
   one can see the original provided value. 


generating test runs with multiple function argument values 
----------------------------------------------------------------------

You can parametrize multiple runs of a test function by 
providing multiple values for function arguments.  Here 
is an example for running the same test function three times. 

.. sourcecode:: python

    def pytest_genfunc(funcspec):
        if "arg1" in funcspec.funcargnames:
            for value in range(3):
                funcspec.addcall(arg1=value)

    def test_function(arg1): 
        assert myfuncarg in (0, 1, 2)

Here is what happens:

1. The ``pytest_genfunc()`` hook will be called once for each test
   function.  The if-statement makes sure that we only add calls
   for functions that actually need the provided value. 
   The `funcspec object`_ provides access to context information. 

2. Subsequently the ``test_function()`` will be called three times 
   with three different values for ``arg1``.  

Funcarg rules and support objects
====================================

.. _`request object`:

funcarg request objects
------------------------

Request objects encapsulate a request for a function argument from a
specific test function.  Request objects provide access to command line
options, the underlying python function and allow interaction 
with other providers and the test running process.  

Attributes of request objects
++++++++++++++++++++++++++++++++++++++++

``request.argname``: name of the requested function argument 

``request.function``: python function object requesting the argument

``request.cls``: class object where the test function is defined in or None. 

``funcspec.module``: module object where the test function is defined in. 

``request.config``: access to command line opts and general config

cleanup after test function execution
++++++++++++++++++++++++++++++++++++++++

Request objects allow to **register a finalizer method** which is 
called after a test function has finished running. 
This is useful for tearing down or cleaning up
test state. Here is a basic example for providing
a ``myfile`` object that will be closed upon test
function finish:

.. sourcecode:: python

    def pytest_funcarg__myfile(self, request):
        # ... create and open a "myfile" object ...
        request.addfinalizer(lambda: myfile.close())
        return myfile


decorating other funcarg providers 
++++++++++++++++++++++++++++++++++++++++

If you want to **decorate a function argument** that is 
provided elsewhere you can ask the request object 
to provide the "next" value:

.. sourcecode:: python

    def pytest_funcarg__myfile(self, request):
        myfile = request.call_next_provider()
        # do something extra 
        return myfile

This will raise a ``request.Error`` exception if there 
is no next provider left.  See the `decorator example`_ 
for a use of this method. 


.. _`lookup order`: 

Order of provider and test generator lookup
----------------------------------------------

Both test generators as well as funcarg providers 
are looked up in the following three scopes:

1. test module 
2. local plugins
3. global plugins 

Using multiple funcargs 
----------------------------------------

Test functions can have multiple arguments
which can either come from a test generator 
or from a provider. 

.. _`funcspec object`:

funcspec objects
------------------------

Runspecs help to inspect a testfunction and 
to generate tests with combinations of function argument values. 

Calling ``funcspec.addcall(**funcargs)`` will add 
a tests function call using the given dictionary
of function arguments.  This addition of a call
happens during test collection. 

Attributes of funcspec objects
++++++++++++++++++++++++++++++++++++++++

``funcspec.funcargnames``: set of required function arguments for given function

``funcspec.function``: underlying python test function 

``funcspec.cls``: class object where the test function is defined in or None. 

``funcspec.module``: the module object where the test function is defined in. 

``funcspec.config``: access to command line opts and general config


Useful Funcarg Tutorial Examples
=======================================

application specific test setup 
---------------------------------------------------------

Here is a basic useful step-wise example for handling application
specific test setup.  The goal is to have one place where we have the 
glue code for bootstrapping and configuring application objects and allow
test modules and test functions to stay ignorant of involved details. 

step 1: use and implement a test/app-specific "mysetup"
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Let's write a simple test function living in a test file
``test_sample.py`` that uses a ``mysetup`` funcarg for accessing test
specific setup. 

.. sourcecode:: python

    # ./test_sample.py
    def test_answer(mysetup): 
        app = mysetup.myapp()
        answer = app.question()
        assert answer == 42

To run this test py.test needs to find and call a provider to 
obtain the required ``mysetup`` function argument.  The test 
function interacts with the provided application specific setup. 

To provide the ``mysetup`` function argument we write down
a provider method in a `local plugin`_ by putting the
following code into a local ``conftest.py``:

.. sourcecode:: python

    # ./conftest.py 
    from myapp import MyApp

    class ConftestPlugin:
        def pytest_funcarg__mysetup(self, request):
            return MySetup()

    class MySetup:
        def myapp(self):
            return MyApp()

py.test finds the ``pytest_funcarg__mysetup`` method by
name, see also `lookup order`_.

To run the example we put a pseudo MyApp object into ``myapp.py``:

.. sourcecode:: python

    # ./myapp.py
    class MyApp:
        def question(self):
            return 6 * 9

You can now run the test with ``py.test test_sample.py`` which will
show this failure:

.. sourcecode:: python 

        def test_answer(mysetup):
            app = mysetup.myapp()
            answer = app.question()
    >       assert answer == 42
    E       assert 54 == 42

If you are confused as to what the concrete question or answers 
mean actually, please visit here_ :) 

.. _here: http://uncyclopedia.wikia.com/wiki/The_Hitchhiker's_Guide_to_the_Galaxy
.. _`local plugin`: ext.html#local-plugin


step 2: adding a command line option 
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

If you provide a "funcarg" from a plugin you can easily make methods
depend on command line options or environment settings.  Let's write a
local plugin that adds a command line option to ``py.test`` invocations: 

.. sourcecode:: python

    class ConftestPlugin:
        def pytest_addoption(self, parser):
            parser.addoption("--ssh", action="store", default=None,
                help="specify ssh host to run tests with")

        pytest_funcarg__mysetup = MySetupFuncarg

    class MySetupFuncarg:
        def __init__(self, request):
            self.request = request 
        def getsshconnection(self):
            host = self.request.config.option.ssh
            if host is None:
                py.test.skip("specify ssh host with --ssh to run this test")
            return py.execnet.SshGateway(host)

Now any test functions can use the ``mysetup.getsshconnection()`` method like this:

.. sourcecode:: python

    class TestClass:
        def test_function(self, mysetup):
            conn = mysetup.getsshconnection()
            # work with conn 
      
Running this without specifying a command line option will result in a skipped
test_function.  

.. _`accept example`: 

example: specifying and selecting acceptance tests 
--------------------------------------------------------------

.. sourcecode:: python

    class ConftestPlugin:
        def pytest_option(self, parser):
            group = parser.getgroup("myproject")
            group.addoption("-A", dest="acceptance", action="store_true", 
                help="run (slow) acceptance tests")

        def pytest_funcarg__accept(self, request):
            return AcceptFuncarg(request)

    class AcceptFuncarg:
        def __init__(self, request):
            if not request.config.option.acceptance:
                py.test.skip("specify -A to run acceptance tests")
            self.tmpdir = request.config.mktemp(request.function.__name__, numbered=True)
            
        def run(self, cmd):
            """ called by test code to execute an acceptance test. """ 
            self.tmpdir.chdir() 
            return py.process.cmdexec(cmd)

   
and the actual test function example:

.. sourcecode:: python

    def test_some_acceptance_aspect(accept):
        accept.tmpdir.mkdir("somesub")
        result = accept.run("ls -la")
        assert "somesub" in result
   
If you run this test without specifying a command line option
the test will get skipped with an appropriate message. Otherwise
you can start to add convenience and test support methods 
to your AcceptFuncarg and drive running of tools or
applications and provide ways to do assertions about
the output. 

.. _`decorator example`: 

example: decorating a funcarg in a test module
--------------------------------------------------------------

For larger scale setups it's sometimes useful to decorare 
a funcarg just for a particular test module.  We can 
extend the `accept example`_ by putting this in our test class:

.. sourcecode:: python

    def pytest_funcarg__accept(self, request):
        arg = request.call_next_provider()
        # create a special layout in our tempdir
        arg.tmpdir.mkdir("special")
        return arg 

    class TestSpecialAcceptance:
        def test_sometest(self, accept):
            assert accept.tmpdir.join("special").check()

According to the the `lookup order`_ our module level provider 
will be invoked first and it can ask ask its request object to 
call the next provider and then decorate its result.  This 
mechanism allows us to stay ignorant of how/where the 
function argument is provided.  

sidenote: the temporary directory used here are instances of 
the `py.path.local`_ class which provides many of the os.path 
methods in a convenient way. 

.. _`py.path.local`: ../path.html#local


parametrize test functions with multiple func args 
--------------------------------------------------------------------------

You can trigger calling test functions which take more
than one function argument.  Consider this example:

.. sourcecode:: python

    def pytest_genfunc(funcspec):
        for arg1 in range(2):
            for arg2 in (10, 20):
                funcspec.addcall(arg1=arg1, arg2=arg2)

    # the actual test function

    def test_function(arg1, arg2):
        pass

Running this test module will result in ``test_function`` 
being called four times, with the following arguments::

    test_function(0, 10)
    test_function(0, 20)
    test_function(1, 10)
    test_function(2, 20)


example: test functions with pre-generated and provided funcargs 
-------------------------------------------------------------------

You can mix generated function arguments and normally 
provided ones. Consider this module:

.. sourcecode:: python

    def pytest_genfunc(funcspec):
        if "arg1" in funcspec.funcargnames: # test_function2 does not have it 
            funcspec.addcall(arg1=10)
            funcspec.addcall(arg1=20)

    def pytest_funcarg__arg2(request):
        return [10, 20]

    def test_function(arg1, arg2):
        assert arg1 in arg2 

    def test_function2(arg2):
        assert args2 == [10, 20]

Running this test module will result in ``test_function`` 
being called twice, with these arguments::

    test_function(10, [10, 20])
    test_function(20, [10, 20])


Questions and Answers
==================================

.. _`why pytest_pyfuncarg__ methods?`:

Why ``pytest_funcarg__*`` methods? 
------------------------------------

When experimenting with funcargs we also
considered an explicit registration mechanism, i.e. calling a register
method on the config object.  But lacking a good use case for this
indirection and flexibility we decided to go for `Convention over
Configuration`_ and allow to directly specify the provider.  It has the
positive implication that you should be able to "grep" for
``pytest_funcarg__MYARG`` and will find all providing sites (usually
exactly one).  

.. _`Convention over Configuration`: http://en.wikipedia.org/wiki/Convention_over_Configuration