p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

updating docs wrt funcargs and deprecations - still need updated implementation 

--HG--
branch : trunk
This commit is contained in:
holger krekel 2009-05-12 19:05:36 +02:00
parent 6843c64a95
commit 6c861c273e
3 changed files with 136 additions and 171 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
doc/test/ext.txt
View File

@ -120,6 +120,7 @@ name. Given a filesystem ``fspath`` it is constructed as follows:
the trailing ``.py``. the trailing ``.py``.
Plugin hooks and events Plugin hooks and events
======================================= =======================================

9
doc/test/features.txt
View File

@ -30,7 +30,6 @@ modules a leading ``test_`` or trailing ``_test`` filename.
From each test module every function with a leading ``test_`` From each test module every function with a leading ``test_``
or class with a leading ``Test`` name is collected. or class with a leading ``Test`` name is collected.
.. _`generative tests`:
.. _`collection process`: ext.html#collection-process .. _`collection process`: ext.html#collection-process
Rapidly write integration, functional, unit tests Rapidly write integration, functional, unit tests
@ -259,11 +258,17 @@ For example, in order to avoid running some tests on Win32::
def test_xxx(self): def test_xxx(self):
... ...
.. _`test generators`: funcargs.html#test-generators
.. _`generative tests`:
generative tests: yielding parametrized tests generative tests: yielding parametrized tests
==================================================== ====================================================
**deprecated since 1.0 in favour of `test generators`_**
*Generative tests* are test methods that are *generator functions* which *Generative tests* are test methods that are *generator functions* which
``yield`` callables and their arguments. This is most useful for running a ``yield`` callables and their arguments. This is useful for running a
test function multiple times against different parameters. Example:: test function multiple times against different parameters. Example::
def test_generative(): def test_generative():

291
doc/test/funcargs.txt
View File

@ -1,27 +1,42 @@
====================================================== ======================================================
**funcargs**: powerful test setup and parametrization **funcargs**: test setup and parametrization
====================================================== ======================================================
Since version 1.0 it is possible to provide arguments to test functions, Since version 1.0 test functions can make great use of
often called "funcargs". The funcarg mechanisms were developed with their arguments or "funcargs" for short. py.test helps
these goals in mind: to setup or generate argument values with the goal
of making it easy to:
* **no boilerplate**: cleanly encapsulate test setup and fixtures * separate test function code from test setup/fixtures
* **flexibility**: easily setup test state depending on command line options or environment * manage test value setup and teardown depending on
* **readability**: write simple to read and debug test functions command line options or configuration
* **parametrizing tests**: run a test function multiple times with different parameters * parametrize multiple runs of the same test functions
* present useful debug info if something goes wrong
Using funcargs, test functions become more expressive,
more "templaty" and more test-aspect oriented. In fact,
funcarg mechanisms are meant to be complete and
convenient enough to
* substitute most usages of `xUnit style`_ setup
* substitute all usages of `old-style generative tests`_,
i.e. test functions that use the "yield" statement.
Using yield in test functions is deprecated since 1.0.
.. contents:: Contents: .. _`xUnit style`: xunit_setup.html
:depth: 2 .. _`old-style generative tests`:
Basic mechanisms by example .. _`funcarg provider`:
=============================================
providing single function arguments as needed funcarg providers: setting up test function arguments
--------------------------------------------------------- ==============================================================
Let's look at a simple example of using funcargs within a test module: Test functions can specify one ore more arguments ("funcargs")
and a test module or plugin can define functions that provide
the function argument. Let's look at a self-contained example
that you can put into a test module:
.. sourcecode:: python .. sourcecode:: python
@ -31,66 +46,30 @@ Let's look at a simple example of using funcargs within a test module:
def test_function(myfuncarg): def test_function(myfuncarg):
assert myfuncarg == 42 assert myfuncarg == 42
1. To setup the running of the ``test_function()`` call, py.test
looks up a provider for the ``myfuncarg`` argument.
The provider method is recognized by its ``pytest_funcarg__`` prefix
followed by the requested function argument name.
The `request object`_ gives access to test context.
2. The ``test_function(42)`` call is executed. If the test fails
one can see the original provided value.
generating test functions call
----------------------------------------------------------------------
You can parametrize multiple runs of a test function by
providing multiple values for function arguments. Here
is an example for running the same test function three times.
.. sourcecode:: python
def pytest_genfunc(funcspec):
if "arg1" in funcspec.funcargnames:
for value in range(3):
funcspec.addcall(arg1=value, arg2=range(10))
def test_function(arg1, arg2):
assert arg1 in arg2
Here is what happens: Here is what happens:
1. The ``pytest_genfunc()`` hook will be called once for each test 1. **lookup funcarg provider**: The ``test_function`` needs an value for
function during the collection process. The if-statement makes ``myfuncarg`` to run. The provider is found by its special
sure that we only add calls for functions that actually need the name, ``pytest_funcarg__`` followed by the function
provided value. The `funcspec object`_ provides access to context argument argument name. If a provider cannot be found,
information. a list of all available function arguments is presented.
2. Subsequently the ``test_function()`` will be called three times 2. **setup funcarg value**: ``pytest_funcarg__myfuncarg(request)`` is
with three different values for ``arg1``. called to setup the value for ``myfuncarg``.
Funcarg rules and support objects 3. **execute test** ``test_function(42)`` call is executed.
==================================== If the test fails one can see the original provided
value in the traceback at the top.
It is helpful to understand when function arguments are setup and when
function calls are added. Both providers deal with function arguments
but at different phases of the testing process. New Function calls
can only be added during the test collection phase whereas function
arguments are provided during the actual setup of (already collected)
test functions.
.. _`request object`: .. _`request object`:
funcarg request objects funcarg request objects
------------------------ ------------------------------------------
Request objects encapsulate a request for a function argument from a Request objects are passed to funcarg providers. Request objects
specific test function. Request objects provide access to command line encapsulate a request for a function argument for a
options, the underlying python function and allow interaction specific test function. Request objects allow providers to access
with other providers and the test running process. test configuration and test context:
Attributes of request objects
++++++++++++++++++++++++++++++++++++++++
``request.argname``: name of the requested function argument ``request.argname``: name of the requested function argument
@ -98,12 +77,15 @@ Attributes of request objects
``request.cls``: class object where the test function is defined in or None. ``request.cls``: class object where the test function is defined in or None.
``funcspec.module``: module object where the test function is defined in. ``request.module``: module object where the test function is defined in.
``request.config``: access to command line opts and general config ``request.config``: access to command line opts and general config
``request.param``: if exists is the argument passed by a `parametrizing test generator`_
cleanup after test function execution cleanup after test function execution
++++++++++++++++++++++++++++++++++++++++ ---------------------------------------------
Request objects allow to **register a finalizer method** which is Request objects allow to **register a finalizer method** which is
called after a test function has finished running. called after a test function has finished running.
@ -139,58 +121,94 @@ is no next provider left. See the `decorator example`_
for a use of this method. for a use of this method.
.. _`lookup order`: .. _`test generators`:
.. _`parametrizing test generator`:
Order of provider and test generator lookup generating parametrized tests with funcargs
---------------------------------------------- ===========================================================
Both test generators as well as funcarg providers You can parametrize multiple runs of the same test function
are looked up in the following three scopes: by schedulings new test function calls which get different
funcarg values. Let's look at a simple self-contained
example:
1. test module .. sourcecode:: python
2. local plugins
3. global plugins
Using multiple funcargs def pytest_generate_tests(metafunc):
---------------------------------------- if "numiter" in metafunc.funcargs:
for i in range(10):
metafunc.addcall(param=i)
Test functions can have multiple arguments def pytest_funcarg__numiter(request):
which can either come from a test generator return request.param
or from a provider.
.. _`funcspec object`: def test_func(numiter):
assert numiter < 10
funcspec objects Here is what happens in detail:
------------------------
Funcspecs help to inspect a testfunction and 1. **add test function calls**:
to generate tests with combinations of function argument values. ``pytest_generate_tests(metafunc)`` hook is called once for each test
function. The `metafunc object`_ has context information.
``metafunc.addcall(param=i)`` schedules a new test call
such that function argument providers will see an additional
``arg`` attribute on their request object.
Calling ``funcspec.addcall(**funcargs)`` will add 2. **setup funcarg values**: the ``pytest_funcarg__arg1(request)`` provider is called
a tests function call using the given dictionary 10 times with ten different request objects all pointing to
of function arguments. This addition of a call the same test function. Our provider here simply returns
happens during test collection. the ``arg`` value but we could of course also setup more
heavyweight resources here.
Note that you can provide a name for your test run 3. **execute tests**: ``test_func(numiter)`` is called ten times with
by passing a ``_id=...`` parameter. This name ten different arguments.
will be shown during verbose and traceback
reporting.
Attributes of funcspec objects .. _`metafunc object`:
++++++++++++++++++++++++++++++++++++++++
``funcspec.funcargnames``: set of required function arguments for given function test generators and metafunc objects
-------------------------------------------
``funcspec.function``: underlying python test function metafunc objects are passed to the ``pytest_generate_tests`` hook.
They help to inspect a testfunction and to generate tests
according to test configuration or values specified
in the class or module where a test function is defined:
``funcspec.cls``: class object where the test function is defined in or None. ``metafunc.funcargnames``: set of required function arguments for given function
``funcspec.module``: the module object where the test function is defined in. ``metafunc.function``: underlying python test function
``funcspec.config``: access to command line opts and general config ``metafunc.cls``: class object where the test function is defined in or None.
``metafunc.module``: the module object where the test function is defined in.
``metafunc.config``: access to command line opts and general config
Useful Funcarg Tutorial Examples the ``metafunc.addcall()`` method
-----------------------------------------------
.. sourcecode:: python
def addcall(id=None, param=None):
""" trigger a later test function call. """
The specified ``param`` will be seen by the
`funcarg provider`_ as a ``request.param`` attribute.
If you provide an `id`` it will be used for reporting
and identification purposes. If you don't supply an `id`
the stringified counter of the list of added calls will be used.
``id`` values needs to be unique between all
invocations for a given test function.
*Test generators are called during test collection which
is separate from the actual test setup and test run.
With distributed testing setting up funcargs will
even happen in a different process. Therefore one should
defer setup of heavyweight objects to funcarg providers.*
Funcarg Tutorial Examples
======================================= =======================================
application specific test setup application specific test setup
@ -237,9 +255,6 @@ following code into a local ``conftest.py``:
def myapp(self): def myapp(self):
return MyApp() return MyApp()
py.test finds the ``pytest_funcarg__mysetup`` method by
name, see also `lookup order`_.
To run the example we put a pseudo MyApp object into ``myapp.py``: To run the example we put a pseudo MyApp object into ``myapp.py``:
.. sourcecode:: python .. sourcecode:: python
@ -369,11 +384,11 @@ extend the `accept example`_ by putting this in our test class:
def test_sometest(self, accept): def test_sometest(self, accept):
assert accept.tmpdir.join("special").check() assert accept.tmpdir.join("special").check()
According to the the `lookup order`_ our module level provider Our module level provider will be invoked first and it can
will be invoked first and it can ask ask its request object to ask its request object to call the next provider and then
call the next provider and then decorate its result. This decorate its result. This mechanism allows us to stay
mechanism allows us to stay ignorant of how/where the ignorant of how/where the function argument is provided -
function argument is provided. in our example from a ConftestPlugin but could be any plugin.
sidenote: the temporary directory used here are instances of sidenote: the temporary directory used here are instances of
the `py.path.local`_ class which provides many of the os.path the `py.path.local`_ class which provides many of the os.path
@ -382,62 +397,6 @@ methods in a convenient way.
.. _`py.path.local`: ../path.html#local .. _`py.path.local`: ../path.html#local
parametrize test functions with multiple func args
--------------------------------------------------------------------------
You can trigger calling test functions which take more
than one function argument. Consider this example:
.. sourcecode:: python
def pytest_genfunc(funcspec):
for arg1 in range(2):
for arg2 in (10, 20):
funcspec.addcall(arg1=arg1, arg2=arg2)
# the actual test function
def test_function(arg1, arg2):
pass
Running this test module will result in ``test_function``
being called four times, with the following arguments::
test_function(0, 10)
test_function(0, 20)
test_function(1, 10)
test_function(2, 20)
example: test functions with pre-generated and provided funcargs
-------------------------------------------------------------------
You can mix generated function arguments and normally
provided ones. Consider this module:
.. sourcecode:: python
def pytest_genfunc(funcspec):
if "arg1" in funcspec.funcargnames: # test_function2 does not have it
funcspec.addcall(arg1=10)
funcspec.addcall(arg1=20)
def pytest_funcarg__arg2(request):
return [10, 20]
def test_function(arg1, arg2):
assert arg1 in arg2
def test_function2(arg2):
assert args2 == [10, 20]
Running this test module will result in ``test_function``
being called twice, with these arguments::
test_function(10, [10, 20])
test_function(20, [10, 20])
Questions and Answers Questions and Answers
================================== ==================================