implement funcargs according to docs, streamline docs
--HG-- branch : trunk
This commit is contained in:
holger krekel
parent
286460b94e
commit
1e3acc66d6
|
|
@ -11,7 +11,7 @@ of making it easy to:
|
|||
* manage test value setup and teardown depending on
|
||||
command line options or configuration
|
||||
* parametrize multiple runs of the same test functions
|
||||
* present useful debug info if something goes wrong
|
||||
* present useful debug info if setup goes wrong
|
||||
|
||||
Using funcargs, test functions become more expressive,
|
||||
more "templaty" and more test-aspect oriented. In fact,
|
||||
|
|
@ -35,8 +35,8 @@ funcarg providers: setting up test function arguments
|
|||
|
||||
Test functions can specify one ore more arguments ("funcargs")
|
||||
and a test module or plugin can define functions that provide
|
||||
the function argument. Let's look at a self-contained example
|
||||
that you can put into a test module:
|
||||
the function argument. Let's look at a simple self-contained
|
||||
example that you can put into a test module:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
|
|
@ -48,28 +48,30 @@ that you can put into a test module:
|
|||
|
||||
Here is what happens:
|
||||
|
||||
1. **lookup funcarg provider**: The ``test_function`` needs an value for
|
||||
``myfuncarg`` to run. The provider is found by its special
|
||||
name, ``pytest_funcarg__`` followed by the function
|
||||
argument argument name. If a provider cannot be found,
|
||||
a list of all available function arguments is presented.
|
||||
1. **lookup funcarg provider**: For executing ``test_function(myfuncarg)``
|
||||
a value is needed. A value provider is found by looking for a
|
||||
function with a special name of ``pytest_funcarg__${ARGNAME}``.
|
||||
|
||||
2. **setup funcarg value**: ``pytest_funcarg__myfuncarg(request)`` is
|
||||
called to setup the value for ``myfuncarg``.
|
||||
called to setup and return the value for ``myfuncarg``.
|
||||
|
||||
3. **execute test** ``test_function(42)`` call is executed.
|
||||
If the test fails one can see the original provided
|
||||
value in the traceback at the top.
|
||||
|
||||
Note that if a provider cannot be found a list of
|
||||
available function arguments will be provided.
|
||||
|
||||
For providers that makes use of the `request object`_
|
||||
please look into the `tutorial examples`_.
|
||||
|
||||
.. _`request object`:
|
||||
|
||||
funcarg request objects
|
||||
------------------------------------------
|
||||
|
||||
Request objects are passed to funcarg providers. Request objects
|
||||
Request objects are passed to funcarg providers. They
|
||||
encapsulate a request for a function argument for a
|
||||
specific test function. Request objects allow providers to access
|
||||
test configuration and test context:
|
||||
specific test function. Request objects allow providers
|
||||
to access test configuration and test context:
|
||||
|
||||
``request.argname``: name of the requested function argument
|
||||
|
||||
|
|
@ -81,7 +83,7 @@ test configuration and test context:
|
|||
|
||||
``request.config``: access to command line opts and general config
|
||||
|
||||
``request.param``: if exists is the argument passed by a `parametrizing test generator`_
|
||||
``request.param``: if exists was passed by a `parametrizing test generator`_
|
||||
|
||||
|
||||
cleanup after test function execution
|
||||
|
|
@ -90,9 +92,9 @@ cleanup after test function execution
|
|||
Request objects allow to **register a finalizer method** which is
|
||||
called after a test function has finished running.
|
||||
This is useful for tearing down or cleaning up
|
||||
test state. Here is a basic example for providing
|
||||
a ``myfile`` object that will be closed upon test
|
||||
function finish:
|
||||
test state related to a function argument. Here is a basic
|
||||
example for providing a ``myfile`` object that will be
|
||||
closed upon test function finish:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
|
|
@ -152,7 +154,7 @@ Here is what happens in detail:
|
|||
function. The `metafunc object`_ has context information.
|
||||
``metafunc.addcall(param=i)`` schedules a new test call
|
||||
such that function argument providers will see an additional
|
||||
``arg`` attribute on their request object.
|
||||
``param`` attribute on their request object.
|
||||
|
||||
2. **setup funcarg values**: the ``pytest_funcarg__arg1(request)`` provider is called
|
||||
10 times with ten different request objects all pointing to
|
||||
|
|
@ -208,6 +210,8 @@ even happen in a different process. Therefore one should
|
|||
defer setup of heavyweight objects to funcarg providers.*
|
||||
|
||||
|
||||
.. _`tutorial examples`:
|
||||
|
||||
Funcarg Tutorial Examples
|
||||
=======================================
|
||||
|
||||
|
|
@ -291,13 +295,7 @@ local plugin that adds a command line option to ``py.test`` invocations:
|
|||
|
||||
.. sourcecode:: python
|
||||
|
||||
class ConftestPlugin:
|
||||
def pytest_addoption(self, parser):
|
||||
parser.addoption("--ssh", action="store", default=None,
|
||||
help="specify ssh host to run tests with")
|
||||
|
||||
pytest_funcarg__mysetup = MySetupFuncarg
|
||||
|
||||
# ./conftest.py
|
||||
class MySetupFuncarg:
|
||||
def __init__(self, request):
|
||||
self.request = request
|
||||
|
|
@ -307,10 +305,19 @@ local plugin that adds a command line option to ``py.test`` invocations:
|
|||
py.test.skip("specify ssh host with --ssh to run this test")
|
||||
return py.execnet.SshGateway(host)
|
||||
|
||||
class ConftestPlugin:
|
||||
def pytest_addoption(self, parser):
|
||||
parser.addoption("--ssh", action="store", default=None,
|
||||
help="specify ssh host to run tests with")
|
||||
|
||||
# alias the above class as the "mysetup" provider
|
||||
pytest_funcarg__mysetup = MySetupFuncarg
|
||||
|
||||
Now any test functions can use the ``mysetup.getsshconnection()`` method like this:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
# ./test_function.py
|
||||
class TestClass:
|
||||
def test_function(self, mysetup):
|
||||
conn = mysetup.getsshconnection()
|
||||
|
|
@ -326,6 +333,7 @@ example: specifying and selecting acceptance tests
|
|||
|
||||
.. sourcecode:: python
|
||||
|
||||
# ./conftest.py
|
||||
class ConftestPlugin:
|
||||
def pytest_option(self, parser):
|
||||
group = parser.getgroup("myproject")
|
||||
|
|
|
|||
|
|
@ -10,14 +10,10 @@ def getfuncargnames(function):
|
|||
|
||||
def fillfuncargs(function):
|
||||
""" fill missing funcargs. """
|
||||
if function._args:
|
||||
# functions yielded from a generator: we don't want
|
||||
# to support that because we want to go here anyway:
|
||||
# http://bitbucket.org/hpk42/py-trunk/issue/2/next-generation-generative-tests
|
||||
pass
|
||||
else:
|
||||
# standard Python Test function/method case
|
||||
for argname in getfuncargnames(function.obj):
|
||||
argnames = getfuncargnames(function.obj)
|
||||
if argnames:
|
||||
assert not function._args, "yielded functions cannot have funcargs"
|
||||
for argname in argnames:
|
||||
if argname not in function.funcargs:
|
||||
request = FuncargRequest(pyfuncitem=function, argname=argname)
|
||||
try:
|
||||
|
|
@ -25,12 +21,15 @@ def fillfuncargs(function):
|
|||
except request.Error:
|
||||
request._raiselookupfailed()
|
||||
|
||||
class CallSpec:
|
||||
def __init__(self, id, funcargs):
|
||||
self.id = id
|
||||
self.funcargs = funcargs
|
||||
|
||||
class FuncSpecs:
|
||||
_notexists = object()
|
||||
class CallSpec:
|
||||
def __init__(self, id, param):
|
||||
self.id = id
|
||||
if param is not _notexists:
|
||||
self.param = param
|
||||
|
||||
class Metafunc:
|
||||
def __init__(self, function, config=None, cls=None, module=None):
|
||||
self.config = config
|
||||
self.module = module
|
||||
|
|
@ -41,20 +40,14 @@ class FuncSpecs:
|
|||
self._calls = []
|
||||
self._ids = py.builtin.set()
|
||||
|
||||
def addcall(self, _id=None, **kwargs):
|
||||
for argname in kwargs:
|
||||
if argname[0] == "_":
|
||||
raise TypeError("argument %r is not a valid keyword." % argname)
|
||||
if argname not in self.funcargnames:
|
||||
raise ValueError("function %r has no funcarg %r" %(
|
||||
self.function, argname))
|
||||
if _id is None:
|
||||
_id = len(self._calls)
|
||||
_id = str(_id)
|
||||
if _id in self._ids:
|
||||
raise ValueError("duplicate id %r" % _id)
|
||||
self._ids.add(_id)
|
||||
self._calls.append(CallSpec(_id, kwargs))
|
||||
def addcall(self, id=None, param=_notexists):
|
||||
if id is None:
|
||||
id = len(self._calls)
|
||||
id = str(id)
|
||||
if id in self._ids:
|
||||
raise ValueError("duplicate id %r" % id)
|
||||
self._ids.add(id)
|
||||
self._calls.append(CallSpec(id, param))
|
||||
|
||||
class FunctionCollector(py.test.collect.Collector):
|
||||
def __init__(self, name, parent, calls):
|
||||
|
|
@ -66,7 +59,7 @@ class FunctionCollector(py.test.collect.Collector):
|
|||
l = []
|
||||
for call in self.calls:
|
||||
function = self.parent.Function(name="%s[%s]" %(self.name, call.id),
|
||||
parent=self, funcargs=call.funcargs, callobj=self.obj)
|
||||
parent=self, requestparam=call.param, callobj=self.obj)
|
||||
l.append(function)
|
||||
return l
|
||||
|
||||
|
|
@ -84,6 +77,8 @@ class FuncargRequest:
|
|||
self.cls = getattr(self.function, 'im_class', None)
|
||||
self.config = pyfuncitem.config
|
||||
self.fspath = pyfuncitem.fspath
|
||||
if hasattr(pyfuncitem, '_requestparam'):
|
||||
self.param = pyfuncitem._requestparam
|
||||
self._plugins = self.config.pluginmanager.getplugins()
|
||||
self._plugins.append(self.module)
|
||||
self._provider = self.config.pluginmanager.listattr(
|
||||
|
|
@ -91,9 +86,6 @@ class FuncargRequest:
|
|||
attrname=self._argprefix + str(argname)
|
||||
)
|
||||
|
||||
def __repr__(self):
|
||||
return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
|
||||
|
||||
def call_next_provider(self):
|
||||
if not self._provider:
|
||||
raise self.Error("no provider methods left")
|
||||
|
|
@ -103,6 +95,9 @@ class FuncargRequest:
|
|||
def addfinalizer(self, finalizer):
|
||||
self._pyfuncitem.addfinalizer(finalizer)
|
||||
|
||||
def __repr__(self):
|
||||
return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
|
||||
|
||||
def _raiselookupfailed(self):
|
||||
available = []
|
||||
for plugin in self._plugins:
|
||||
|
|
|
|||
|
|
@ -53,7 +53,7 @@ class PluginHooks:
|
|||
""" return custom item/collector for a python object in a module, or None. """
|
||||
pytest_pycollect_obj.firstresult = True
|
||||
|
||||
def pytest_genfunc(self, funcspec):
|
||||
def pytest_generate_tests(self, metafunc):
|
||||
""" generate (multiple) parametrized calls to a test function."""
|
||||
|
||||
def pytest_collectstart(self, collector):
|
||||
|
|
|
|||
|
|
@ -151,13 +151,13 @@ class PyCollectorMixin(PyobjMixin, py.test.collect.Collector):
|
|||
# to work to get at the class
|
||||
clscol = self._getparent(Class)
|
||||
cls = clscol and clscol.obj or None
|
||||
funcspec = funcargs.FuncSpecs(funcobj, config=self.config, cls=cls, module=module)
|
||||
gentesthook = self.config.hook.pytest_genfunc.clone(extralookup=module)
|
||||
gentesthook(funcspec=funcspec)
|
||||
if not funcspec._calls:
|
||||
metafunc = funcargs.Metafunc(funcobj, config=self.config, cls=cls, module=module)
|
||||
gentesthook = self.config.hook.pytest_generate_tests.clone(extralookup=module)
|
||||
gentesthook(metafunc=metafunc)
|
||||
if not metafunc._calls:
|
||||
return self.Function(name, parent=self)
|
||||
return funcargs.FunctionCollector(name=name,
|
||||
parent=self, calls=funcspec._calls)
|
||||
parent=self, calls=metafunc._calls)
|
||||
|
||||
class Module(py.test.collect.File, PyCollectorMixin):
|
||||
def _getobj(self):
|
||||
|
|
@ -325,13 +325,15 @@ class Function(FunctionMixin, py.test.collect.Item):
|
|||
""" a Function Item is responsible for setting up
|
||||
and executing a Python callable test object.
|
||||
"""
|
||||
def __init__(self, name, parent=None, config=None, args=(), funcargs=None, callobj=_dummy):
|
||||
def __init__(self, name, parent=None, config=None, args=(),
|
||||
requestparam=_dummy, callobj=_dummy):
|
||||
super(Function, self).__init__(name, parent, config=config)
|
||||
self._finalizers = []
|
||||
self._args = args
|
||||
if funcargs is None:
|
||||
funcargs = {}
|
||||
self.funcargs = funcargs
|
||||
if not args: # yielded functions (deprecated) have positional args
|
||||
self.funcargs = {}
|
||||
if requestparam is not _dummy:
|
||||
self._requestparam = requestparam
|
||||
if callobj is not _dummy:
|
||||
self._obj = callobj
|
||||
|
||||
|
|
@ -352,11 +354,13 @@ class Function(FunctionMixin, py.test.collect.Item):
|
|||
|
||||
def runtest(self):
|
||||
""" execute the given test function. """
|
||||
self.config.hook.pytest_pyfunc_call(pyfuncitem=self,
|
||||
args=self._args, kwargs=self.funcargs)
|
||||
kwargs = getattr(self, 'funcargs', {})
|
||||
self.config.hook.pytest_pyfunc_call(
|
||||
pyfuncitem=self, args=self._args, kwargs=kwargs)
|
||||
|
||||
def setup(self):
|
||||
super(Function, self).setup()
|
||||
if hasattr(self, 'funcargs'):
|
||||
funcargs.fillfuncargs(self)
|
||||
|
||||
def __eq__(self, other):
|
||||
|
|
|
|||
|
|
@ -131,100 +131,91 @@ class TestRequest:
|
|||
req = funcargs.FuncargRequest(item, "xxx")
|
||||
assert req.fspath == modcol.fspath
|
||||
|
||||
class TestFuncSpecs:
|
||||
class TestMetafunc:
|
||||
def test_no_funcargs(self, testdir):
|
||||
def function(): pass
|
||||
funcspec = funcargs.FuncSpecs(function)
|
||||
assert not funcspec.funcargnames
|
||||
metafunc = funcargs.Metafunc(function)
|
||||
assert not metafunc.funcargnames
|
||||
|
||||
def test_function_basic(self):
|
||||
def func(arg1, arg2="qwe"): pass
|
||||
funcspec = funcargs.FuncSpecs(func)
|
||||
assert len(funcspec.funcargnames) == 1
|
||||
assert 'arg1' in funcspec.funcargnames
|
||||
assert funcspec.function is func
|
||||
assert funcspec.cls is None
|
||||
metafunc = funcargs.Metafunc(func)
|
||||
assert len(metafunc.funcargnames) == 1
|
||||
assert 'arg1' in metafunc.funcargnames
|
||||
assert metafunc.function is func
|
||||
assert metafunc.cls is None
|
||||
|
||||
def test_addcall_with_id(self):
|
||||
def test_addcall_no_args(self):
|
||||
def func(arg1): pass
|
||||
funcspec = funcargs.FuncSpecs(func)
|
||||
py.test.raises(TypeError, """
|
||||
funcspec.addcall(_xyz=10)
|
||||
""")
|
||||
funcspec.addcall(_id="hello", arg1=100)
|
||||
py.test.raises(ValueError, "funcspec.addcall(_id='hello', arg1=100)")
|
||||
call = funcspec._calls[0]
|
||||
assert call.id == "hello"
|
||||
assert call.funcargs == {'arg1': 100}
|
||||
metafunc = funcargs.Metafunc(func)
|
||||
metafunc.addcall()
|
||||
assert len(metafunc._calls) == 1
|
||||
call = metafunc._calls[0]
|
||||
assert call.id == "0"
|
||||
assert not hasattr(call, 'param')
|
||||
|
||||
def test_addcall_basic(self):
|
||||
def test_addcall_id(self):
|
||||
def func(arg1): pass
|
||||
funcspec = funcargs.FuncSpecs(func)
|
||||
py.test.raises(ValueError, """
|
||||
funcspec.addcall(notexists=100)
|
||||
""")
|
||||
funcspec.addcall(arg1=100)
|
||||
assert len(funcspec._calls) == 1
|
||||
assert funcspec._calls[0].funcargs == {'arg1': 100}
|
||||
metafunc = funcargs.Metafunc(func)
|
||||
metafunc.addcall(id=1)
|
||||
py.test.raises(ValueError, "metafunc.addcall(id=1)")
|
||||
py.test.raises(ValueError, "metafunc.addcall(id='1')")
|
||||
metafunc.addcall(id=2)
|
||||
assert len(metafunc._calls) == 2
|
||||
assert metafunc._calls[0].id == "1"
|
||||
assert metafunc._calls[1].id == "2"
|
||||
|
||||
def test_addcall_two(self):
|
||||
def test_addcall_param(self):
|
||||
def func(arg1): pass
|
||||
funcspec = funcargs.FuncSpecs(func)
|
||||
funcspec.addcall(arg1=100)
|
||||
funcspec.addcall(arg1=101)
|
||||
assert len(funcspec._calls) == 2
|
||||
assert funcspec._calls[0].funcargs == {'arg1': 100}
|
||||
assert funcspec._calls[1].funcargs == {'arg1': 101}
|
||||
metafunc = funcargs.Metafunc(func)
|
||||
class obj: pass
|
||||
metafunc.addcall(param=obj)
|
||||
metafunc.addcall(param=obj)
|
||||
metafunc.addcall(param=1)
|
||||
assert len(metafunc._calls) == 3
|
||||
assert metafunc._calls[0].param == obj
|
||||
assert metafunc._calls[1].param == obj
|
||||
assert metafunc._calls[2].param == 1
|
||||
|
||||
|
||||
class TestGenfuncFunctional:
|
||||
def test_attributes(self, testdir):
|
||||
p = testdir.makepyfile("""
|
||||
# assumes that generate/provide runs in the same process
|
||||
import py
|
||||
def pytest_genfunc(funcspec):
|
||||
funcspec.addcall(funcspec=funcspec)
|
||||
def pytest_generate_tests(metafunc):
|
||||
metafunc.addcall(param=metafunc)
|
||||
|
||||
def pytest_funcarg__metafunc(request):
|
||||
return request.param
|
||||
|
||||
def test_function(metafunc):
|
||||
assert metafunc.config == py.test.config
|
||||
assert metafunc.module.__name__ == __name__
|
||||
assert metafunc.function == test_function
|
||||
assert metafunc.cls is None
|
||||
|
||||
def test_function(funcspec):
|
||||
assert funcspec.config == py.test.config
|
||||
assert funcspec.module.__name__ == __name__
|
||||
assert funcspec.function == test_function
|
||||
assert funcspec.cls is None
|
||||
class TestClass:
|
||||
def test_method(self, funcspec):
|
||||
assert funcspec.config == py.test.config
|
||||
assert funcspec.module.__name__ == __name__
|
||||
# XXX actually have the unbound test function here?
|
||||
assert funcspec.function == TestClass.test_method.im_func
|
||||
assert funcspec.cls == TestClass
|
||||
def test_method(self, metafunc):
|
||||
assert metafunc.config == py.test.config
|
||||
assert metafunc.module.__name__ == __name__
|
||||
# XXX actually have an unbound test function here?
|
||||
assert metafunc.function == TestClass.test_method.im_func
|
||||
assert metafunc.cls == TestClass
|
||||
""")
|
||||
result = testdir.runpytest(p, "-v")
|
||||
result.stdout.fnmatch_lines([
|
||||
"*2 passed in*",
|
||||
])
|
||||
|
||||
def test_arg_twice(self, testdir):
|
||||
testdir.makeconftest("""
|
||||
class ConftestPlugin:
|
||||
def pytest_genfunc(self, funcspec):
|
||||
assert "arg" in funcspec.funcargnames
|
||||
funcspec.addcall(arg=10)
|
||||
funcspec.addcall(arg=20)
|
||||
""")
|
||||
p = testdir.makepyfile("""
|
||||
def test_myfunc(arg):
|
||||
assert arg == 10
|
||||
""")
|
||||
result = testdir.runpytest("-v", p)
|
||||
assert result.stdout.fnmatch_lines([
|
||||
"*test_myfunc*PASS*", # case for 10
|
||||
"*test_myfunc*FAIL*", # case for 20
|
||||
"*1 failed, 1 passed*"
|
||||
])
|
||||
|
||||
def test_two_functions(self, testdir):
|
||||
p = testdir.makepyfile("""
|
||||
def pytest_genfunc(funcspec):
|
||||
funcspec.addcall(arg1=10)
|
||||
funcspec.addcall(arg1=20)
|
||||
def pytest_generate_tests(metafunc):
|
||||
metafunc.addcall(param=10)
|
||||
metafunc.addcall(param=20)
|
||||
|
||||
def pytest_funcarg__arg1(request):
|
||||
return request.param
|
||||
|
||||
def test_func1(arg1):
|
||||
assert arg1 == 10
|
||||
|
|
@ -239,16 +230,21 @@ class TestGenfuncFunctional:
|
|||
"*1 failed, 3 passed*"
|
||||
])
|
||||
|
||||
def test_genfuncarg_inmodule(self, testdir):
|
||||
def test_generate_plugin_and_module(self, testdir):
|
||||
testdir.makeconftest("""
|
||||
class ConftestPlugin:
|
||||
def pytest_genfunc(self, funcspec):
|
||||
assert "arg1" in funcspec.funcargnames
|
||||
funcspec.addcall(_id="world", arg1=1, arg2=2)
|
||||
def pytest_generate_tests(self, metafunc):
|
||||
assert "arg1" in metafunc.funcargnames
|
||||
metafunc.addcall(id="world", param=(2,100))
|
||||
""")
|
||||
p = testdir.makepyfile("""
|
||||
def pytest_genfunc(funcspec):
|
||||
funcspec.addcall(_id="hello", arg1=10, arg2=10)
|
||||
def pytest_generate_tests(metafunc):
|
||||
metafunc.addcall(param=(1,1), id="hello")
|
||||
|
||||
def pytest_funcarg__arg1(request):
|
||||
return request.param[0]
|
||||
def pytest_funcarg__arg2(request):
|
||||
return request.param[1]
|
||||
|
||||
class TestClass:
|
||||
def test_myfunc(self, arg1, arg2):
|
||||
|
|
|
|||
Loading…
Reference in New Issue