implement funcargs according to docs, streamline docs

--HG--
branch : trunk

doc/test/funcargs.txt

@@ -11,7 +11,7 @@ of making it easy to:
 * manage test value setup and teardown depending on 
   command line options or configuration
 * parametrize multiple runs of the same test functions 
-* present useful debug info if something goes wrong
+* present useful debug info if setup goes wrong
 
 Using funcargs, test functions become more expressive, 
 more "templaty" and more test-aspect oriented.  In fact, 
@@ -35,8 +35,8 @@ funcarg providers: setting up test function arguments
 
 Test functions can specify one ore more arguments ("funcargs") 
 and a test module or plugin can define functions that provide
-the function argument.  Let's look at a self-contained example 
+the function argument.  Let's look at a simple self-contained 
-that you can put into a test module: 
+example that you can put into a test module: 
 
 .. sourcecode:: python
 
@@ -48,28 +48,30 @@ that you can put into a test module:
 
 Here is what happens: 
 
-1. **lookup funcarg provider**: The ``test_function`` needs an value for 
+1. **lookup funcarg provider**: For executing ``test_function(myfuncarg)`` 
-   ``myfuncarg`` to run.  The provider is found by its special 
+   a value is needed.  A value provider is found by looking for a
-   name, ``pytest_funcarg__`` followed by the function 
+   function with a special name of ``pytest_funcarg__${ARGNAME}``.  
-   argument argument name.  If a provider cannot be found, 
-   a list of all available function arguments is presented. 
 
 2. **setup funcarg value**: ``pytest_funcarg__myfuncarg(request)`` is 
-   called to setup the value for ``myfuncarg``.   
+   called to setup and return the value for ``myfuncarg``.   
 
 3. **execute test** ``test_function(42)`` call is executed.  
-   If the test fails one can see the original provided 
+
-   value in the traceback at the top. 
+Note that if a provider cannot be found a list of
+available function arguments will be provided. 
+
+For providers that makes use of the `request object`_ 
+please look into the `tutorial examples`_. 
 
 .. _`request object`: 
 
 funcarg request objects
 ------------------------------------------
 
-Request objects are passed to funcarg providers.  Request objects 
+Request objects are passed to funcarg providers.  They 
 encapsulate a request for a function argument for a
-specific test function.  Request objects allow providers to access 
+specific test function.  Request objects allow providers 
-test configuration and test context: 
+to access test configuration and test context: 
 
 ``request.argname``: name of the requested function argument 
 
@@ -81,7 +83,7 @@ test configuration and test context:
 
 ``request.config``: access to command line opts and general config
 
-``request.param``: if exists is the argument passed by a `parametrizing test generator`_ 
+``request.param``: if exists was passed by a `parametrizing test generator`_ 
 
 
 cleanup after test function execution
@@ -90,9 +92,9 @@ cleanup after test function execution
 Request objects allow to **register a finalizer method** which is 
 called after a test function has finished running. 
 This is useful for tearing down or cleaning up
-test state. Here is a basic example for providing
+test state related to a function argument.  Here is a basic 
-a ``myfile`` object that will be closed upon test
+example for providing a ``myfile`` object that will be 
-function finish:
+closed upon test function finish:
 
 .. sourcecode:: python
 
@@ -152,7 +154,7 @@ Here is what happens in detail:
    function.  The `metafunc object`_ has context information. 
    ``metafunc.addcall(param=i)`` schedules a new test call
    such that function argument providers will see an additional 
-   ``arg`` attribute on their request object. 
+   ``param`` attribute on their request object. 
 
 2. **setup funcarg values**: the ``pytest_funcarg__arg1(request)`` provider is called 
    10 times with ten different request objects all pointing to 
@@ -208,6 +210,8 @@ even happen in a different process.  Therefore one should
 defer setup of heavyweight objects to funcarg providers.*
 
 
+.. _`tutorial examples`: 
+
 Funcarg Tutorial Examples
 =======================================
 
@@ -291,13 +295,7 @@ local plugin that adds a command line option to ``py.test`` invocations:
 
 .. sourcecode:: python
 
-    class ConftestPlugin:
+    # ./conftest.py 
-        def pytest_addoption(self, parser):
-            parser.addoption("--ssh", action="store", default=None,
-                help="specify ssh host to run tests with")
-
-        pytest_funcarg__mysetup = MySetupFuncarg
-
     class MySetupFuncarg:
         def __init__(self, request):
             self.request = request 
@@ -307,10 +305,19 @@ local plugin that adds a command line option to ``py.test`` invocations:
                 py.test.skip("specify ssh host with --ssh to run this test")
             return py.execnet.SshGateway(host)
 
+    class ConftestPlugin:
+        def pytest_addoption(self, parser):
+            parser.addoption("--ssh", action="store", default=None,
+                help="specify ssh host to run tests with")
+
+        # alias the above class as the "mysetup" provider 
+        pytest_funcarg__mysetup = MySetupFuncarg
+
 Now any test functions can use the ``mysetup.getsshconnection()`` method like this:
 
 .. sourcecode:: python
 
+    # ./test_function.py 
     class TestClass:
         def test_function(self, mysetup):
             conn = mysetup.getsshconnection()
@@ -326,6 +333,7 @@ example: specifying and selecting acceptance tests
 
 .. sourcecode:: python
 
+    # ./conftest.py 
     class ConftestPlugin:
         def pytest_option(self, parser):
             group = parser.getgroup("myproject")

py/test/funcargs.py

@@ -10,14 +10,10 @@ def getfuncargnames(function):
     
 def fillfuncargs(function):
     """ fill missing funcargs. """ 
-    if function._args:
+    argnames = getfuncargnames(function.obj)
-        # functions yielded from a generator: we don't want
+    if argnames:
-        # to support that because we want to go here anyway: 
+        assert not function._args, "yielded functions cannot have funcargs" 
-        # http://bitbucket.org/hpk42/py-trunk/issue/2/next-generation-generative-tests
+        for argname in argnames:
-        pass
-    else:
-        # standard Python Test function/method case  
-        for argname in getfuncargnames(function.obj):
             if argname not in function.funcargs:
                 request = FuncargRequest(pyfuncitem=function, argname=argname) 
                 try:
@@ -25,12 +21,15 @@ def fillfuncargs(function):
                 except request.Error:
                     request._raiselookupfailed()
 
-class CallSpec:
-    def __init__(self, id, funcargs):
-        self.id = id
-        self.funcargs = funcargs 
 
-class FuncSpecs:
+_notexists = object()
+class CallSpec:
+    def __init__(self, id, param):
+        self.id = id
+        if param is not _notexists:
+            self.param = param 
+
+class Metafunc:
     def __init__(self, function, config=None, cls=None, module=None):
         self.config = config
         self.module = module 
@@ -41,20 +40,14 @@ class FuncSpecs:
         self._calls = []
         self._ids = py.builtin.set()
 
-    def addcall(self, _id=None, **kwargs):
+    def addcall(self, id=None, param=_notexists):
-        for argname in kwargs:
+        if id is None:
-            if argname[0] == "_":
+            id = len(self._calls)
-                raise TypeError("argument %r is not a valid keyword." % argname)
+        id = str(id)
-            if argname not in self.funcargnames:
+        if id in self._ids:
-                raise ValueError("function %r has no funcarg %r" %(
+            raise ValueError("duplicate id %r" % id)
-                        self.function, argname))
+        self._ids.add(id)
-        if _id is None:
+        self._calls.append(CallSpec(id, param))
-            _id = len(self._calls)
-        _id = str(_id)
-        if _id in self._ids:
-            raise ValueError("duplicate id %r" % _id)
-        self._ids.add(_id)
-        self._calls.append(CallSpec(_id, kwargs))
 
 class FunctionCollector(py.test.collect.Collector):
     def __init__(self, name, parent, calls):
@@ -66,7 +59,7 @@ class FunctionCollector(py.test.collect.Collector):
         l = []
         for call in self.calls:
             function = self.parent.Function(name="%s[%s]" %(self.name, call.id),
-                parent=self, funcargs=call.funcargs, callobj=self.obj)
+                parent=self, requestparam=call.param, callobj=self.obj)
             l.append(function)
         return l 
 
@@ -75,7 +68,7 @@ class FuncargRequest:
 
     class Error(LookupError):
         """ error on performing funcarg request. """ 
-        
+
     def __init__(self, pyfuncitem, argname):
         self._pyfuncitem = pyfuncitem
         self.argname = argname 
@@ -84,6 +77,8 @@ class FuncargRequest:
         self.cls = getattr(self.function, 'im_class', None)
         self.config = pyfuncitem.config
         self.fspath = pyfuncitem.fspath
+        if hasattr(pyfuncitem, '_requestparam'):
+            self.param = pyfuncitem._requestparam 
         self._plugins = self.config.pluginmanager.getplugins()
         self._plugins.append(self.module)
         self._provider = self.config.pluginmanager.listattr(
@@ -91,9 +86,6 @@ class FuncargRequest:
             attrname=self._argprefix + str(argname)
         )
 
-    def __repr__(self):
-        return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
-
     def call_next_provider(self):
         if not self._provider:
             raise self.Error("no provider methods left")
@@ -103,6 +95,9 @@ class FuncargRequest:
     def addfinalizer(self, finalizer):
         self._pyfuncitem.addfinalizer(finalizer)
 
+    def __repr__(self):
+        return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
+
     def _raiselookupfailed(self):
         available = []
         for plugin in self._plugins:

py/test/plugin/api.py

@@ -53,7 +53,7 @@ class PluginHooks:
         """ return custom item/collector for a python object in a module, or None.  """
     pytest_pycollect_obj.firstresult = True
 
-    def pytest_genfunc(self, funcspec):
+    def pytest_generate_tests(self, metafunc):
         """ generate (multiple) parametrized calls to a test function."""
 
     def pytest_collectstart(self, collector):

py/test/pycollect.py

@@ -151,13 +151,13 @@ class PyCollectorMixin(PyobjMixin, py.test.collect.Collector):
         # to work to get at the class 
         clscol = self._getparent(Class)
         cls = clscol and clscol.obj or None
-        funcspec = funcargs.FuncSpecs(funcobj, config=self.config, cls=cls, module=module)
+        metafunc = funcargs.Metafunc(funcobj, config=self.config, cls=cls, module=module)
-        gentesthook = self.config.hook.pytest_genfunc.clone(extralookup=module)
+        gentesthook = self.config.hook.pytest_generate_tests.clone(extralookup=module)
-        gentesthook(funcspec=funcspec)
+        gentesthook(metafunc=metafunc)
-        if not funcspec._calls:
+        if not metafunc._calls:
             return self.Function(name, parent=self)
         return funcargs.FunctionCollector(name=name, 
-            parent=self, calls=funcspec._calls)
+            parent=self, calls=metafunc._calls)
         
 class Module(py.test.collect.File, PyCollectorMixin):
     def _getobj(self):
@@ -325,13 +325,15 @@ class Function(FunctionMixin, py.test.collect.Item):
     """ a Function Item is responsible for setting up  
         and executing a Python callable test object.
     """
-    def __init__(self, name, parent=None, config=None, args=(), funcargs=None, callobj=_dummy):
+    def __init__(self, name, parent=None, config=None, args=(), 
+                 requestparam=_dummy, callobj=_dummy):
         super(Function, self).__init__(name, parent, config=config) 
         self._finalizers = []
-        self._args = args
+        self._args = args 
-        if funcargs is None:
+        if not args: # yielded functions (deprecated) have positional args 
-            funcargs = {}
+            self.funcargs = {}
-        self.funcargs = funcargs 
+            if requestparam is not _dummy:
+                self._requestparam = requestparam
         if callobj is not _dummy: 
             self._obj = callobj 
 
@@ -352,12 +354,14 @@ class Function(FunctionMixin, py.test.collect.Item):
 
     def runtest(self):
         """ execute the given test function. """
-        self.config.hook.pytest_pyfunc_call(pyfuncitem=self, 
+        kwargs = getattr(self, 'funcargs', {})
-            args=self._args, kwargs=self.funcargs)
+        self.config.hook.pytest_pyfunc_call(
+            pyfuncitem=self, args=self._args, kwargs=kwargs)
 
     def setup(self):
         super(Function, self).setup()
-        funcargs.fillfuncargs(self)
+        if hasattr(self, 'funcargs'): 
+            funcargs.fillfuncargs(self)
 
     def __eq__(self, other):
         try:

py/test/testing/test_funcargs.py

@@ -131,100 +131,91 @@ class TestRequest:
         req = funcargs.FuncargRequest(item, "xxx")
         assert req.fspath == modcol.fspath 
 
-class TestFuncSpecs:
+class TestMetafunc:
     def test_no_funcargs(self, testdir):
         def function(): pass
-        funcspec = funcargs.FuncSpecs(function)
+        metafunc = funcargs.Metafunc(function)
-        assert not funcspec.funcargnames
+        assert not metafunc.funcargnames
 
     def test_function_basic(self):
         def func(arg1, arg2="qwe"): pass
-        funcspec = funcargs.FuncSpecs(func)
+        metafunc = funcargs.Metafunc(func)
-        assert len(funcspec.funcargnames) == 1
+        assert len(metafunc.funcargnames) == 1
-        assert 'arg1' in funcspec.funcargnames
+        assert 'arg1' in metafunc.funcargnames
-        assert funcspec.function is func 
+        assert metafunc.function is func 
-        assert funcspec.cls is None
+        assert metafunc.cls is None
 
-    def test_addcall_with_id(self):
+    def test_addcall_no_args(self):
         def func(arg1): pass
-        funcspec = funcargs.FuncSpecs(func)
+        metafunc = funcargs.Metafunc(func)
-        py.test.raises(TypeError, """
+        metafunc.addcall()
-            funcspec.addcall(_xyz=10)
+        assert len(metafunc._calls) == 1
-        """)
+        call = metafunc._calls[0]
-        funcspec.addcall(_id="hello", arg1=100)
+        assert call.id == "0"
-        py.test.raises(ValueError, "funcspec.addcall(_id='hello', arg1=100)")
+        assert not hasattr(call, 'param')
-        call = funcspec._calls[0]
-        assert call.id == "hello"
-        assert call.funcargs == {'arg1': 100}
 
-    def test_addcall_basic(self):
+    def test_addcall_id(self):
         def func(arg1): pass
-        funcspec = funcargs.FuncSpecs(func)
+        metafunc = funcargs.Metafunc(func)
-        py.test.raises(ValueError, """
+        metafunc.addcall(id=1)
-            funcspec.addcall(notexists=100)
+        py.test.raises(ValueError, "metafunc.addcall(id=1)")
-        """)
+        py.test.raises(ValueError, "metafunc.addcall(id='1')")
-        funcspec.addcall(arg1=100)
+        metafunc.addcall(id=2)
-        assert len(funcspec._calls) == 1
+        assert len(metafunc._calls) == 2
-        assert funcspec._calls[0].funcargs == {'arg1': 100}
+        assert metafunc._calls[0].id == "1"
+        assert metafunc._calls[1].id == "2"
 
-    def test_addcall_two(self):
+    def test_addcall_param(self):
         def func(arg1): pass
-        funcspec = funcargs.FuncSpecs(func)
+        metafunc = funcargs.Metafunc(func)
-        funcspec.addcall(arg1=100)
+        class obj: pass 
-        funcspec.addcall(arg1=101)
+        metafunc.addcall(param=obj) 
-        assert len(funcspec._calls) == 2
+        metafunc.addcall(param=obj) 
-        assert funcspec._calls[0].funcargs == {'arg1': 100}
+        metafunc.addcall(param=1) 
-        assert funcspec._calls[1].funcargs == {'arg1': 101}
+        assert len(metafunc._calls) == 3
+        assert metafunc._calls[0].param == obj 
+        assert metafunc._calls[1].param == obj 
+        assert metafunc._calls[2].param == 1
+
 
 class TestGenfuncFunctional:
     def test_attributes(self, testdir):
         p = testdir.makepyfile("""
+            # assumes that generate/provide runs in the same process
             import py
-            def pytest_genfunc(funcspec):
+            def pytest_generate_tests(metafunc):
-                funcspec.addcall(funcspec=funcspec)
+                metafunc.addcall(param=metafunc) 
+
+            def pytest_funcarg__metafunc(request):
+                return request.param 
+
+            def test_function(metafunc):
+                assert metafunc.config == py.test.config
+                assert metafunc.module.__name__ == __name__
+                assert metafunc.function == test_function
+                assert metafunc.cls is None
 
-            def test_function(funcspec):
-                assert funcspec.config == py.test.config
-                assert funcspec.module.__name__ == __name__
-                assert funcspec.function == test_function
-                assert funcspec.cls is None
             class TestClass:
-                def test_method(self, funcspec):
+                def test_method(self, metafunc):
-                    assert funcspec.config == py.test.config
+                    assert metafunc.config == py.test.config
-                    assert funcspec.module.__name__ == __name__
+                    assert metafunc.module.__name__ == __name__
-                    # XXX actually have the unbound test function here?
+                    # XXX actually have an unbound test function here?
-                    assert funcspec.function == TestClass.test_method.im_func
+                    assert metafunc.function == TestClass.test_method.im_func
-                    assert funcspec.cls == TestClass
+                    assert metafunc.cls == TestClass
         """)
         result = testdir.runpytest(p, "-v")
         result.stdout.fnmatch_lines([
             "*2 passed in*",
         ])
 
-    def test_arg_twice(self, testdir):
-        testdir.makeconftest("""
-            class ConftestPlugin:
-                def pytest_genfunc(self, funcspec):
-                    assert "arg" in funcspec.funcargnames 
-                    funcspec.addcall(arg=10)
-                    funcspec.addcall(arg=20)
-        """)
-        p = testdir.makepyfile("""
-            def test_myfunc(arg):
-                assert arg == 10
-        """)
-        result = testdir.runpytest("-v", p)
-        assert result.stdout.fnmatch_lines([
-            "*test_myfunc*PASS*", # case for 10
-            "*test_myfunc*FAIL*", # case for 20
-            "*1 failed, 1 passed*"
-        ])
-
     def test_two_functions(self, testdir):
         p = testdir.makepyfile("""
-            def pytest_genfunc(funcspec):
+            def pytest_generate_tests(metafunc):
-                funcspec.addcall(arg1=10)
+                metafunc.addcall(param=10)
-                funcspec.addcall(arg1=20)
+                metafunc.addcall(param=20) 
+
+            def pytest_funcarg__arg1(request):
+                return request.param
 
             def test_func1(arg1):
                 assert arg1 == 10
@@ -239,16 +230,21 @@ class TestGenfuncFunctional:
             "*1 failed, 3 passed*"
         ])
 
-    def test_genfuncarg_inmodule(self, testdir):
+    def test_generate_plugin_and_module(self, testdir):
         testdir.makeconftest("""
             class ConftestPlugin:
-                def pytest_genfunc(self, funcspec):
+                def pytest_generate_tests(self, metafunc):
-                    assert "arg1" in funcspec.funcargnames 
+                    assert "arg1" in metafunc.funcargnames 
-                    funcspec.addcall(_id="world", arg1=1, arg2=2)
+                    metafunc.addcall(id="world", param=(2,100))
         """)
         p = testdir.makepyfile("""
-            def pytest_genfunc(funcspec):
+            def pytest_generate_tests(metafunc):
-                funcspec.addcall(_id="hello", arg1=10, arg2=10)
+                metafunc.addcall(param=(1,1), id="hello")
+
+            def pytest_funcarg__arg1(request):
+                return request.param[0]
+            def pytest_funcarg__arg2(request):
+                return request.param[1]
 
             class TestClass:
                 def test_myfunc(self, arg1, arg2):