test_ok2/_pytest/impl

Sorting per-resource
-----------------------------

for any given set of items:

- collect items per session-scoped parametrized funcarg
- re-order until items no parametrizations are mixed

  examples:

        test()
        test1(s1)
        test1(s2)     
        test2()
        test3(s1)
        test3(s2)

  gets sorted to:

        test()
        test2()
        test1(s1)
        test3(s1)
        test1(s2)
        test3(s2)
 

the new @setup functions 
--------------------------------------

Consider a given @setup-marked function::

    @pytest.mark.setup(maxscope=SCOPE)
    def mysetup(request, arg1, arg2, ...)
        ...
        request.addfinalizer(fin)
        ...

then FUNCARGSET denotes the set of (arg1, arg2, ...) funcargs and
all of its dependent funcargs.  The mysetup function will execute
for any matching test item once per scope.  

The scope is determined as the minimum scope of all scopes of the args
in FUNCARGSET and the given "maxscope". 

If mysetup has been called and no finalizers have been called it is
called "active".

Furthermore the following rules apply:

- if an arg value in FUNCARGSET is about to be torn down, the 
  mysetup-registered finalizers will execute as well.

- There will never be two active mysetup invocations.

Example 1, session scope::

    @pytest.mark.funcarg(scope="session", params=[1,2])
    def db(request):
        request.addfinalizer(db_finalize)

    @pytest.mark.setup
    def mysetup(request, db):
        request.addfinalizer(mysetup_finalize)
        ...

And a given test module:

    def test_something():
        ...
    def test_otherthing():
        pass

Here is what happens::

    db(request) executes with request.param == 1
        mysetup(request, db) executes
            test_something() executes
            test_otherthing() executes
            mysetup_finalize() executes
    db_finalize() executes
    db(request) executes with request.param == 2
        mysetup(request, db) executes
            test_something() executes
            test_otherthing() executes
        mysetup_finalize() executes
    db_finalize() executes

Example 2, session/function scope::

    @pytest.mark.funcarg(scope="session", params=[1,2])
    def db(request):
        request.addfinalizer(db_finalize)

    @pytest.mark.setup(scope="function")
    def mysetup(request, db):
        ...
        request.addfinalizer(mysetup_finalize)
        ...

And a given test module:

    def test_something():
        ...
    def test_otherthing():
        pass

Here is what happens::

    db(request) executes with request.param == 1
        mysetup(request, db) executes
            test_something() executes
        mysetup_finalize() executes
        mysetup(request, db) executes
            test_otherthing() executes
        mysetup_finalize() executes
    db_finalize() executes
    db(request) executes with request.param == 2
        mysetup(request, db) executes
            test_something() executes
        mysetup_finalize() executes
        mysetup(request, db) executes
            test_otherthing() executes
        mysetup_finalize() executes
    db_finalize() executes


Example 3 - funcargs session-mix
----------------------------------------

Similar with funcargs, an example::

    @pytest.mark.funcarg(scope="session", params=[1,2])
    def db(request):
        request.addfinalizer(db_finalize)

    @pytest.mark.funcarg(scope="function")
    def table(request, db):
        ...
        request.addfinalizer(table_finalize)
        ...

And a given test module:

    def test_something(table):
        ...
    def test_otherthing(table):
        pass
    def test_thirdthing():
        pass

Here is what happens::

    db(request) executes with param == 1
        table(request, db)
            test_something(table)
        table_finalize()
        table(request, db)
            test_otherthing(table)
        table_finalize()
    db_finalize
    db(request) executes with param == 2
        table(request, db)
            test_something(table)
        table_finalize()
        table(request, db)
            test_otherthing(table)
        table_finalize()
    db_finalize
    test_thirdthing()
    
Data structures
--------------------

pytest internally maintains a dict of active funcargs with cache, param,
finalizer, (scopeitem?) information:

    active_funcargs = dict()

if a parametrized "db" is activated:
    
    active_funcargs["db"] = FuncargInfo(dbvalue, paramindex, 
                                        FuncargFinalize(...), scopeitem)

if a test is torn down and the next test requires a differently 
parametrized "db":

    for argname in item.callspec.params:
        if argname in active_funcargs:
            funcarginfo = active_funcargs[argname]
            if funcarginfo.param != item.callspec.params[argname]:
                funcarginfo.callfinalizer()
                del node2funcarg[funcarginfo.scopeitem]
                del active_funcargs[argname]
    nodes_to_be_torn_down = ...
    for node in nodes_to_be_torn_down:
        if node in node2funcarg:
            argname = node2funcarg[node]
            active_funcargs[argname].callfinalizer()
            del node2funcarg[node]
            del active_funcargs[argname]

if a test is setup requiring a "db" funcarg:

    if "db" in active_funcargs:
        return active_funcargs["db"][0]
    funcarginfo = setup_funcarg()
    active_funcargs["db"] = funcarginfo
    node2funcarg[funcarginfo.scopeitem] = "db"

Implementation plan for resources
------------------------------------------

1. Revert FuncargRequest to the old form, unmerge item/request
   (done)
2. make funcarg factories be discovered at collection time
3. Introduce funcarg marker
4. Introduce funcarg scope parameter
5. Introduce funcarg parametrize parameter
6. make setup functions be discovered at collection time
7. (Introduce a pytest_fixture_protocol/setup_funcargs hook)

methods and data structures
--------------------------------

A FuncarcManager holds all information about funcarg definitions
including parametrization and scope definitions.  It implements
a pytest_generate_tests hook which performs parametrization as appropriate.

as a simple example, let's consider a tree where a test function requires
a "abc" funcarg and its factory defines it as parametrized and scoped
for Modules.  When collections hits the function item, it creates
the metafunc object, and calls funcargdb.pytest_generate_tests(metafunc)
which looks up available funcarg factories and their scope and parametrization.
This information is equivalent to what can be provided today directly
at the function site and it should thus be relatively straight forward
to implement the additional way of defining parametrization/scoping.

conftest loading:
    each funcarg-factory will populate the session.funcargmanager

When a test item is collected, it grows a dictionary 
(funcargname2factorycalllist).  A factory lookup is performed 
for each required funcarg.  The resulting factory call is stored 
with the item.  If a function is parametrized multiple items are 
created with respective factory calls. Else if a factory is parametrized
multiple items and calls to the factory function are created as well.

At setup time, an item populates a funcargs mapping, mapping names
to values.  If a value is funcarg factories are queried for a given item
test functions and setup functions are put in a class
which looks up required funcarg factories.