184 lines
7.4 KiB
Plaintext
184 lines
7.4 KiB
Plaintext
|
|
.. _`funcargcompare`:
|
|
|
|
=============================================================
|
|
pytest-2.3: reasoning for the new funcarg and setup functions
|
|
=============================================================
|
|
|
|
**Target audience**: Reading this document requires basic knowledge of
|
|
python testing, xUnit setup methods and the (previous) basic pytest
|
|
funcarg mechanism, see http://pytest.org/2.2.4/funcargs.html
|
|
|
|
.. currentmodule:: _pytest
|
|
|
|
Shortcomings of the previous pytest_funcarg__ mechanism
|
|
===========================================================
|
|
|
|
The pre pytest-2.3 funcarg mechanism calls a factory each time a
|
|
funcarg for a test function is required. If a factory wants to
|
|
re-use a resource across different scopes, it often used
|
|
the ``request.cached_setup()`` helper to manage caching of
|
|
resources. Here is a basic example how we could implement
|
|
a per-session Database object::
|
|
|
|
# content of conftest.py
|
|
class Database:
|
|
def __init__(self):
|
|
print ("database instance created")
|
|
def destroy(self):
|
|
print ("database instance destroyed")
|
|
|
|
def pytest_funcarg__db(request):
|
|
return request.cached_setup(setup=DataBase,
|
|
teardown=lambda db: db.destroy,
|
|
scope="session")
|
|
|
|
There are several limitations and difficulties with this approach:
|
|
|
|
1. Scoping funcarg resource creation is not straight forward, instead one must
|
|
understand the intricate cached_setup() method mechanics.
|
|
|
|
2. parametrizing the "db" resource is not straight forward:
|
|
you need to apply a "parametrize" decorator or implement a
|
|
:py:func:`~hookspec.pytest_generate_tests` hook
|
|
calling :py:func:`~python.Metafunc.parametrize` which
|
|
performs parametrization at the places where the resource
|
|
is used. Moreover, you need to modify the factory to use an
|
|
``extrakey`` parameter containing ``request.param`` to the
|
|
:py:func:`~python.Request.cached_setup` call.
|
|
|
|
3. Multiple parametrized session-scoped resources will be active
|
|
at the same time, making it hard for them to affect global state
|
|
of the application under test.
|
|
|
|
4. there is no way how you can make use of funcarg factories
|
|
in xUnit setup methods.
|
|
|
|
5. A non-parametrized funcarg factory cannot use a parametrized
|
|
funcarg resource if it isn't stated in the test function signature.
|
|
|
|
All of these limitations are addressed with pytest-2.3 and its
|
|
new facilities.
|
|
|
|
Direct scoping of funcarg factories
|
|
--------------------------------------------------------
|
|
|
|
Instead of calling cached_setup(), you can use the :ref:`@pytest.factory <@pytest.factory>` decorator and directly state the scope::
|
|
|
|
@pytest.factory(scope="session")
|
|
def db(testcontext):
|
|
# factory will only be invoked once per session -
|
|
db = DataBase()
|
|
testcontext.addfinalizer(db.destroy) # destroy when session is finished
|
|
return db
|
|
|
|
This factory implementation does not need to call ``cached_setup()`` anymore
|
|
because it will only be invoked once per session. Moreover, the
|
|
``testcontext.addfinalizer()`` registers a finalizer according to the specified
|
|
resource scope on which the factory function is operating.
|
|
|
|
|
|
Direct parametrization of funcarg resource factories
|
|
----------------------------------------------------------
|
|
|
|
Previously, funcarg factories could not directly cause parametrization.
|
|
You needed to specify a ``@parametrize`` decorator on your test function
|
|
or implement a ``pytest_generate_tests`` hook to perform
|
|
parametrization, i.e. calling a test multiple times with different value
|
|
sets. pytest-2.3 introduces a decorator for use on the factory itself::
|
|
|
|
@pytest.factory(params=["mysql", "pg"])
|
|
def pytest_funcarg__db(testcontext):
|
|
... # use testcontext.param
|
|
|
|
Here the factory will be invoked twice (with the respective "mysql"
|
|
and "pg" values set as ``testcontext.param`` attributes) and and all of
|
|
the tests requiring "db" will run twice as well. The "mysql" and
|
|
"pg" values will also be used for reporting the test-invocation variants.
|
|
|
|
This new way of parametrizing funcarg factories should in many cases
|
|
allow to re-use already written factories because effectively
|
|
``testcontext.param`` are already the parametrization attribute for test
|
|
functions/classes were parametrized via
|
|
:py:func:`~_pytest.python.Metafunc.parametrize(indirect=True)` calls.
|
|
|
|
Of course it's perfectly fine to combine parametrization and scoping::
|
|
|
|
@pytest.factory(scope="session", params=["mysql", "pg"])
|
|
def pytest_funcarg__db(testcontext):
|
|
if testcontext.param == "mysql":
|
|
db = MySQL()
|
|
elif testcontext.param == "pg":
|
|
db = PG()
|
|
testcontext.addfinalizer(db.destroy) # destroy when session is finished
|
|
return db
|
|
|
|
This would execute all tests requiring the per-session "db" resource twice,
|
|
receiving the values created by the two respective invocations to the
|
|
factory function.
|
|
|
|
|
|
No ``pytest_funcarg__`` prefix when using @factory decorator
|
|
-------------------------------------------------------------------
|
|
|
|
When using the ``@funcarg`` decorator the name of the function
|
|
denotes the name under which the resource can be accessed as a function
|
|
argument::
|
|
|
|
@pytest.factory()
|
|
def db(testcontext):
|
|
...
|
|
|
|
The name under which the funcarg resource can be requested is ``db``.
|
|
|
|
You can still use the "old" non-decorator way of specifying funcarg factories
|
|
aka::
|
|
|
|
def pytest_funcarg__db(request):
|
|
...
|
|
|
|
It is recommended to use the resource decorator, however.
|
|
|
|
|
|
solving per-session setup / the new @setup marker
|
|
--------------------------------------------------------------
|
|
|
|
pytest for a long time offered a pytest_configure and a pytest_sessionstart
|
|
hook which are often used to setup global resources. This suffers from
|
|
several problems:
|
|
|
|
1. in distributed testing the master process would setup test resources
|
|
that are never needed because it only co-ordinates the test run
|
|
activities of the slave processes.
|
|
|
|
2. if you only perform a collection (with "--collectonly")
|
|
resource-setup will still be executed.
|
|
|
|
3. If a pytest_sessionstart is contained in some subdirectories
|
|
conftest.py file, it will not be called. This stems from the
|
|
fact that this hook is actually used for reporting, in particular
|
|
the test-header with platform/custom information.
|
|
|
|
Moreover, it was not easy to define a scoped setup from plugins or
|
|
conftest files other than to implement a ``pytest_runtest_setup()`` hook
|
|
and caring for scoping/caching yourself. And it's virtually impossible
|
|
to do this with parametrization as ``pytest_runtest_setup()`` is called
|
|
during test execution and parametrization happens at collection time.
|
|
|
|
It follows that pytest_configure/session/runtest_setup are often not
|
|
appropriate for implementing common fixture needs. Therefore,
|
|
pytest-2.3 introduces a new :ref:`@pytest.setup <setup>` marker
|
|
for setup functions and it accepts an optional "scope" parameter.
|
|
|
|
See :ref:`setup` for more explanation and examples.
|
|
|
|
funcarg and setup discovery now happens at collection time
|
|
---------------------------------------------------------------------
|
|
|
|
pytest-2.3 takes care to discover funcarg factories and @setup methods
|
|
at collection time. This is more efficient especially for large test suites.
|
|
Moreover, a call to "py.test --collectonly" should be able to in the future
|
|
show a lot of setup-information and thus presents a nice method to get an
|
|
overview of resource management in your project.
|
|
|