test_ok2/doc/en/funcarg_compare.rst

231 lines
8.6 KiB
ReStructuredText

:orphan:
.. _`funcargcompare`:
pytest-2.3: reasoning for fixture/funcarg evolution
=============================================================
**Target audience**: Reading this document requires basic knowledge of
python testing, xUnit setup methods and the (previous) basic pytest
funcarg mechanism, see :ref:`historical funcargs and pytest.funcargs`.
If you are new to pytest, then you can simply ignore this
section and read the other sections.
.. currentmodule:: _pytest
Shortcomings of the previous ``pytest_funcarg__`` mechanism
--------------------------------------------------------------
The pre pytest-2.3 funcarg mechanism calls a factory each time a
funcarg for a test function is required. If a factory wants to
re-use a resource across different scopes, it often used
the ``request.cached_setup()`` helper to manage caching of
resources. Here is a basic example how we could implement
a per-session Database object:
.. code-block:: python
# content of conftest.py
class Database:
def __init__(self):
print("database instance created")
def destroy(self):
print("database instance destroyed")
def pytest_funcarg__db(request):
return request.cached_setup(
setup=DataBase, teardown=lambda db: db.destroy, scope="session"
)
There are several limitations and difficulties with this approach:
1. Scoping funcarg resource creation is not straight forward, instead one must
understand the intricate cached_setup() method mechanics.
2. parametrizing the "db" resource is not straight forward:
you need to apply a "parametrize" decorator or implement a
:py:func:`~hookspec.pytest_generate_tests` hook
calling :py:func:`~pytest.Metafunc.parametrize` which
performs parametrization at the places where the resource
is used. Moreover, you need to modify the factory to use an
``extrakey`` parameter containing ``request.param`` to the
``Request.cached_setup`` call.
3. Multiple parametrized session-scoped resources will be active
at the same time, making it hard for them to affect global state
of the application under test.
4. there is no way how you can make use of funcarg factories
in xUnit setup methods.
5. A non-parametrized fixture function cannot use a parametrized
funcarg resource if it isn't stated in the test function signature.
All of these limitations are addressed with pytest-2.3 and its
improved :ref:`fixture mechanism <fixture>`.
Direct scoping of fixture/funcarg factories
--------------------------------------------------------
Instead of calling cached_setup() with a cache scope, you can use the
:ref:`@pytest.fixture <pytest.fixture>` decorator and directly state
the scope:
.. code-block:: python
@pytest.fixture(scope="session")
def db(request):
# factory will only be invoked once per session -
db = DataBase()
request.addfinalizer(db.destroy) # destroy when session is finished
return db
This factory implementation does not need to call ``cached_setup()`` anymore
because it will only be invoked once per session. Moreover, the
``request.addfinalizer()`` registers a finalizer according to the specified
resource scope on which the factory function is operating.
Direct parametrization of funcarg resource factories
----------------------------------------------------------
Previously, funcarg factories could not directly cause parametrization.
You needed to specify a ``@parametrize`` decorator on your test function
or implement a ``pytest_generate_tests`` hook to perform
parametrization, i.e. calling a test multiple times with different value
sets. pytest-2.3 introduces a decorator for use on the factory itself:
.. code-block:: python
@pytest.fixture(params=["mysql", "pg"])
def db(request):
... # use request.param
Here the factory will be invoked twice (with the respective "mysql"
and "pg" values set as ``request.param`` attributes) and all of
the tests requiring "db" will run twice as well. The "mysql" and
"pg" values will also be used for reporting the test-invocation variants.
This new way of parametrizing funcarg factories should in many cases
allow to re-use already written factories because effectively
``request.param`` was already used when test functions/classes were
parametrized via
:py:func:`metafunc.parametrize(indirect=True) <pytest.Metafunc.parametrize>` calls.
Of course it's perfectly fine to combine parametrization and scoping:
.. code-block:: python
@pytest.fixture(scope="session", params=["mysql", "pg"])
def db(request):
if request.param == "mysql":
db = MySQL()
elif request.param == "pg":
db = PG()
request.addfinalizer(db.destroy) # destroy when session is finished
return db
This would execute all tests requiring the per-session "db" resource twice,
receiving the values created by the two respective invocations to the
factory function.
No ``pytest_funcarg__`` prefix when using @fixture decorator
-------------------------------------------------------------------
When using the ``@fixture`` decorator the name of the function
denotes the name under which the resource can be accessed as a function
argument:
.. code-block:: python
@pytest.fixture()
def db(request):
...
The name under which the funcarg resource can be requested is ``db``.
You can still use the "old" non-decorator way of specifying funcarg factories
aka:
.. code-block:: python
def pytest_funcarg__db(request):
...
But it is then not possible to define scoping and parametrization.
It is thus recommended to use the factory decorator.
solving per-session setup / autouse fixtures
--------------------------------------------------------------
pytest for a long time offered a pytest_configure and a pytest_sessionstart
hook which are often used to setup global resources. This suffers from
several problems:
1. in distributed testing the managing process would setup test resources
that are never needed because it only co-ordinates the test run
activities of the worker processes.
2. if you only perform a collection (with "--collect-only")
resource-setup will still be executed.
3. If a pytest_sessionstart is contained in some subdirectories
conftest.py file, it will not be called. This stems from the
fact that this hook is actually used for reporting, in particular
the test-header with platform/custom information.
Moreover, it was not easy to define a scoped setup from plugins or
conftest files other than to implement a ``pytest_runtest_setup()`` hook
and caring for scoping/caching yourself. And it's virtually impossible
to do this with parametrization as ``pytest_runtest_setup()`` is called
during test execution and parametrization happens at collection time.
It follows that pytest_configure/session/runtest_setup are often not
appropriate for implementing common fixture needs. Therefore,
pytest-2.3 introduces :ref:`autouse fixtures` which fully
integrate with the generic :ref:`fixture mechanism <fixture>`
and obsolete many prior uses of pytest hooks.
funcargs/fixture discovery now happens at collection time
---------------------------------------------------------------------
Since pytest-2.3, discovery of fixture/funcarg factories are taken care of
at collection time. This is more efficient especially for large test suites.
Moreover, a call to "pytest --collect-only" should be able to in the future
show a lot of setup-information and thus presents a nice method to get an
overview of fixture management in your project.
.. _`compatibility notes`:
.. _`funcargscompat`:
Conclusion and compatibility notes
---------------------------------------------------------
**funcargs** were originally introduced to pytest-2.0. In pytest-2.3
the mechanism was extended and refined and is now described as
fixtures:
* previously funcarg factories were specified with a special
``pytest_funcarg__NAME`` prefix instead of using the
``@pytest.fixture`` decorator.
* Factories received a ``request`` object which managed caching through
``request.cached_setup()`` calls and allowed using other funcargs via
``request.getfuncargvalue()`` calls. These intricate APIs made it hard
to do proper parametrization and implement resource caching. The
new :py:func:`pytest.fixture` decorator allows to declare the scope
and let pytest figure things out for you.
* if you used parametrization and funcarg factories which made use of
``request.cached_setup()`` it is recommended to invest a few minutes
and simplify your fixture function code to use the :ref:`@pytest.fixture`
decorator instead. This will also allow to take advantage of
the automatic per-resource grouping of tests.