split plugin documentation into "using" and "writing plugins",
referencing each other. Also add tryfirst/trylast examples. --HG-- branch : more_plugin
This commit is contained in:
holger krekel
parent
d2a5c7f99b
commit
ea50ef1588
|
|
@ -33,6 +33,9 @@
|
|||
now deprecated use of ``pytest.mark`` which is meant to
|
||||
contain markers for test functions only.
|
||||
|
||||
- write/refine docs for "writing plugins" which now have their
|
||||
own page and are separate from the "using/installing plugins`` page.
|
||||
|
||||
|
||||
2.7.1.dev (compared to 2.7.0)
|
||||
-----------------------------
|
||||
|
|
|
|||
|
|
@ -56,6 +56,7 @@ pytest: helps you write better programs
|
|||
- all collection, reporting, running aspects are delegated to hook functions
|
||||
- customizations can be per-directory, per-project or per PyPI released plugin
|
||||
- it is easy to add command line options or customize existing behaviour
|
||||
- :ref:`easy to write your own plugins <writing-plugins>`
|
||||
|
||||
|
||||
.. _`easy`: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html
|
||||
|
|
|
|||
|
|
@ -1,64 +1,14 @@
|
|||
.. _plugins:
|
||||
|
||||
Working with plugins and conftest files
|
||||
=======================================
|
||||
|
||||
``pytest`` implements all aspects of configuration, collection, running and reporting by calling `well specified hooks`_. Virtually any Python module can be registered as a plugin. It can implement any number of hook functions (usually two or three) which all have a ``pytest_`` prefix, making hook functions easy to distinguish and find. There are three basic location types:
|
||||
|
||||
* `builtin plugins`_: loaded from pytest's internal ``_pytest`` directory.
|
||||
* `external plugins`_: modules discovered through `setuptools entry points`_
|
||||
* `conftest.py plugins`_: modules auto-discovered in test directories
|
||||
|
||||
.. _`pytest/plugin`: http://bitbucket.org/pytest-dev/pytest/src/tip/pytest/plugin/
|
||||
.. _`conftest.py plugins`:
|
||||
.. _`conftest.py`:
|
||||
.. _`localplugin`:
|
||||
.. _`conftest`:
|
||||
|
||||
conftest.py: local per-directory plugins
|
||||
----------------------------------------
|
||||
|
||||
local ``conftest.py`` plugins contain directory-specific hook
|
||||
implementations. Session and test running activities will
|
||||
invoke all hooks defined in ``conftest.py`` files closer to the
|
||||
root of the filesystem. Example: Assume the following layout
|
||||
and content of files::
|
||||
|
||||
a/conftest.py:
|
||||
def pytest_runtest_setup(item):
|
||||
# called for running each test in 'a' directory
|
||||
print ("setting up", item)
|
||||
|
||||
a/test_sub.py:
|
||||
def test_sub():
|
||||
pass
|
||||
|
||||
test_flat.py:
|
||||
def test_flat():
|
||||
pass
|
||||
|
||||
Here is how you might run it::
|
||||
|
||||
py.test test_flat.py # will not show "setting up"
|
||||
py.test a/test_sub.py # will show "setting up"
|
||||
|
||||
.. Note::
|
||||
If you have ``conftest.py`` files which do not reside in a
|
||||
python package directory (i.e. one containing an ``__init__.py``) then
|
||||
"import conftest" can be ambiguous because there might be other
|
||||
``conftest.py`` files as well on your PYTHONPATH or ``sys.path``.
|
||||
It is thus good practise for projects to either put ``conftest.py``
|
||||
under a package scope or to never import anything from a
|
||||
conftest.py file.
|
||||
|
||||
.. _`external plugins`:
|
||||
.. _`extplugins`:
|
||||
.. _`using plugins`:
|
||||
|
||||
Installing External Plugins / Searching
|
||||
---------------------------------------
|
||||
Installing and Using plugins
|
||||
============================
|
||||
|
||||
Installing a plugin happens through any usual Python installation
|
||||
tool, for example::
|
||||
This section talks about installing and using third party plugins.
|
||||
For writing your own plugins, please refer to :ref:`writing-plugins`.
|
||||
|
||||
Installing a third party plugin can be easily done with ``pip``::
|
||||
|
||||
pip install pytest-NAME
|
||||
pip uninstall pytest-NAME
|
||||
|
|
@ -120,118 +70,20 @@ You may also discover more plugins through a `pytest- pypi.python.org search`_.
|
|||
.. _`pytest- pypi.python.org search`: http://pypi.python.org/pypi?%3Aaction=search&term=pytest-&submit=search
|
||||
|
||||
|
||||
Writing a plugin by looking at examples
|
||||
---------------------------------------
|
||||
|
||||
.. _`setuptools`: http://pypi.python.org/pypi/setuptools
|
||||
|
||||
If you want to write a plugin, there are many real-life examples
|
||||
you can copy from:
|
||||
|
||||
* a custom collection example plugin: :ref:`yaml plugin`
|
||||
* around 20 `builtin plugins`_ which provide pytest's own functionality
|
||||
* many `external plugins`_ providing additional features
|
||||
|
||||
All of these plugins implement the documented `well specified hooks`_
|
||||
to extend and add functionality.
|
||||
|
||||
You can also :ref:`contribute your plugin to pytest-dev<submitplugin>`
|
||||
once it has some happy users other than yourself.
|
||||
|
||||
|
||||
.. _`setuptools entry points`:
|
||||
|
||||
Making your plugin installable by others
|
||||
----------------------------------------
|
||||
|
||||
If you want to make your plugin externally available, you
|
||||
may define a so-called entry point for your distribution so
|
||||
that ``pytest`` finds your plugin module. Entry points are
|
||||
a feature that is provided by `setuptools`_. pytest looks up
|
||||
the ``pytest11`` entrypoint to discover its
|
||||
plugins and you can thus make your plugin available by defining
|
||||
it in your setuptools-invocation:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
# sample ./setup.py file
|
||||
from setuptools import setup
|
||||
|
||||
setup(
|
||||
name="myproject",
|
||||
packages = ['myproject']
|
||||
|
||||
# the following makes a plugin available to pytest
|
||||
entry_points = {
|
||||
'pytest11': [
|
||||
'name_of_plugin = myproject.pluginmodule',
|
||||
]
|
||||
},
|
||||
)
|
||||
|
||||
If a package is installed this way, ``pytest`` will load
|
||||
``myproject.pluginmodule`` as a plugin which can define
|
||||
`well specified hooks`_.
|
||||
|
||||
|
||||
.. _`pluginorder`:
|
||||
|
||||
Plugin discovery order at tool startup
|
||||
--------------------------------------
|
||||
|
||||
``pytest`` loads plugin modules at tool startup in the following way:
|
||||
|
||||
* by loading all builtin plugins
|
||||
|
||||
* by loading all plugins registered through `setuptools entry points`_.
|
||||
|
||||
* by pre-scanning the command line for the ``-p name`` option
|
||||
and loading the specified plugin before actual command line parsing.
|
||||
|
||||
* by loading all :file:`conftest.py` files as inferred by the command line
|
||||
invocation:
|
||||
|
||||
- if no test paths are specified use current dir as a test path
|
||||
- if exists, load ``conftest.py`` and ``test*/conftest.py`` relative
|
||||
to the directory part of the first test path.
|
||||
|
||||
Note that pytest does not find ``conftest.py`` files in deeper nested
|
||||
sub directories at tool startup. It is usually a good idea to keep
|
||||
your conftest.py file in the top level test or project root directory.
|
||||
|
||||
* by recursively loading all plugins specified by the
|
||||
``pytest_plugins`` variable in ``conftest.py`` files
|
||||
|
||||
|
||||
Requiring/Loading plugins in a test module or conftest file
|
||||
-----------------------------------------------------------
|
||||
|
||||
You can require plugins in a test module or a conftest file like this::
|
||||
|
||||
pytest_plugins = "name1", "name2",
|
||||
pytest_plugins = "myapp.testsupport.myplugin",
|
||||
|
||||
When the test module or conftest plugin is loaded the specified plugins
|
||||
will be loaded as well. You can also use dotted path like this::
|
||||
will be loaded as well.
|
||||
|
||||
pytest_plugins = "myapp.testsupport.myplugin"
|
||||
|
||||
which will import the specified module as a ``pytest`` plugin.
|
||||
|
||||
|
||||
Accessing another plugin by name
|
||||
--------------------------------
|
||||
|
||||
If a plugin wants to collaborate with code from
|
||||
another plugin it can obtain a reference through
|
||||
the plugin manager like this:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
plugin = config.pluginmanager.getplugin("name_of_plugin")
|
||||
|
||||
If you want to look at the names of existing plugins, use
|
||||
the ``--traceconfig`` option.
|
||||
|
||||
.. _`findpluginname`:
|
||||
|
||||
Finding out which plugins are active
|
||||
|
|
@ -293,223 +145,3 @@ in the `pytest repository <http://bitbucket.org/pytest-dev/pytest/>`_.
|
|||
_pytest.tmpdir
|
||||
_pytest.unittest
|
||||
|
||||
.. _`well specified hooks`:
|
||||
|
||||
pytest hook reference
|
||||
=====================
|
||||
|
||||
Hook specification and validation
|
||||
---------------------------------
|
||||
|
||||
``pytest`` calls hook functions to implement initialization, running,
|
||||
test execution and reporting. When ``pytest`` loads a plugin it validates
|
||||
that each hook function conforms to its respective hook specification.
|
||||
Each hook function name and its argument names need to match a hook
|
||||
specification. However, a hook function may accept *fewer* parameters
|
||||
by simply not specifying them. If you mistype argument names or the
|
||||
hook name itself you get an error showing the available arguments.
|
||||
|
||||
Initialization, command line and configuration hooks
|
||||
----------------------------------------------------
|
||||
|
||||
.. currentmodule:: _pytest.hookspec
|
||||
|
||||
.. autofunction:: pytest_load_initial_conftests
|
||||
.. autofunction:: pytest_cmdline_preparse
|
||||
.. autofunction:: pytest_cmdline_parse
|
||||
.. autofunction:: pytest_namespace
|
||||
.. autofunction:: pytest_addoption
|
||||
.. autofunction:: pytest_cmdline_main
|
||||
.. autofunction:: pytest_configure
|
||||
.. autofunction:: pytest_unconfigure
|
||||
|
||||
Generic "runtest" hooks
|
||||
-----------------------
|
||||
|
||||
All runtest related hooks receive a :py:class:`pytest.Item` object.
|
||||
|
||||
.. autofunction:: pytest_runtest_protocol
|
||||
.. autofunction:: pytest_runtest_setup
|
||||
.. autofunction:: pytest_runtest_call
|
||||
.. autofunction:: pytest_runtest_teardown
|
||||
.. autofunction:: pytest_runtest_makereport
|
||||
|
||||
For deeper understanding you may look at the default implementation of
|
||||
these hooks in :py:mod:`_pytest.runner` and maybe also
|
||||
in :py:mod:`_pytest.pdb` which interacts with :py:mod:`_pytest.capture`
|
||||
and its input/output capturing in order to immediately drop
|
||||
into interactive debugging when a test failure occurs.
|
||||
|
||||
The :py:mod:`_pytest.terminal` reported specifically uses
|
||||
the reporting hook to print information about a test run.
|
||||
|
||||
Collection hooks
|
||||
----------------
|
||||
|
||||
``pytest`` calls the following hooks for collecting files and directories:
|
||||
|
||||
.. autofunction:: pytest_ignore_collect
|
||||
.. autofunction:: pytest_collect_directory
|
||||
.. autofunction:: pytest_collect_file
|
||||
|
||||
For influencing the collection of objects in Python modules
|
||||
you can use the following hook:
|
||||
|
||||
.. autofunction:: pytest_pycollect_makeitem
|
||||
.. autofunction:: pytest_generate_tests
|
||||
|
||||
After collection is complete, you can modify the order of
|
||||
items, delete or otherwise amend the test items:
|
||||
|
||||
.. autofunction:: pytest_collection_modifyitems
|
||||
|
||||
Reporting hooks
|
||||
---------------
|
||||
|
||||
Session related reporting hooks:
|
||||
|
||||
.. autofunction:: pytest_collectstart
|
||||
.. autofunction:: pytest_itemcollected
|
||||
.. autofunction:: pytest_collectreport
|
||||
.. autofunction:: pytest_deselected
|
||||
|
||||
And here is the central hook for reporting about
|
||||
test execution:
|
||||
|
||||
.. autofunction:: pytest_runtest_logreport
|
||||
|
||||
|
||||
Debugging/Interaction hooks
|
||||
---------------------------
|
||||
|
||||
There are few hooks which can be used for special
|
||||
reporting or interaction with exceptions:
|
||||
|
||||
.. autofunction:: pytest_internalerror
|
||||
.. autofunction:: pytest_keyboard_interrupt
|
||||
.. autofunction:: pytest_exception_interact
|
||||
|
||||
|
||||
Declaring new hooks
|
||||
------------------------
|
||||
|
||||
Plugins and ``conftest.py`` files may declare new hooks that can then be
|
||||
implemented by other plugins in order to alter behaviour or interact with
|
||||
the new plugin:
|
||||
|
||||
.. autofunction:: pytest_addhooks
|
||||
|
||||
Hooks are usually declared as do-nothing functions that contain only
|
||||
documentation describing when the hook will be called and what return values
|
||||
are expected.
|
||||
|
||||
For an example, see `newhooks.py`_ from :ref:`xdist`.
|
||||
|
||||
.. _`newhooks.py`: https://bitbucket.org/pytest-dev/pytest-xdist/src/52082f70e7dd04b00361091b8af906c60fd6700f/xdist/newhooks.py?at=default
|
||||
|
||||
|
||||
Using hooks from 3rd party plugins
|
||||
-------------------------------------
|
||||
|
||||
Using new hooks from plugins as explained above might be a little tricky
|
||||
because the standard `Hook specification and validation`_ mechanism:
|
||||
if you depend on a plugin that is not installed,
|
||||
validation will fail and the error message will not make much sense to your users.
|
||||
|
||||
One approach is to defer the hook implementation to a new plugin instead of
|
||||
declaring the hook functions directly in your plugin module, for example::
|
||||
|
||||
# contents of myplugin.py
|
||||
|
||||
class DeferPlugin(object):
|
||||
"""Simple plugin to defer pytest-xdist hook functions."""
|
||||
|
||||
def pytest_testnodedown(self, node, error):
|
||||
"""standard xdist hook function.
|
||||
"""
|
||||
|
||||
def pytest_configure(config):
|
||||
if config.pluginmanager.hasplugin('xdist'):
|
||||
config.pluginmanager.register(DeferPlugin())
|
||||
|
||||
|
||||
This has the added benefit of allowing you to conditionally install hooks
|
||||
depending on which plugins are installed.
|
||||
|
||||
hookwrapper: executing around other hooks
|
||||
-------------------------------------------------
|
||||
|
||||
.. currentmodule:: _pytest.core
|
||||
|
||||
.. versionadded:: 2.7 (experimental)
|
||||
|
||||
pytest plugins can implement hook wrappers which which wrap the execution
|
||||
of other hook implementations. A hook wrapper is a generator function
|
||||
which yields exactly once. When pytest invokes hooks it first executes
|
||||
hook wrappers and passes the same arguments as to the regular hooks.
|
||||
|
||||
At the yield point of the hook wrapper pytest will execute the next hook
|
||||
implementations and return their result to the yield point in the form of
|
||||
a :py:class:`CallOutcome` instance which encapsulates a result or
|
||||
exception info. The yield point itself will thus typically not raise
|
||||
exceptions (unless there are bugs).
|
||||
|
||||
Here is an example definition of a hook wrapper::
|
||||
|
||||
import pytest
|
||||
|
||||
@pytest.hookimpl_opts(hookwrapper=True)
|
||||
def pytest_pyfunc_call(pyfuncitem):
|
||||
# do whatever you want before the next hook executes
|
||||
outcome = yield
|
||||
# outcome.excinfo may be None or a (cls, val, tb) tuple
|
||||
res = outcome.get_result() # will raise if outcome was exception
|
||||
# postprocess result
|
||||
|
||||
Note that hook wrappers don't return results themselves, they merely
|
||||
perform tracing or other side effects around the actual hook implementations.
|
||||
If the result of the underlying hook is a mutable object, they may modify
|
||||
that result, however.
|
||||
|
||||
|
||||
Reference of objects involved in hooks
|
||||
======================================
|
||||
|
||||
.. autoclass:: _pytest.config.Config()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.config.Parser()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.main.Node()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.main.Collector()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.main.Item()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Module()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Class()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Function()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.runner.CallInfo()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.runner.TestReport()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.core.CallOutcome()
|
||||
:members:
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,469 @@
|
|||
.. _plugins:
|
||||
.. _`writing-plugins`:
|
||||
|
||||
Writing plugins
|
||||
===============
|
||||
|
||||
It is easy to implement `local conftest plugins`_ for your own project
|
||||
or `pip-installable plugins`_ that can be used throughout many projects,
|
||||
including third party projects. Please refer to :ref:`using plugins` if you
|
||||
only want to use but not write plugins.
|
||||
|
||||
A plugin contains one or multiple hook functions. :ref:`Writing hooks <writinghooks>`
|
||||
explains the basics and details of how you can write a hook function yourself.
|
||||
``pytest`` implements all aspects of configuration, collection, running and
|
||||
reporting by calling `well specified hooks`_ of the following plugins:
|
||||
|
||||
* :ref:`builtin plugins`: loaded from pytest's internal ``_pytest`` directory.
|
||||
|
||||
* :ref:`external plugins <extplugin>`: modules discovered through
|
||||
`setuptools entry points`_
|
||||
|
||||
* `conftest.py plugins`_: modules auto-discovered in test directories
|
||||
|
||||
In principle, each hook call is a ``1:N`` Python function call where ``N`` is the
|
||||
number of registered implementation functions for a given specification.
|
||||
All specifications and implementations following the ``pytest_`` prefix
|
||||
naming convention, making them easy to distinguish and find.
|
||||
|
||||
.. _`pluginorder`:
|
||||
|
||||
Plugin discovery order at tool startup
|
||||
--------------------------------------
|
||||
|
||||
``pytest`` loads plugin modules at tool startup in the following way:
|
||||
|
||||
* by loading all builtin plugins
|
||||
|
||||
* by loading all plugins registered through `setuptools entry points`_.
|
||||
|
||||
* by pre-scanning the command line for the ``-p name`` option
|
||||
and loading the specified plugin before actual command line parsing.
|
||||
|
||||
* by loading all :file:`conftest.py` files as inferred by the command line
|
||||
invocation:
|
||||
|
||||
- if no test paths are specified use current dir as a test path
|
||||
- if exists, load ``conftest.py`` and ``test*/conftest.py`` relative
|
||||
to the directory part of the first test path.
|
||||
|
||||
Note that pytest does not find ``conftest.py`` files in deeper nested
|
||||
sub directories at tool startup. It is usually a good idea to keep
|
||||
your conftest.py file in the top level test or project root directory.
|
||||
|
||||
* by recursively loading all plugins specified by the
|
||||
``pytest_plugins`` variable in ``conftest.py`` files
|
||||
|
||||
|
||||
.. _`pytest/plugin`: http://bitbucket.org/pytest-dev/pytest/src/tip/pytest/plugin/
|
||||
.. _`conftest.py plugins`:
|
||||
.. _`conftest.py`:
|
||||
.. _`localplugin`:
|
||||
.. _`conftest`:
|
||||
.. _`local conftest plugins`:
|
||||
|
||||
conftest.py: local per-directory plugins
|
||||
----------------------------------------
|
||||
|
||||
Local ``conftest.py`` plugins contain directory-specific hook
|
||||
implementations. Hook Session and test running activities will
|
||||
invoke all hooks defined in ``conftest.py`` files closer to the
|
||||
root of the filesystem. Example of implementing the
|
||||
``pytest_runtest_setup`` hook so that is called for tests in the ``a``
|
||||
sub directory but not for other directories::
|
||||
|
||||
a/conftest.py:
|
||||
def pytest_runtest_setup(item):
|
||||
# called for running each test in 'a' directory
|
||||
print ("setting up", item)
|
||||
|
||||
a/test_sub.py:
|
||||
def test_sub():
|
||||
pass
|
||||
|
||||
test_flat.py:
|
||||
def test_flat():
|
||||
pass
|
||||
|
||||
Here is how you might run it::
|
||||
|
||||
py.test test_flat.py # will not show "setting up"
|
||||
py.test a/test_sub.py # will show "setting up"
|
||||
|
||||
.. Note::
|
||||
If you have ``conftest.py`` files which do not reside in a
|
||||
python package directory (i.e. one containing an ``__init__.py``) then
|
||||
"import conftest" can be ambiguous because there might be other
|
||||
``conftest.py`` files as well on your PYTHONPATH or ``sys.path``.
|
||||
It is thus good practise for projects to either put ``conftest.py``
|
||||
under a package scope or to never import anything from a
|
||||
conftest.py file.
|
||||
|
||||
|
||||
Writing a plugin by looking at examples
|
||||
---------------------------------------
|
||||
|
||||
.. _`setuptools`: http://pypi.python.org/pypi/setuptools
|
||||
|
||||
If you want to write a plugin, there are many real-life examples
|
||||
you can copy from:
|
||||
|
||||
* a custom collection example plugin: :ref:`yaml plugin`
|
||||
* around 20 doc:`builtin plugins` which provide pytest's own functionality
|
||||
* many :doc:`external plugins` providing additional features
|
||||
|
||||
All of these plugins implement the documented `well specified hooks`_
|
||||
to extend and add functionality.
|
||||
|
||||
You can also :ref:`contribute your plugin to pytest-dev<submitplugin>`
|
||||
once it has some happy users other than yourself.
|
||||
|
||||
|
||||
.. _`setuptools entry points`:
|
||||
.. _`pip-installable plugins`:
|
||||
|
||||
Making your plugin installable by others
|
||||
----------------------------------------
|
||||
|
||||
If you want to make your plugin externally available, you
|
||||
may define a so-called entry point for your distribution so
|
||||
that ``pytest`` finds your plugin module. Entry points are
|
||||
a feature that is provided by `setuptools`_. pytest looks up
|
||||
the ``pytest11`` entrypoint to discover its
|
||||
plugins and you can thus make your plugin available by defining
|
||||
it in your setuptools-invocation:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
# sample ./setup.py file
|
||||
from setuptools import setup
|
||||
|
||||
setup(
|
||||
name="myproject",
|
||||
packages = ['myproject']
|
||||
|
||||
# the following makes a plugin available to pytest
|
||||
entry_points = {
|
||||
'pytest11': [
|
||||
'name_of_plugin = myproject.pluginmodule',
|
||||
]
|
||||
},
|
||||
)
|
||||
|
||||
If a package is installed this way, ``pytest`` will load
|
||||
``myproject.pluginmodule`` as a plugin which can define
|
||||
`well specified hooks`_.
|
||||
|
||||
|
||||
|
||||
|
||||
Requiring/Loading plugins in a test module or conftest file
|
||||
-----------------------------------------------------------
|
||||
|
||||
You can require plugins in a test module or a conftest file like this::
|
||||
|
||||
pytest_plugins = "name1", "name2",
|
||||
|
||||
When the test module or conftest plugin is loaded the specified plugins
|
||||
will be loaded as well. You can also use dotted path like this::
|
||||
|
||||
pytest_plugins = "myapp.testsupport.myplugin"
|
||||
|
||||
which will import the specified module as a ``pytest`` plugin.
|
||||
|
||||
|
||||
Accessing another plugin by name
|
||||
--------------------------------
|
||||
|
||||
If a plugin wants to collaborate with code from
|
||||
another plugin it can obtain a reference through
|
||||
the plugin manager like this:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
plugin = config.pluginmanager.getplugin("name_of_plugin")
|
||||
|
||||
If you want to look at the names of existing plugins, use
|
||||
the ``--traceconfig`` option.
|
||||
|
||||
|
||||
.. _`writinghooks`:
|
||||
|
||||
Writing hook functions
|
||||
======================
|
||||
|
||||
.. _validation:
|
||||
|
||||
hook function validation and execution
|
||||
--------------------------------------
|
||||
|
||||
pytest calls hook functions from registered plugins for any
|
||||
given hook specification. Let's look at a typical hook function
|
||||
for the ``pytest_collection_modifyitems(session, config,
|
||||
items)`` hook which pytest calls after collection of all test items is
|
||||
completed.
|
||||
|
||||
When we implement a ``pytest_collection_modifyitems`` function in our plugin
|
||||
pytest will during registration verify that you use argument
|
||||
names which match the specification and bail out if not.
|
||||
|
||||
Let's look at a possible implementation::
|
||||
|
||||
def pytest_collection_modifyitems(config, items):
|
||||
# called after collectin is completed
|
||||
# you can modify the ``items`` list
|
||||
|
||||
Here, ``pytest`` will pass in ``config`` (the pytest config object)
|
||||
and ``items`` (the list of collected test items) but will not pass
|
||||
in the ``session`` argument because we didn't list it in the function
|
||||
signature. This dynamic "pruning" of arguments allows ``pytest`` to
|
||||
be "future-compatible": we can introduce new hook named parameters without
|
||||
breaking the signatures of existing hook implementations. It is one of
|
||||
the reasons for the general long-lived compatibility of pytest plugins.
|
||||
|
||||
Hook function results
|
||||
---------------------
|
||||
|
||||
Most calls to ``pytest`` hooks result in a **list of results** which contains
|
||||
all non-None results of the called hook functions.
|
||||
|
||||
Some hooks are specified so that the hook call only executes until the
|
||||
first function returned a non-None value which is then also the
|
||||
result of the overall hook call. The remaining hook functions will
|
||||
not be called in this case.
|
||||
|
||||
Note that hook functions other than ``pytest_runtest_*`` are not
|
||||
allowed to raise exceptions. Doing so will break the pytest run.
|
||||
|
||||
Hook function ordering
|
||||
----------------------
|
||||
|
||||
For any given hook there may be more than one implementation and we thus
|
||||
generally view ``hook`` execution as a ``1:N`` function call where ``N``
|
||||
is the number of registered functions. There are ways to
|
||||
influence if a hook implementation comes before or after others, i.e.
|
||||
the position in the ``N``-sized list of functions::
|
||||
|
||||
@pytest.hookimpl_spec(tryfirst=True)
|
||||
def pytest_collection_modifyitems(items):
|
||||
# will execute as early as possible
|
||||
|
||||
@pytest.hookimpl_spec(trylast=True)
|
||||
def pytest_collection_modifyitems(items):
|
||||
# will execute as late as possible
|
||||
|
||||
|
||||
hookwrapper: executing around other hooks
|
||||
-------------------------------------------------
|
||||
|
||||
.. currentmodule:: _pytest.core
|
||||
|
||||
.. versionadded:: 2.7 (experimental)
|
||||
|
||||
pytest plugins can implement hook wrappers which wrap the execution
|
||||
of other hook implementations. A hook wrapper is a generator function
|
||||
which yields exactly once. When pytest invokes hooks it first executes
|
||||
hook wrappers and passes the same arguments as to the regular hooks.
|
||||
|
||||
At the yield point of the hook wrapper pytest will execute the next hook
|
||||
implementations and return their result to the yield point in the form of
|
||||
a :py:class:`CallOutcome` instance which encapsulates a result or
|
||||
exception info. The yield point itself will thus typically not raise
|
||||
exceptions (unless there are bugs).
|
||||
|
||||
Here is an example definition of a hook wrapper::
|
||||
|
||||
import pytest
|
||||
|
||||
@pytest.hookimpl_opts(hookwrapper=True)
|
||||
def pytest_pyfunc_call(pyfuncitem):
|
||||
# do whatever you want before the next hook executes
|
||||
|
||||
outcome = yield
|
||||
# outcome.excinfo may be None or a (cls, val, tb) tuple
|
||||
|
||||
res = outcome.get_result() # will raise if outcome was exception
|
||||
# postprocess result
|
||||
|
||||
Note that hook wrappers don't return results themselves, they merely
|
||||
perform tracing or other side effects around the actual hook implementations.
|
||||
If the result of the underlying hook is a mutable object, they may modify
|
||||
that result, however.
|
||||
|
||||
Declaring new hooks
|
||||
------------------------
|
||||
|
||||
.. currentmodule:: _pytest.hookspec
|
||||
|
||||
Plugins and ``conftest.py`` files may declare new hooks that can then be
|
||||
implemented by other plugins in order to alter behaviour or interact with
|
||||
the new plugin:
|
||||
|
||||
.. autofunction:: pytest_addhooks
|
||||
|
||||
Hooks are usually declared as do-nothing functions that contain only
|
||||
documentation describing when the hook will be called and what return values
|
||||
are expected.
|
||||
|
||||
For an example, see `newhooks.py`_ from :ref:`xdist`.
|
||||
|
||||
.. _`newhooks.py`: https://bitbucket.org/pytest-dev/pytest-xdist/src/52082f70e7dd04b00361091b8af906c60fd6700f/xdist/newhooks.py?at=default
|
||||
|
||||
|
||||
Using hooks from 3rd party plugins
|
||||
-------------------------------------
|
||||
|
||||
Using new hooks from plugins as explained above might be a little tricky
|
||||
because the standard :ref:`validation mechanism <validation>`:
|
||||
if you depend on a plugin that is not installed, validation will fail and
|
||||
the error message will not make much sense to your users.
|
||||
|
||||
One approach is to defer the hook implementation to a new plugin instead of
|
||||
declaring the hook functions directly in your plugin module, for example::
|
||||
|
||||
# contents of myplugin.py
|
||||
|
||||
class DeferPlugin(object):
|
||||
"""Simple plugin to defer pytest-xdist hook functions."""
|
||||
|
||||
def pytest_testnodedown(self, node, error):
|
||||
"""standard xdist hook function.
|
||||
"""
|
||||
|
||||
def pytest_configure(config):
|
||||
if config.pluginmanager.hasplugin('xdist'):
|
||||
config.pluginmanager.register(DeferPlugin())
|
||||
|
||||
This has the added benefit of allowing you to conditionally install hooks
|
||||
depending on which plugins are installed.
|
||||
|
||||
|
||||
.. _`well specified hooks`:
|
||||
|
||||
.. currentmodule:: _pytest.hookspec
|
||||
|
||||
pytest hook reference
|
||||
=====================
|
||||
|
||||
|
||||
Initialization, command line and configuration hooks
|
||||
----------------------------------------------------
|
||||
|
||||
.. autofunction:: pytest_load_initial_conftests
|
||||
.. autofunction:: pytest_cmdline_preparse
|
||||
.. autofunction:: pytest_cmdline_parse
|
||||
.. autofunction:: pytest_namespace
|
||||
.. autofunction:: pytest_addoption
|
||||
.. autofunction:: pytest_cmdline_main
|
||||
.. autofunction:: pytest_configure
|
||||
.. autofunction:: pytest_unconfigure
|
||||
|
||||
Generic "runtest" hooks
|
||||
-----------------------
|
||||
|
||||
All runtest related hooks receive a :py:class:`pytest.Item` object.
|
||||
|
||||
.. autofunction:: pytest_runtest_protocol
|
||||
.. autofunction:: pytest_runtest_setup
|
||||
.. autofunction:: pytest_runtest_call
|
||||
.. autofunction:: pytest_runtest_teardown
|
||||
.. autofunction:: pytest_runtest_makereport
|
||||
|
||||
For deeper understanding you may look at the default implementation of
|
||||
these hooks in :py:mod:`_pytest.runner` and maybe also
|
||||
in :py:mod:`_pytest.pdb` which interacts with :py:mod:`_pytest.capture`
|
||||
and its input/output capturing in order to immediately drop
|
||||
into interactive debugging when a test failure occurs.
|
||||
|
||||
The :py:mod:`_pytest.terminal` reported specifically uses
|
||||
the reporting hook to print information about a test run.
|
||||
|
||||
Collection hooks
|
||||
----------------
|
||||
|
||||
``pytest`` calls the following hooks for collecting files and directories:
|
||||
|
||||
.. autofunction:: pytest_ignore_collect
|
||||
.. autofunction:: pytest_collect_directory
|
||||
.. autofunction:: pytest_collect_file
|
||||
|
||||
For influencing the collection of objects in Python modules
|
||||
you can use the following hook:
|
||||
|
||||
.. autofunction:: pytest_pycollect_makeitem
|
||||
.. autofunction:: pytest_generate_tests
|
||||
|
||||
After collection is complete, you can modify the order of
|
||||
items, delete or otherwise amend the test items:
|
||||
|
||||
.. autofunction:: pytest_collection_modifyitems
|
||||
|
||||
Reporting hooks
|
||||
---------------
|
||||
|
||||
Session related reporting hooks:
|
||||
|
||||
.. autofunction:: pytest_collectstart
|
||||
.. autofunction:: pytest_itemcollected
|
||||
.. autofunction:: pytest_collectreport
|
||||
.. autofunction:: pytest_deselected
|
||||
|
||||
And here is the central hook for reporting about
|
||||
test execution:
|
||||
|
||||
.. autofunction:: pytest_runtest_logreport
|
||||
|
||||
|
||||
Debugging/Interaction hooks
|
||||
---------------------------
|
||||
|
||||
There are few hooks which can be used for special
|
||||
reporting or interaction with exceptions:
|
||||
|
||||
.. autofunction:: pytest_internalerror
|
||||
.. autofunction:: pytest_keyboard_interrupt
|
||||
.. autofunction:: pytest_exception_interact
|
||||
|
||||
|
||||
|
||||
Reference of objects involved in hooks
|
||||
======================================
|
||||
|
||||
.. autoclass:: _pytest.config.Config()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.config.Parser()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.main.Node()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.main.Collector()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.main.Item()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Module()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Class()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.python.Function()
|
||||
:members:
|
||||
:show-inheritance:
|
||||
|
||||
.. autoclass:: _pytest.runner.CallInfo()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.runner.TestReport()
|
||||
:members:
|
||||
|
||||
.. autoclass:: _pytest.core.CallOutcome()
|
||||
:members:
|
||||
|
||||
Loading…
Reference in New Issue