================================================
Extending and customizing py.test 
================================================

.. _`local plugin`:

py.test implements much of its functionality by calling `well specified
hooks`_.  Python modules which contain such hook functions are called
plugins.  Hook functions are discovered in ``conftest.py`` files or in
`named plugins`_.  ``conftest.py`` files are sometimes called
"anonymous" or conftest plugins. They are useful for keeping test
extensions close to your application.  Named plugins are normal python
modules or packages that can be distributed separately.   Named plugins
need to follow a naming pattern; they have an all lowercase ``pytest_``
prefixed name.  While conftest plugins are discovered automatically,
named plugins must be explicitely specified. 

.. _`named plugins`: plugin/index.html

.. _`tool startup`:
.. _`test tool starts up`: 

Plugin discovery at tool startup
--------------------------------------------

py.test loads plugin modules at tool startup in the following way:

* by reading the ``PYTEST_PLUGINS`` environment variable 
  and importing the comma-separated list of named plugins. 

* by pre-scanning the command line for the ``-p name`` option
  and loading the specified plugin before actual command line parsing. 

* by loading all `conftest.py plugin`_ files as inferred by the command line
  invocation 

* by recursively loading all plugins specified by the 
  ``pytest_plugins`` variable in a ``conftest.py`` file 

Note that at tool startup only ``conftest.py`` files in
the directory of the specified test modules (or the current dir if None) 
or any of the parent directories are found.  There is no try to 
pre-scan all subdirectories to find ``conftest.py`` files or test 
modules.  

Specifying plugins in a test module or plugin
-----------------------------------------------

You can specify plugins in a test module or a plugin like this:

.. sourcecode:: python

    pytest_plugins = "name1", "name2", 

When the test module or plugin is loaded the specified plugins
will be loaded.  If you specify plugins without the ``pytest_``
prefix it will be automatically added.  All plugin names 
must be lowercase. 

.. _`conftest.py plugin`:
.. _`conftestplugin`:

conftest.py as anonymous per-project plugins 
--------------------------------------------------

The purpose of ``conftest.py`` files is to allow `project-specific 
test configuration`_.  They thus make for a good place to implement 
project-specific test related features through hooks.  For example you may 
set the `collect_ignore`_ variable depending on a command line option
by defining the following hook in a ``conftest.py`` file:

.. _`exclude-file-example`: 

.. sourcecode:: python

    # ./conftest.py in your root or package dir 
    collect_ignore = ['hello', 'test_world.py'] 
    def pytest_addoption(parser):
        parser.addoption("--runall", action="store_true", default=False)
    def pytest_configure(config):
        if config.getvalue("runall"):
            collect_ignore[:] = []

.. _`project-specific test configuration`: config.html#conftestpy 
.. _`collect_ignore`: config.html#collectignore

.. _`well specified hooks`: 

Available py.test hooks 
====================================

py.test calls hooks functions to implement its `test collection`_, running and 
reporting process.  Upon loading of a plugin py.test performs
strict checking on contained hook functions.  Function and argument names 
need to match exactly one of `hook definition specification`_. It thus 
provides useful error reporting on mistyped hook or argument names
and minimizes version incompatibilites.  Below you find some introductory 
information on particular hooks.  It's sensible to look at existing 
plugins so see example usages and start off with your own plugin. 

.. _`hook definition specification`: plugin/hookspec.html

.. _`configuration hooks`: 

command line parsing and configuration hooks 
--------------------------------------------------------------------

When the `test tool starts up`_ it will invoke all hooks that add
command line options in the python standard optparse style. 

.. sourcecode:: python

    def pytest_addoption(parser): 
        """ add command line options. """" 
        parser.addoption("--myopt", dest="myopt", action="store_true")

After all these hooks have been called, the command line is parser 
and a ``config`` object is created and another hook is invoked, 
for example: 

.. sourcecode:: python

    def pytest_configure(config): 
        config.getvalue("myopt") 

When the test run finishes this corresponding finalizer hook is called:

    def pytest_unconfigure(config): 
        ... 


adding global py.test helpers and functionality 
--------------------------------------------------------------------

If you want to make global helper functions or objects available 
to your test code you can implement: 

    def pytest_namespace():
        """ return dictionary with items to be made available on py.test. namespace """

All such returned items will be made available directly on 
the ``py.test`` namespace.  

If you want to provide helpers that are specific to a test function run or need
to be setup per test function run, please refer to the `funcargs mechanism`_. 

.. _`funcargs mechanism`: funcargs.html


generic "runtest" hooks 
------------------------------

Each test item is usually executed by calling the following three hooks:

.. sourcecode:: python

    pytest_runtest_setup(item) 
    pytest_runtest_call(item) 
    pytest_runtest_teardown(item) 

For each of the three invocations a `call object`_ encapsulates
information about the outcome of the call and is subsequently used 
to make a report object: 

.. sourcecode:: python

    report = hook.pytest_runtest_makereport(item, call)

For example, the `pytest_pdb plugin`_ uses this hook to activate
interactive debugging on failures when ``--pdb`` is specified on the
command line. 

Usually three reports will be generated for a single test item.  However, 
if the ``pytest_runtest_setup`` fails no call or teardown hooks 
will be called and only one report will be created. 

Each of the up to three reports is eventually fed to the logreport hook:

.. sourcecode:: python

    pytest_runtest_logreport(report) 

A ``report`` object contains status and reporting information:

.. sourcecode:: python

    report.longrepr = string/lines/object to print 
    report.when = "setup", "call" or "teardown" 
    report.shortrepr = letter for progress-report 
    report.passed = True or False
    report.failed = True or False
    report.skipped = True or False

The `pytest_terminal plugin`_ uses this hook to print information
about a test run. 
    
The protocol described here is implemented via this hook:

.. sourcecode:: python
 
    pytest_runtest_protocol(item) -> True 

.. _`call object`: 

The call object contains information about a performed call:

.. sourcecode:: python

    call.excinfo = ExceptionInfo object or None 
    call.when = "setup", "call" or "teardown" 
    call.outerr = None or tuple of strings representing captured stdout/stderr 

.. _`pytest_pdb plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_pdb.py
.. _`pytest_terminal plugin`: http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/pytest_terminal.py


generic collection hooks 
------------------------------

py.test calls the following two fundamental hooks for collecting files and directories: 

.. sourcecode:: python

    def pytest_collect_directory(path, parent):
        """ return Collection node or None for the given path. """

    def pytest_collect_file(path, parent):
        """ return Collection node or None for the given path. """

Both return a `collection node`_ for a given path.  All returned 
nodes from all hook implementations will participate in the
collection and running protocol.  The ``parent`` object is
the parent node and may be used to access command line
options via the ``parent.config`` object. 


Python test function and module hooks 
----------------------------------------------------

For influencing the collection of objects in Python modules 
you can use the following hook:

.. sourcecode:: python

    def pytest_pycollect_makeitem(collector, name, obj):
        """ return custom item/collector for a python object in a module, or None.  """

This hook will be called for each Python object in a collected
Python module.  The return value is a custom `collection node`_ or None. 

.. XXX or ``False`` if you want to indicate that the given item should not be collected. 



Included default plugins
=============================

You can find the source code of all default plugins in 

    http://bitbucket.org/hpk42/py-trunk/src/tip/py/test/plugin/

Additionally you can check out some more contributed plugins here

    http://bitbucket.org/hpk42/py-trunk/src/tip/contrib/


.. _`collection process`: 
.. _`collection node`: 
.. _`test collection`:


Test Collection process 
====================================================== 

The collecting process is iterative so that distribution
and execution of tests can start as soon as the first test
item is collected.  Collection nodes with children are 
called "Collectors" and terminal nodes are called "Items".  
Here is an example of such a tree, generated with the 
command ``py.test --collectonly py/xmlobj``:: 

    <Directory 'xmlobj'>
        <Directory 'testing'>
            <Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
                <Function 'test_html_name_stickyness'>
                <Function 'test_stylenames'>
                <Function 'test_class_None'>
                <Function 'test_alternating_style'>
            <Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
                <Function 'test_tag_with_text'>
                <Function 'test_class_identity'>
                <Function 'test_tag_with_text_and_attributes'>
                <Function 'test_tag_with_subclassed_attr_simple'>
                <Function 'test_tag_nested'>
                <Function 'test_tag_xmlname'>

By default all directories not starting with a dot are traversed, 
looking for ``test_*.py`` and ``*_test.py`` files.  Those Python 
files are imported under their `package name`_. 

The Module collector looks for test functions 
and test classes and methods. Test functions and methods
are prefixed ``test`` by default.  Test classes must 
start with a capitalized ``Test`` prefix. 

.. _`package name`: 

constructing the package name for test modules
-------------------------------------------------

Test modules are imported under their fully qualified 
name.  Given a filesystem ``fspath`` it is constructed as follows:

* walk the directories up to the last one that contains 
  an ``__init__.py`` file.  

* perform ``sys.path.insert(0, basedir)``.

* import the root package as ``root``

* determine the fully qualified name for ``fspath`` by either:

  * calling ``root.__pkg__.getimportname(fspath)`` if the
    ``__pkg__`` exists.` or 

  * otherwise use the relative path of the module path to
    the base dir and turn slashes into dots and strike
    the trailing ``.py``.