================================================
Customizing and Extending py.test
================================================

basic test configuration
===================================

available command line options
---------------------------------

You can see command line options by running::

    py.test -h

This will display all available command line options
in your specific environment.

.. _`project-specific test configuration`:
.. _`collect_ignore`:

conftest.py: project specific hooks and configuration
--------------------------------------------------------

A unique feature of py.test are its ``conftest.py`` files which
allow to:

* `set option defaults`_

* `implement hooks`_

* `specify funcargs`_

or set particular variables to influence the testing process:

* ``pytest_plugins``: list of named plugins to load

* ``collect_ignore``: list of paths to ignore during test collection, relative to the containing ``conftest.py`` file

* ``rsyncdirs``: list of to-be-rsynced directories for distributed
  testing, relative to the containing ``conftest.py`` file.

You may put a conftest.py files in your project root directory or into
your package directory if you want to add project-specific test options.

``py.test`` loads all ``conftest.py`` files upwards from the command
line file arguments.  It usually looks up configuration values
right-to-left, i.e. the closer conftest files will be checked first.
This means you can have a ``conftest.py`` in your very home directory to
have some global configuration values.

.. _`specify funcargs`: funcargs.html#application-setup-tutorial-example

.. _`set option defaults`:

setting persistent option defaults
------------------------------------

py.test will lookup option values in this order:

* command line
* conftest.py files
* environment variables

To find out about the particular switches and type::

    py.test --help-config

This will print information about all options in your
environment, including your local plugins.

.. _`basetemp`:

Temporary directories
-------------------------------------------

You can create directories by calling one of two methods
on the config object:

- ``config.mktemp(basename)``: create and return a new tempdir

- ``config.ensuretemp(basename)``: create or return a new tempdir

temporary directories are created as sub directories of a per-session
testdir and will keep around the directories of the last three test
runs.  You can set the base temporary directory through the command line
`--basetemp`` option.  When distributing tests on the same machine,
``py.test`` takes care to configure a basetemp directory for the sub
processes such that all temporary data lands below below a single
per-test run basetemp directory.

.. _`function arguments`: funcargs.html
.. _`extensions`:

Plugin basics and project configuration
=============================================

.. _`local plugin`:

project specific "local" or named "global" plugins
--------------------------------------------------------------

py.test implements much of its functionality by calling `well specified
hooks`_.  Python modules which contain such hook functions are called
plugins.  Hook functions are discovered in ``conftest.py`` files or in
`named plugins`_.  ``conftest.py`` files are sometimes called
"anonymous" or conftest plugins. They are useful for keeping test
extensions close to your application.  Named plugins are normal python
modules or packages that can be distributed separately.   Named plugins
need to follow a naming pattern; they have an all lowercase ``pytest_``
prefixed name.  While conftest plugins are discovered automatically,
named plugins must be explicitely specified.

.. _`named plugins`: plugin/index.html

.. _`tool startup`:
.. _`loaded at tool startup`:
.. _`test tool starts up`:

Plugin discovery at tool startup
--------------------------------------------

py.test loads plugin modules at tool startup in the following way:

* by loading all plugins registered through `setuptools entry points`_.

* by reading the ``PYTEST_PLUGINS`` environment variable
  and importing the comma-separated list of named plugins.

* by pre-scanning the command line for the ``-p name`` option
  and loading the specified plugin before actual command line parsing.

* by loading all `conftest.py plugin`_ files as inferred by the command line
  invocation (test files and all of its parent directories).
  Note that ``conftest.py`` files from sub directories are loaded
  during test collection and not at tool startup.

* by recursively loading all plugins specified by the
  ``pytest_plugins`` variable in a ``conftest.py`` file

Specifying plugins in a test module or plugin
-----------------------------------------------

You can specify plugins in a test module or a plugin like this:

.. sourcecode:: python

    pytest_plugins = "name1", "name2",

When the test module or plugin is loaded the specified plugins
will be loaded.  If you specify plugins without the ``pytest_``
prefix it will be automatically added.  All plugin names
must be lowercase.

.. _`conftest.py plugin`:
.. _`conftestplugin`:

Writing per-project plugins (conftest.py)
------------------------------------------------------

The purpose of ``conftest.py`` files is to allow `project-specific
test configuration`_.  They thus make for a good place to implement
project-specific test related features through hooks.  For example you may
set the `collect_ignore`_ variable depending on a command line option
by defining the following hook in a ``conftest.py`` file:

.. _`exclude-file-example`:

.. sourcecode:: python

    # ./conftest.py in your root or package dir
    collect_ignore = ['hello', 'test_world.py']
    def pytest_addoption(parser):
        parser.addoption("--runall", action="store_true", default=False)
    def pytest_configure(config):
        if config.getvalue("runall"):
            collect_ignore[:] = []

.. _`setuptools entry points`:

Writing setuptools-registered plugins
------------------------------------------------------

.. _`Distribute`: http://pypi.python.org/pypi/distribute
.. _`setuptools`: http://pypi.python.org/pypi/setuptools

If you want to make your plugin publically available, you
can use `setuptools`_ or `Distribute`_ which both allow
to register an entry point.  ``py.test`` will register
all objects with the ``pytest11`` entry point.
To make your plugin available you may insert the following
lines in your setuptools/distribute-based setup-invocation:

.. sourcecode:: python

    # sample ./setup.py file
    from setuptools import setup

    setup(
        name="myproject",
        packages = ['myproject']

        # the following makes a plugin available to py.test
        entry_points = {
            'pytest11': [
                'name_of_plugin = myproject.pluginmodule',
            ]
        },
    )

If a package is installed with this setup, py.test will load
``myproject.pluginmodule`` under the ``name_of_plugin`` name
and use it as a plugin.

Accessing another plugin by name
--------------------------------------------

If a plugin wants to collaborate with code from
another plugin it can obtain a reference through
the plugin manager like this:

.. sourcecode:: python

    plugin = config.pluginmanager.getplugin("name_of_plugin")

If you want to look at the names of existing plugins, use
the ``--traceconfig`` option.

.. _`well specified hooks`:
.. _`implement hooks`:

py.test hook reference
====================================

hook specification and validation
-----------------------------------------

py.test calls hook functions to implement initialization, running,
test execution and reporting.  When py.test loads a plugin it validates
that all hook functions conform to their respective hook specification.
Each hook function name and its argument names need to match a hook
specification exactly but it is allowed for a hook function to accept 
*less* parameters than specified.  If you mistype argument names or the
hook name itself you get useful errors.

initialisation, command line and configuration hooks
--------------------------------------------------------------------

.. currentmodule:: pytest.hookspec

.. autofunction:: pytest_namespace
.. autofunction:: pytest_addoption
.. autofunction:: pytest_cmdline_main
.. autofunction:: pytest_configure
.. autofunction:: pytest_unconfigure

generic "runtest" hooks
------------------------------

Almost runtest related hooks receive a :py:class:`pytest.collect.Item` object.

.. autofunction:: pytest_runtest_protocol
.. autofunction:: pytest_runtest_setup
.. autofunction:: pytest_runtest_call
.. autofunction:: pytest_runtest_teardown
.. autofunction:: pytest_runtest_makereport

For deeper understanding you may look at the default implementation of
these hooks in :py:mod:`pytest.plugin.runner` and maybe also
in :py:mod:`pytest.plugin.pdb` which intercepts creation
of reports in order to drop to interactive debugging.

The :py:mod:`pytest.plugin.terminal` reported specifically uses
the reporting hook to print information about a test run.

collection hooks
------------------------------

py.test calls the following hooks for collecting files and directories:

.. autofunction:: pytest_ignore_collect
.. autofunction:: pytest_collect_directory
.. autofunction:: pytest_collect_file

For influencing the collection of objects in Python modules
you can use the following hook:

.. autofunction:: pytest_pycollect_makeitem


reporting hooks
------------------------------

Collection related reporting hooks:

.. autofunction: pytest_collectstart
.. autofunction: pytest_log_itemcollect
.. autofunction: pytest_collectreport
.. autofunction: pytest_deselected

And here is the central hook for reporting about
test execution:

.. autofunction: pytest_runtest_logreport

The test collection tree
======================================================


Default filesystem test discovery
-----------------------------------------------

Test collection starts from specified paths or from the current
directory.  All tests are collected ahead of running the first test.
(This used to be different in earlier versions of ``py.test`` where
collection and running was interweaved which made test randomization
and distributed testing harder).

Collection nodes which have children are called "Collectors" and otherwise
they are called "Items" or "test items".  Here is an example of such a
tree::

    testing $ py.test --collectonly test_parseonly.py
    <Directory 'testing'>
      <Module 'test_parseopt.py'>
        <Class 'TestParser'>
          <Instance '()'>
            <Function 'test_init'>
            <Function 'test_group_add_and_get'>
            <Function 'test_addgroup_deprecation'>
            <Function 'test_getgroup_simple'>
            <Function 'test_group_ordering'>
            <Function 'test_group_addoption'>
            <Function 'test_group_shortopt_lowercase'>
            <Function 'test_parser_addoption'>
            <Function 'test_parse'>
            <Function 'test_parse_will_set_default'>
            <Function 'test_parse_setoption'>
            <Function 'test_parse_defaultgetter'>
        <Function 'test_addoption_parser_epilog'>

By default all directories not starting with a dot are traversed,
looking for ``test_*.py`` and ``*_test.py`` files.  Those Python
files are imported under their `package name`_.

The Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed ``test`` by default.  Test classes must
start with a capitalized ``Test`` prefix.

Customizing error messages
-------------------------------------------------

On test and collection nodes ``py.test`` will invoke
the ``node.repr_failure(excinfo)`` function which
you may override and make it return an error
representation string of your choice.  It
will be reported as a (red) string.

.. _`package name`:

constructing the package name for test modules
-------------------------------------------------

Test modules are imported under their fully qualified
name.  Given a filesystem ``fspath`` it is constructed as follows:

* walk the directories up to the last one that contains
  an ``__init__.py`` file.

* perform ``sys.path.insert(0, basedir)``.

* import the root package as ``root``

Complete reference of objects involved in hooks
===========================================================

.. autoclass:: pytest.plugin.runner.CallInfo
    :members:

.. autoclass:: pytest.plugin.runner.TestReport
    :members:

.. autoclass:: pytest.collect.Node
    :members:

.. autoclass:: pytest.collect.Item
    :inherited-members: