================================================
Customizing and Extending py.test
================================================

basic test configuration
===================================

Command line options
---------------------------------

You can see command line options by running::

    py.test -h

This will display all available command line options
in your specific environment.


setting persistent option defaults
------------------------------------

py.test will lookup option values in this order:

* command line
* conftest.py files
* environment variables

To get an overview on existing names and settings type::

    py.test --help-config

This will print information about all available options
in your environment, including your local plugins and
command line options.

.. _`function arguments`: funcargs.html
.. _`extensions`:

Plugin basics and project configuration
=============================================

.. _`local plugin`:

py.test implements all aspects of its functionality by calling `well specified
hooks`_.  Hook functions are discovered in :file:`conftest.py` files or in
`named plugins`_.  :file:`conftest.py` files are useful for keeping test
extensions and customizations close to test code.

local conftest.py plugins
--------------------------------------------------------------

local `conftest.py` plugins are usually automatically loaded and
registered but its contained hooks are only called when collecting or
running tests in files or directories next to or below the ``conftest.py``
file.  Assume the following layout and content of files::

    a/conftest.py:
        def pytest_runtest_setup(item):
            print ("setting up", item)

    a/test_in_subdir.py:
        def test_sub():
            pass

    test_flat.py:
        def test_flat():
            pass

Here is how you might run it::

     py.test test_flat.py   # will not show "setting up"
     py.test a/test_sub.py  # will show "setting up"

``py.test`` loads all ``conftest.py`` files upwards from the command
line file arguments.  It usually looks up configuration values or hooks
right-to-left, i.e. the closer conftest files are checked before
the further away ones.  This means you can have a ``conftest.py``
in your home directory to provide global configuration values.

.. Note::
    if you have ``conftest.py`` files which do not reside in a
    python package directory (one containing an ``__init__.py``) then
    "import conftest" will be ambigous and should be avoided. **If you
    want to import anything from a ``conftest.py`` file it is better
    to put it inside a package to disambiguate.

.. _`named plugins`: plugin/index.html

Plugin discovery at tool startup
--------------------------------------------

py.test loads plugin modules at tool startup in the following way:

* by loading all plugins registered through `setuptools entry points`_.

* by reading the ``PYTEST_PLUGINS`` environment variable
  and importing the comma-separated list of named plugins.

* by pre-scanning the command line for the ``-p name`` option
  and loading the specified plugin before actual command line parsing.

* by loading all :file:`conftest.py` files as inferred by the command line
  invocation (test files and all of its *parent* directories).
  Note that ``conftest.py`` files from *sub* directories are by default
  not loaded at tool startup.

* by recursively loading all plugins specified by the
  ``pytest_plugins`` variable in a ``conftest.py`` file

Requiring/Loading plugins in a test module or plugin
-------------------------------------------------------------

You can specify plugins in a test module or a plugin like this::

    pytest_plugins = "name1", "name2",

When the test module or plugin is loaded the specified plugins
will be loaded.  If you specify plugins without the ``pytest_``
prefix it will be automatically added.  All plugin names
must be lowercase.

.. _`conftest.py plugin`:
.. _`conftestplugin`:

Writing per-project plugins (conftest.py)
------------------------------------------------------

The purpose of :file:`conftest.py` files is to allow project-specific
test customization.  They thus make for a good place to implement
project-specific test related features through hooks.  For example you may
set the `collect_ignore`_ variable depending on a command line option
by defining the following hook in a ``conftest.py`` file::

    # ./conftest.py in your root or package dir
    collect_ignore = ['hello', 'test_world.py']
    def pytest_addoption(parser):
        parser.addoption("--runall", action="store_true", default=False)
    def pytest_configure(config):
        if config.getvalue("runall"):
            collect_ignore[:] = []

.. _`setuptools entry points`:
.. _registered:

Writing setuptools-registered plugins
------------------------------------------------------

.. _`Distribute`: http://pypi.python.org/pypi/distribute
.. _`setuptools`: http://pypi.python.org/pypi/setuptools

If you want to make your plugin publically available, you
can use `setuptools`_ or `Distribute`_ which both allow
to register an entry point.  ``py.test`` will register
all objects with the ``pytest11`` entry point.
To make your plugin available you may insert the following
lines in your setuptools/distribute-based setup-invocation:

.. sourcecode:: python

    # sample ./setup.py file
    from setuptools import setup

    setup(
        name="myproject",
        packages = ['myproject']

        # the following makes a plugin available to py.test
        entry_points = {
            'pytest11': [
                'name_of_plugin = myproject.pluginmodule',
            ]
        },
    )

If a package is installed with this setup, py.test will load
``myproject.pluginmodule`` under the ``name_of_plugin`` name
and use it as a plugin.

Accessing another plugin by name
--------------------------------------------

If a plugin wants to collaborate with code from
another plugin it can obtain a reference through
the plugin manager like this:

.. sourcecode:: python

    plugin = config.pluginmanager.getplugin("name_of_plugin")

If you want to look at the names of existing plugins, use
the ``--traceconfig`` option.

.. _`well specified hooks`:

py.test hook reference
====================================

hook specification and validation
-----------------------------------------

py.test calls hook functions to implement initialization, running,
test execution and reporting.  When py.test loads a plugin it validates
that all hook functions conform to their respective hook specification.
Each hook function name and its argument names need to match a hook
specification exactly but it is allowed for a hook function to accept 
*less* parameters than specified.  If you mistype argument names or the
hook name itself you get useful errors.

initialisation, command line and configuration hooks
--------------------------------------------------------------------

.. currentmodule:: pytest.hookspec

.. autofunction:: pytest_cmdline_parse
.. autofunction:: pytest_namespace
.. autofunction:: pytest_addoption
.. autofunction:: pytest_cmdline_main
.. autofunction:: pytest_configure
.. autofunction:: pytest_unconfigure

generic "runtest" hooks
------------------------------

All all runtest related hooks receive a :py:class:`pytest.collect.Item` object.

.. autofunction:: pytest_runtest_protocol
.. autofunction:: pytest_runtest_setup
.. autofunction:: pytest_runtest_call
.. autofunction:: pytest_runtest_teardown
.. autofunction:: pytest_runtest_makereport

For deeper understanding you may look at the default implementation of
these hooks in :py:mod:`pytest.plugin.runner` and maybe also
in :py:mod:`pytest.plugin.pdb` which intercepts creation
of reports in order to drop to interactive debugging.

The :py:mod:`pytest.plugin.terminal` reported specifically uses
the reporting hook to print information about a test run.

collection hooks
------------------------------

py.test calls the following hooks for collecting files and directories:

.. autofunction:: pytest_ignore_collect
.. autofunction:: pytest_collect_directory
.. autofunction:: pytest_collect_file

For influencing the collection of objects in Python modules
you can use the following hook:

.. autofunction:: pytest_pycollect_makeitem


reporting hooks
------------------------------

Collection related reporting hooks:

.. autofunction: pytest_collectstart
.. autofunction: pytest_log_itemcollect
.. autofunction: pytest_collectreport
.. autofunction: pytest_deselected

And here is the central hook for reporting about
test execution:

.. autofunction: pytest_runtest_logreport

The test collection tree
======================================================


Default filesystem test discovery
-----------------------------------------------

Test collection starts from specified paths or from the current
directory.  All tests are collected ahead of running the first test.
(This used to be different in earlier versions of ``py.test`` where
collection and running was interweaved which made test randomization
and distributed testing harder).

Collection nodes which have children are called "Collectors" and otherwise
they are called "Items" or "test items".  Here is an example of such a
tree::

    example $ py.test --collectonly test_collectonly.py
    <Directory 'example'>
      <Module 'test_collectonly.py'>
        <Function 'test_function'>
        <Class 'TestClass'>
          <Instance '()'>
            <Function 'test_method'>
            <Function 'test_anothermethod'>

By default all directories not starting with a dot are traversed,
looking for ``test_*.py`` and ``*_test.py`` files.  Those Python
files are imported under their `package name`_.

The Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed ``test`` by default.  Test classes must
start with a capitalized ``Test`` prefix.

Customizing error messages
-------------------------------------------------

On test and collection nodes ``py.test`` will invoke
the ``node.repr_failure(excinfo)`` function which
you may override and make it return an error
representation string of your choice.  It
will be reported as a (red) string.

.. _`package name`:

constructing the package name for test modules
-------------------------------------------------

Test modules are imported under their fully qualified
name.  Given a filesystem ``fspath`` it is constructed as follows:

* walk the directories up to the last one that contains
  an ``__init__.py`` file.

* perform ``sys.path.insert(0, basedir)``.

* import the root package as ``root``

Reference of important objects involved in hooks
===========================================================

.. autoclass:: pytest.plugin.config.Config
    :members:

.. autoclass:: pytest.plugin.session.Item
    :inherited-members:

.. autoclass:: pytest.plugin.session.Node
    :members:

.. autoclass:: pytest.plugin.runner.CallInfo
    :members:

.. autoclass:: pytest.plugin.runner.TestReport
    :members:


conftest.py configuration files
=================================================

conftest.py reference docs

A unique feature of py.test are its ``conftest.py`` files which allow
project and directory specific customizations to testing.

* `set option defaults`_

or set particular variables to influence the testing process:

* ``pytest_plugins``: list of named plugins to load

* ``collect_ignore``: list of paths to ignore during test collection, relative to the containing ``conftest.py`` file

* ``rsyncdirs``: list of to-be-rsynced directories for distributed
  testing, relative to the containing ``conftest.py`` file.

You may put a conftest.py files in your project root directory or into
your package directory if you want to add project-specific test options.


.. _`specify funcargs`: funcargs.html#application-setup-tutorial-example

.. _`set option defaults`: