test_ok1/doc/test/customize.txt

================================================
Customizing and Extending py.test 
================================================

.. contents:: 
    :local:
    :depth: 2

basic test configuration 
===================================

available command line options 
---------------------------------

You can see command line options by running::

    py.test -h 

This will display all available command line options
in your specific environment. 

.. _`project-specific test configuration`: 
.. _`collect_ignore`: 

conftest.py: project specific hooks and configuration
--------------------------------------------------------

A unique feature of py.test are its ``conftest.py`` files which
allow to:

* `set option defaults`_

* `implement hooks`_

* `specify funcargs`_

or set particular variables to influence the testing process:

* ``pytest_plugins``: list of named plugins to load 

* ``collect_ignore``: list of paths to ignore during test collection, relative to the containing ``conftest.py`` file 

* ``rsyncdirs``: list of to-be-rsynced directories for distributed
  testing, relative to the containing ``conftest.py`` file. 

You may put a conftest.py files in your project root directory or into
your package directory if you want to add project-specific test options. 

``py.test`` loads all ``conftest.py`` files upwards from the command
line file arguments.  It usually looks up configuration values
right-to-left, i.e. the closer conftest files will be checked first.
This means you can have a ``conftest.py`` in your very home directory to
have some global configuration values. 

.. _`specify funcargs`: funcargs.html#application-setup-tutorial-example

.. _`set option defaults`: 

setting persistent option defaults  
------------------------------------

py.test will lookup option values in this order: 

* command line
* conftest.py files 
* environment variables

To find out about the particular switches and type::

    py.test --help-config

This will print information about all options in your
environment, including your local plugins. 

.. _`basetemp`: 

Temporary directories 
-------------------------------------------

You can create directories by calling one of two methods
on the config object: 

- ``config.mktemp(basename)``: create and return a new tempdir 

- ``config.ensuretemp(basename)``: create or return a new tempdir 

temporary directories are created as sub directories of a per-session
testdir and will keep around the directories of the last three test
runs.  You can set the base temporary directory through the command line
`--basetemp`` option.  When distributing tests on the same machine,
``py.test`` takes care to configure a basetemp directory for the sub
processes such that all temporary data lands below below a single
per-test run basetemp directory.  

.. _`function arguments`: funcargs.html
.. _`extensions`: 

Plugin basics
=========================

.. _`local plugin`:

project specific "local" or named "global" plugins
--------------------------------------------------------------

py.test implements much of its functionality by calling `well specified
hooks`_.  Python modules which contain such hook functions are called
plugins.  Hook functions are discovered in ``conftest.py`` files or in
`named plugins`_.  ``conftest.py`` files are sometimes called
"anonymous" or conftest plugins. They are useful for keeping test
extensions close to your application.  Named plugins are normal python
modules or packages that can be distributed separately.   Named plugins
need to follow a naming pattern; they have an all lowercase ``pytest_``
prefixed name.  While conftest plugins are discovered automatically,
named plugins must be explicitely specified. 

.. _`named plugins`: plugin/index.html

.. _`tool startup`:
.. _`loaded at tool startup`: 
.. _`test tool starts up`: 

Plugin discovery at tool startup
--------------------------------------------

py.test loads plugin modules at tool startup in the following way:

* by loading all plugins registered through `setuptools entry points`_. 

* by reading the ``PYTEST_PLUGINS`` environment variable 
  and importing the comma-separated list of named plugins. 

* by pre-scanning the command line for the ``-p name`` option
  and loading the specified plugin before actual command line parsing. 

* by loading all `conftest.py plugin`_ files as inferred by the command line
  invocation (test files and all of its parent directories).  
  Note that ``conftest.py`` files from sub directories are loaded
  during test collection and not at tool startup. 

* by recursively loading all plugins specified by the 
  ``pytest_plugins`` variable in a ``conftest.py`` file 

Specifying plugins in a test module or plugin
-----------------------------------------------

You can specify plugins in a test module or a plugin like this:

.. sourcecode:: python

    pytest_plugins = "name1", "name2", 

When the test module or plugin is loaded the specified plugins
will be loaded.  If you specify plugins without the ``pytest_``
prefix it will be automatically added.  All plugin names 
must be lowercase. 

.. _`conftest.py plugin`:
.. _`conftestplugin`:

Writing per-project plugins (conftest.py)
------------------------------------------------------

The purpose of ``conftest.py`` files is to allow `project-specific 
test configuration`_.  They thus make for a good place to implement 
project-specific test related features through hooks.  For example you may 
set the `collect_ignore`_ variable depending on a command line option
by defining the following hook in a ``conftest.py`` file:

.. _`exclude-file-example`: 

.. sourcecode:: python

    # ./conftest.py in your root or package dir 
    collect_ignore = ['hello', 'test_world.py'] 
    def pytest_addoption(parser):
        parser.addoption("--runall", action="store_true", default=False)
    def pytest_configure(config):
        if config.getvalue("runall"):
            collect_ignore[:] = []

.. _`setuptools entry points`:

Writing setuptools-registered plugins 
------------------------------------------------------

.. _`Distribute`: http://pypi.python.org/pypi/distribute
.. _`setuptools`: http://pypi.python.org/pypi/setuptools

If you want to make your plugin publically available, you
can use `setuptools`_ or `Distribute`_ which both allow
to register an entry point.  ``py.test`` will register
all objects with the ``pytest11`` entry point. 
To make your plugin available you may insert the following
lines in your setuptools/distribute-based setup-invocation:

.. sourcecode:: python

    # sample ./setup.py file 
    from setuptools import setup

    setup(
        name="myproject",
        packages = ['myproject']

        # the following makes a plugin available to py.test
        entry_points = { 
            'pytest11': [
                'name_of_plugin = myproject.pluginmodule',
            ]
        },
    )

If a package is installed with this setup, py.test will load 
``myproject.pluginmodule`` under the ``name_of_plugin`` name
and use it as a plugin. 

Accessing another plugin by name
--------------------------------------------

If a plugin wants to collaborate with code from
another plugin it can obtain a reference through
the plugin manager like this:

.. sourcecode:: python

    plugin = config.pluginmanager.getplugin("name_of_plugin")

If you want to look at the names of existing plugins, use 
the ``--traceconfig`` option.

.. _`well specified hooks`: 
.. _`implement hooks`: 

Important py.test hooks 
====================================

py.test calls hooks functions to implement its `test collection`_,
running and reporting process.  When py.test loads a plugin it validates
that all hook functions conform to the `hook definition specification`_.

The hook function name and its 
argument names need to match exactly but it is allowed for an implementation
to accept *less* parameters.  You'll get useful errors on mistyped hook or 
argument names.  Read on for some introductory information on particular 
hooks.  It's sensible to look at existing plugins so see example usages 
and start off with your own plugin. 

.. _`hook definition specification`: plugin/hookspec.html

.. _`configuration hooks`: 

command line parsing and configuration hooks 
--------------------------------------------------------------------

When the `test tool starts up`_ it will invoke all hooks that add
command line options in the python standard optparse style. 

.. sourcecode:: python

    def pytest_addoption(parser): 
        """ add command line options. """" 
        parser.addoption("--myopt", dest="myopt", action="store_true")

After all these hooks have been called, the command line is parser 
and a ``config`` object is created and another hook is invoked, 
for example: 

.. sourcecode:: python

    def pytest_configure(config): 
        config.getvalue("myopt") 

When the test run finishes this corresponding finalizer hook is called:

    def pytest_unconfigure(config): 
        ... 


adding global py.test helpers and functionality 
--------------------------------------------------------------------

If you want to make global helper functions or objects available 
to your test code you can implement: 

    def pytest_namespace():
        """ return dictionary with items to be made available on py.test. namespace """

All such returned items will be made available directly on 
the ``py.test`` namespace.  

If you want to provide helpers that are specific to a test function run or need
to be setup per test function run, please refer to the `funcargs mechanism`_. 

.. _`funcargs mechanism`: funcargs.html


generic "runtest" hooks 
------------------------------

Each test item is usually executed by calling the following three hooks:

.. sourcecode:: python

    pytest_runtest_setup(item) 
    pytest_runtest_call(item) 
    pytest_runtest_teardown(item) 

For each of the three invocations a `call object`_ encapsulates
information about the outcome of the call and is subsequently used 
to make a report object: 

.. sourcecode:: python

    report = hook.pytest_runtest_makereport(item, call)

For example, the `pytest_pdb plugin`_ uses this hook to activate
interactive debugging on failures when ``--pdb`` is specified on the
command line. 

Usually three reports will be generated for a single test item for each
of the three runtest hooks respectively.  If ``pytest_runtest_setup`` 
fails then ``pytest_runtest_teardown`` will be called but not 
``pytest_runtest_call``. 

Each of the up to three reports is eventually fed to the logreport hook:

.. sourcecode:: python

    pytest_runtest_logreport(report) 

A ``report`` object contains status and reporting information:

.. sourcecode:: python

    report.longrepr = string/lines/object to print 
    report.when = "setup", "call" or "teardown" 
    report.shortrepr = letter for progress-report 
    report.passed = True or False
    report.failed = True or False
    report.skipped = True or False

The `terminal plugin`_ uses this hook to print information
about a test run. 
    
The whole protocol described here is implemented via this hook:

.. sourcecode:: python
 
    pytest_runtest_protocol(item) -> True 

.. _`call object`: 

The call object contains information about a performed call:

.. sourcecode:: python

    call.excinfo = ExceptionInfo object or None 
    call.when = "setup", "call" or "teardown" 
    call.outerr = None or tuple of strings representing captured stdout/stderr 

.. _`pytest_pdb plugin`: plugin/pdb.html
.. _`terminal plugin`: plugin/terminal.html


generic collection hooks 
------------------------------

py.test calls the following two fundamental hooks for collecting files and directories: 

.. sourcecode:: python

    def pytest_collect_directory(path, parent):
        """ return Collection node or None for the given path. """

    def pytest_collect_file(path, parent):
        """ return Collection node or None for the given path. """

Both return a `collection node`_ for a given path.  All returned 
nodes from all hook implementations will participate in the
collection and running protocol.  The ``parent`` object is
the parent node and may be used to access command line
options via the ``parent.config`` object. 


Python test function and module hooks 
----------------------------------------------------

For influencing the collection of objects in Python modules 
you can use the following hook:

.. sourcecode:: python

    def pytest_pycollect_makeitem(collector, name, obj):
        """ return custom item/collector for a python object in a module, or None.  """

This hook will be called for each Python object in a collected
Python module.  The return value is a custom `collection node`_ or None. 

.. XXX or ``False`` if you want to indicate that the given item should not be collected. 


Gateway initialization (distributed testing)
----------------------------------------------------

(alpha) For distributed testing it can be useful to prepare the
remote environment.  For this you can implement the newgateway hook:

.. sourcecode:: python

    def pytest_gwmanage_newgateway(gateway, platinfo):
        """ called after a gateway is instantiated. """

The ``gateway`` object here has a ``spec`` attribute which is an ``execnet.XSpec`` 
object, which has attributes that map key/values as specified from a ``--txspec``
option. The platinfo object is a dictionary with information about the remote process: 

* ``version``: remote python's ``sys.version_info`` 
* ``platform``: remote ``sys.platform``
* ``cwd``: remote ``os.getcwd``


.. _`collection process`: 
.. _`collection node`: 
.. _`test collection`:


Test Collection process 
====================================================== 

the collection tree 
---------------------------------

The collecting process is iterative so that distribution
and execution of tests can start as soon as the first test
item is collected.  Collection nodes with children are 
called "Collectors" and terminal nodes are called "Items".  
Here is an example of such a tree, generated with the 
command ``py.test --collectonly py/xmlobj``:: 

    <Directory 'xmlobj'>
        <Directory 'testing'>
            <Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
                <Function 'test_html_name_stickyness'>
                <Function 'test_stylenames'>
                <Function 'test_class_None'>
                <Function 'test_alternating_style'>
            <Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
                <Function 'test_tag_with_text'>
                <Function 'test_class_identity'>
                <Function 'test_tag_with_text_and_attributes'>
                <Function 'test_tag_with_subclassed_attr_simple'>
                <Function 'test_tag_nested'>
                <Function 'test_tag_xmlname'>

By default all directories not starting with a dot are traversed, 
looking for ``test_*.py`` and ``*_test.py`` files.  Those Python 
files are imported under their `package name`_. 

The Module collector looks for test functions 
and test classes and methods. Test functions and methods
are prefixed ``test`` by default.  Test classes must 
start with a capitalized ``Test`` prefix. 

.. _`package name`: 

constructing the package name for test modules
-------------------------------------------------

Test modules are imported under their fully qualified 
name.  Given a filesystem ``fspath`` it is constructed as follows:

* walk the directories up to the last one that contains 
  an ``__init__.py`` file.  

* perform ``sys.path.insert(0, basedir)``.

* import the root package as ``root``