jiuyuan
/
test_ok1
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

advance customization docs, enhance docstrings, add more reference object docs. 

--HG--
branch : trunk
This commit is contained in:
holger krekel 2010-10-11 12:54:28 +02:00
parent 431a582132
commit b5b8e5f0c2
5 changed files with 139 additions and 157 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

198
doc/customize.txt
View File

@ -256,158 +256,87 @@ initialisation, command line and configuration hooks
generic "runtest" hooks
------------------------------
Each test item is usually executed by calling the following three hooks:
Almost runtest related hooks receive a :py:class:`pytest.collect.Item` object.
.. autofunction:: pytest_runtest_protocol
.. autofunction:: pytest_runtest_setup
.. autofunction:: pytest_runtest_call
.. autofunction:: pytest_runtest_teardown
.. autofunction:: pytest_runtest_makereport
For each of the three invocations a `call object`_ encapsulates
information about the outcome of the call and is subsequently used
to make a report object:
For deeper understanding you may look at the default implementation of
these hooks in :py:mod:`pytest.plugin.runner` and maybe also
in :py:mod:`pytest.plugin.pdb` which intercepts creation
of reports in order to drop to interactive debugging.
.. sourcecode:: python
The :py:mod:`pytest.plugin.terminal` reported specifically uses
the reporting hook to print information about a test run.
report = hook.pytest_runtest_makereport(item, call)
For example, the `pytest_pdb plugin`_ uses this hook to activate
interactive debugging on failures when ``--pdb`` is specified on the
command line.
Usually three reports will be generated for a single test item for each
of the three runtest hooks respectively. If ``pytest_runtest_setup``
fails then ``pytest_runtest_teardown`` will be called but not
``pytest_runtest_call``.
Each of the up to three reports is eventually fed to the logreport hook:
.. sourcecode:: python
pytest_runtest_logreport(report)
A ``report`` object contains status and reporting information:
.. sourcecode:: python
report.longrepr = string/lines/object to print
report.when = "setup", "call" or "teardown"
report.shortrepr = letter for progress-report
report.passed = True or False
report.failed = True or False
report.skipped = True or False
The `terminal plugin`_ uses this hook to print information
about a test run.
The whole protocol described here is implemented via this hook:
.. sourcecode:: python
pytest_runtest_protocol(item) -> True
.. _`call object`:
The call object contains information about a performed call:
.. sourcecode:: python
call.excinfo = ExceptionInfo object or None
call.when = "setup", "call" or "teardown"
call.outerr = None or tuple of strings representing captured stdout/stderr
.. _`pytest_pdb plugin`: plugin/pdb.html
.. _`terminal plugin`: plugin/terminal.html
generic collection hooks
collection hooks
------------------------------
py.test calls the following two fundamental hooks for collecting files and directories:
py.test calls the following hooks for collecting files and directories:
.. sourcecode:: python
def pytest_collect_directory(path, parent):
""" return Collection node or None for the given path. """
def pytest_collect_file(path, parent):
""" return Collection node or None for the given path. """
Both return a `collection node`_ for a given path. All returned
nodes from all hook implementations will participate in the
collection and running protocol. The ``parent`` object is
the parent node and may be used to access command line
options via the ``parent.config`` object.
Python test function and module hooks
----------------------------------------------------
.. autofunction:: pytest_ignore_collect
.. autofunction:: pytest_collect_directory
.. autofunction:: pytest_collect_file
For influencing the collection of objects in Python modules
you can use the following hook:
.. sourcecode:: python
def pytest_pycollect_makeitem(collector, name, obj):
""" return custom item/collector for a python object in a module, or None. """
This hook will be called for each Python object in a collected
Python module. The return value is a custom `collection node`_ or None.
.. XXX or ``False`` if you want to indicate that the given item should not be collected.
.. autofunction:: pytest_pycollect_makeitem
Gateway initialization (distributed testing)
----------------------------------------------------
reporting hooks
------------------------------
(alpha) For distributed testing it can be useful to prepare the
remote environment. For this you can implement the newgateway hook:
Collection related reporting hooks:
.. sourcecode:: python
.. autofunction: pytest_collectstart
.. autofunction: pytest_log_itemcollect
.. autofunction: pytest_collectreport
.. autofunction: pytest_deselected
def pytest_gwmanage_newgateway(gateway, platinfo):
""" called after a gateway is instantiated. """
And here is the central hook for reporting about
test execution:
The ``gateway`` object here has a ``spec`` attribute which is an ``execnet.XSpec``
object, which has attributes that map key/values as specified from a ``--txspec``
option. The platinfo object is a dictionary with information about the remote process:
.. autofunction: pytest_runtest_logreport
* ``version``: remote python's ``sys.version_info``
* ``platform``: remote ``sys.platform``
* ``cwd``: remote ``os.getcwd``
.. _`collection process`:
.. _`collection node`:
.. _`test collection`:
Test Collection process
The test collection tree
======================================================
the collection tree
---------------------------------
The collecting process is iterative so that distribution
and execution of tests can start as soon as the first test
item is collected. Collection nodes with children are
called "Collectors" and terminal nodes are called "Items".
Here is an example of such a tree, generated with the
command ``py.test --collectonly py/xmlobj``::
Default filesystem test discovery
-----------------------------------------------
<Directory 'xmlobj'>
<Directory 'testing'>
<Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
<Function 'test_html_name_stickyness'>
<Function 'test_stylenames'>
<Function 'test_class_None'>
<Function 'test_alternating_style'>
<Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
<Function 'test_tag_with_text'>
<Function 'test_class_identity'>
<Function 'test_tag_with_text_and_attributes'>
<Function 'test_tag_with_subclassed_attr_simple'>
<Function 'test_tag_nested'>
<Function 'test_tag_xmlname'>
Test collection starts from specified paths or from the current
directory. All tests are collected ahead of running the first test.
(This used to be different in earlier versions of ``py.test`` where
collection and running was interweaved which made test randomization
and distributed testing harder).
Collection nodes which have children are called "Collectors" and otherwise
they are called "Items" or "test items". Here is an example of such a
tree::
testing $ py.test --collectonly test_parseonly.py
<Directory 'testing'>
<Module 'test_parseopt.py'>
<Class 'TestParser'>
<Instance '()'>
<Function 'test_init'>
<Function 'test_group_add_and_get'>
<Function 'test_addgroup_deprecation'>
<Function 'test_getgroup_simple'>
<Function 'test_group_ordering'>
<Function 'test_group_addoption'>
<Function 'test_group_shortopt_lowercase'>
<Function 'test_parser_addoption'>
<Function 'test_parse'>
<Function 'test_parse_will_set_default'>
<Function 'test_parse_setoption'>
<Function 'test_parse_defaultgetter'>
<Function 'test_addoption_parser_epilog'>
By default all directories not starting with a dot are traversed,
looking for ``test_*.py`` and ``*_test.py`` files. Those Python
@ -441,3 +370,18 @@ name. Given a filesystem ``fspath`` it is constructed as follows:
* perform ``sys.path.insert(0, basedir)``.
* import the root package as ``root``
Complete reference of objects involved in hooks
===========================================================
.. autoclass:: pytest.plugin.runner.CallInfo
:members:
.. autoclass:: pytest.plugin.runner.TestReport
:members:
.. autoclass:: pytest.collect.Node
:members:
.. autoclass:: pytest.collect.Item
:inherited-members:

2
doc/install.txt
View File

@ -82,7 +82,7 @@ all core features and runs unchanged under Python2 and Python3 interpreters.
.. _`Python for Windows`: http://www.imladris.com/Scripts/PythonForWindows.html
.. _`Distribute`:
.. _`Distribute for installation`: http://pypi.python.org/pypi/distribute#installation-instructions
.. _`distribute installation`: http://pypi.python.org/pypi/distribute
.. include:: links.inc

43
pytest/collect.py
View File

@ -25,13 +25,20 @@ class HookProxy:
class Node(object):
""" base class for all Nodes in the collection tree.
Collector subclasses have children, Items are terminal nodes.
"""
Collector subclasses have children, Items are terminal nodes."""
def __init__(self, name, parent=None, config=None, collection=None):
#: a unique name with the scope of the parent
self.name = name
#: the parent collector node.
self.parent = parent
self.config = config or parent.config
#: the collection this node is part of.
self.collection = collection or getattr(parent, 'collection', None)
#: the file where this item is contained/collected from.
self.fspath = getattr(parent, 'fspath', None)
self.ihook = HookProxy(self)
self.keywords = self.readkeywords()
@ -130,13 +137,8 @@ class Node(object):
repr_failure = _repr_failure_py
class Collector(Node):
"""
Collector instances create children through collect()
and thus iteratively build a tree. attributes::
parent: attribute pointing to the parent collector
(or None if this is the root collector)
name: basename of this collector object
""" Collector instances create children through collect()
and thus iteratively build a tree.
"""
Directory = configproperty('Directory')
Module = configproperty('Module')
@ -166,6 +168,15 @@ class Collector(Node):
""" internal helper method to cache results of calling collect(). """
return self._memoizedcall('_collected', self.collect)
def _prunetraceback(self, traceback):
if hasattr(self, 'fspath'):
path = self.fspath
ntraceback = traceback.cut(path=self.fspath)
if ntraceback == traceback:
ntraceback = ntraceback.cut(excludepath=py._pydir)
traceback = ntraceback.filter()
return traceback
# **********************************************************************
# DEPRECATED METHODS
# **********************************************************************
@ -197,15 +208,6 @@ class Collector(Node):
"""
return self.collect_by_name(name)
def _prunetraceback(self, traceback):
if hasattr(self, 'fspath'):
path = self.fspath
ntraceback = traceback.cut(path=self.fspath)
if ntraceback == traceback:
ntraceback = ntraceback.cut(excludepath=py._pydir)
traceback = ntraceback.filter()
return traceback
class FSCollector(Collector):
def __init__(self, fspath, parent=None, config=None, collection=None):
fspath = py.path.local(fspath)
@ -264,7 +266,10 @@ class Directory(FSCollector):
return self.ihook.pytest_collect_directory(path=path, parent=self)
class Item(Node):
""" a basic test item. """
""" a basic test invocation item. Note that for a single function
there might be multiple test invocation items. Attributes:
"""
def _deprecated_testexecution(self):
if self.__class__.run != Item.run:
warnoldtestrun(function=self.run)

29
pytest/hookspec.py
View File

@ -53,18 +53,20 @@ def pytest_log_finishcollection(collection):
""" called after collection has finished. """
def pytest_ignore_collect(path, config):
""" return true value to prevent considering this path for collection.
This hook is consulted for all files and directories prior to considering
collection hooks.
""" return True to prevent considering this path for collection.
This hook is consulted for all files and directories prior to calling
more specific hooks.
"""
pytest_ignore_collect.firstresult = True
def pytest_collect_directory(path, parent):
""" return Collection node or None for the given path. """
""" return collection Node or None for the given path. Any new node
needs to have the specified ``parent`` as a parent."""
pytest_collect_directory.firstresult = True
def pytest_collect_file(path, parent):
""" return Collection node or None for the given path. """
""" return collection Node or None for the given path. Any new node
needs to have the specified ``parent`` as a parent."""
# logging hooks for collection
def pytest_collectstart(collector):
@ -113,23 +115,30 @@ def pytest_itemstart(item, node=None):
""" (deprecated, use pytest_runtest_logstart). """
def pytest_runtest_protocol(item):
""" implement fixture, run and report about the given test item. """
""" implements the standard runtest_setup/call/teardown protocol including
capturing exceptions and calling reporting hooks on the results accordingly.
:return boolean: True if no further hook implementations should be invoked.
"""
pytest_runtest_protocol.firstresult = True
def pytest_runtest_logstart(nodeid, location, fspath):
""" signal the start of a test run. """
def pytest_runtest_setup(item):
""" called before pytest_runtest_call(). """
""" called before ``pytest_runtest_call(item)``. """
def pytest_runtest_call(item):
""" execute test item. """
""" called to execute the test ``item``. """
def pytest_runtest_teardown(item):
""" called after pytest_runtest_call(). """
""" called after ``pytest_runtest_call``. """
def pytest_runtest_makereport(item, call):
""" make a test report for the given item and call outcome. """
""" return a :py:class:`pytest.plugin.runner.TestReport` object
for the given :py:class:`pytest.collect.Item` and
:py:class:`pytest.plugin.runner.CallInfo`.
"""
pytest_runtest_makereport.firstresult = True
def pytest_runtest_logreport(report):

24
pytest/plugin/runner.py
View File

@ -113,8 +113,11 @@ def call_runtest_hook(item, when):
return CallInfo(lambda: ihook(item=item), when=when)
class CallInfo:
""" Call Information about a hook call. """
#: None or ExceptionInfo object.
excinfo = None
def __init__(self, func, when):
#: one of "setup", "call", "teardown" specifying the runtest phase.
self.when = when
try:
self.result = func()
@ -169,15 +172,36 @@ def pytest_runtest_makereport(item, call):
keywords, outcome, longrepr, when)
class TestReport(BaseReport):
""" Basic test report object (also used for setup and teardown calls if
they fail).
"""
def __init__(self, nodeid, nodenames, fspath, location,
keywords, outcome, longrepr, when):
#: normalized collection node id
self.nodeid = nodeid
#: list of names indicating position in collection tree.
self.nodenames = nodenames
#: the collected path of the file containing the test.
self.fspath = fspath # where the test was collected
#: a (filesystempath, lineno, domaininfo) tuple indicating the
#: actual location of a test item - it might be different from the
#: collected one e.g. if a method is inherited from a different module.
self.location = location
#: a name -> value dictionary containing all keywords and
#: markers associated with a test invocation.
self.keywords = keywords
#: test outcome, always one of "passed", "failed", "skipped".
self.outcome = outcome
#: None or a failure representation.
self.longrepr = longrepr
#: one of 'setup', 'call', 'teardown' to indicate runtest phase.
self.when = when
def __repr__(self):