merging funcargs docs and branch

--HG--
branch : trunk
This commit is contained in:
holger krekel 2009-04-14 22:31:37 +02:00
commit 6dd3807914
32 changed files with 787 additions and 404 deletions

View File

@ -11,6 +11,8 @@ provides a number of implementations of this API.
Path implementations provided by :api:`py.path`
===============================================
.. _`local`:
:api:`py.path.local`
--------------------

View File

@ -21,3 +21,26 @@ behind the ``--`` double-dash.
IOW, you can set default values for options per project, per
home-directoray, per shell session or per test-run.
.. _`basetemp`:
per-testrun temporary directories
-------------------------------------------
``py.test`` runs provide means to create per-test session
temporary (sub) directories. You can create such directories
like this:
.. sourcecode: python
import py
basetemp = py.test.config.ensuretemp()
basetemp_subdir = py.test.config.ensuretemp("subdir")
By default, ``py.test`` creates a ``pytest-NUMBER`` directory
and will keep around the directories of the last three
test runs. You can also set the base temporary directory
with the `--basetemp`` option. When distributing
tests on the same machine, ``py.test`` takes care to
pass around the basetemp directory such that all temporary
files land below the same basetemp directory.

View File

@ -1,7 +1,20 @@
======================================
Writing plugins and extensions
======================================
.. _`local plugin`:
Local Plugins
==================================
You can easily specify a project-specific or "local"
plugin by defining a ``ConftestPlugin`` in a ``conftest.py``
file like this::
class ConftestPlugin:
""" my local plugin. """
===============
Writing plugins
===============
Learning by examples
=====================

View File

@ -36,8 +36,19 @@ or class with a leading ``Test`` name is collected.
Rapidly write integration, functional, unit tests
===================================================
py.test provides
XXX
funcargs and xUnit style setups
===================================================
py.test provides powerful means for managing test
state and fixtures. Apart from the `traditional
xUnit style setup`_ for unittests it features the
simple and powerful `funcargs mechanism`_ for handling
both complex and simple test scenarious.
.. _`funcargs mechanism`: funcargs.html
.. _`traditional xUnit style setup`: xunit_setup.html
load-balance tests to multiple CPUs
===================================

View File

@ -1,142 +1,314 @@
======================================================
**funcargs**: powerful and simple test setup
======================================================
=====================================
Python test function arguments
=====================================
In version 1.0 py.test introduces a new mechanism for setting up test
state for use by Python test functions. It is particularly useful
for functional and integration testing but also for unit testing.
Using funcargs you can easily:
py.test enables a new way to separate test configuration
and test setup from actual test code in test functions.
When it runs a test functions it will lookup function
arguments by name and provide a value.
Here is a simple example for such a test function:
* write self-contained, simple to read and debug test functions
* cleanly encapsulate glue code between your app and your tests
* setup test state depending on command line options or environment
def test_function(mysetup):
# work with mysetup
To provide a value py.test looks for a ``pytest_funcargs``
dictionary in the test module, for example::
Using the funcargs mechanism will increase readability
and allow for easier refactoring of your application
and its test suites.
class MySetup:
def __init__(self, pyfuncitem):
self.pyfuncitem = pyfuncitem
pytest_funcargs = {'mysetup': MySetup}
.. contents:: Contents:
:depth: 2
This is already enough to run the test. Of course
up until now our ``mysetup`` does not provide
much value. But it is now easy to add new
methods on the ``MySetup`` class that have
full access to the test collection process.
The basic funcarg request/provide mechanism
=============================================
Plugins can register their funcargs via
the config object, usually upon initial configure::
To use funcargs you only need to specify
a named argument for your test function:
.. sourcecode:: python
def test_function(myarg):
# use myarg
For each test function that requests this ``myarg``
argument a matching so called funcarg provider
will be invoked. A Funcarg provider for ``myarg``
is written down liks this:
.. sourcecode:: python
def pytest_funcarg__myarg(self, request):
# return value for myarg here
Such a provider method can live on a test class,
test module or on a local or global plugin.
The method is recognized by the ``pytest_funcarg__``
prefix and is correlated to the argument
name which follows this prefix. The passed in
``request`` object allows to interact
with test configuration, test collection
and test running aspects.
.. _`request object`:
funcarg request objects
------------------------
Request objects encapsulate a request for a function argument from a
specific test function. Request objects provide access to command line
options, the underlying python function and allow interaction
with other providers and the test running process.
Attributes of request objects
++++++++++++++++++++++++++++++++++++++++
``request.argname``: name of the requested function argument
``request.function``: python function object requesting the argument
``request.fspath``: filesystem path of containing module
``request.config``: access to command line opts and general config
finalizing after test function executed
++++++++++++++++++++++++++++++++++++++++
Request objects allow to **register a finalizer method** which is
called after a test function has finished running.
This is useful for tearing down or cleaning up
test state. Here is a basic example for providing
a ``myfile`` object that will be closed upon test
function finish:
.. sourcecode:: python
def pytest_funcarg__myfile(self, request):
# ... create and open a "myfile" object ...
request.addfinalizer(lambda: myfile.close())
return myfile
a unique temporary directory
++++++++++++++++++++++++++++++++++++++++
request objects allow to create unique temporary
directories. These directories will be created
as subdirectories under the `per-testsession
temporary directory`_. Each request object
receives its own unique subdirectory whose
basenames starts with the name of the function
that triggered the funcarg request. You
can further work with the provided `py.path.local`_
object to e.g. create subdirs or config files::
def pytest_funcarg__mysetup(self, request):
tmpdir = request.maketempdir()
tmpdir.mkdir("mysubdir")
tmpdir.join("config.ini").write("[default")
return tmpdir
Note that you do not need to perform finalization,
i.e. remove the temporary directory as this is
part of the global management of the base temporary
directory.
.. _`per-testsession temporary directory`: config.html#basetemp
decorating/adding to existing funcargs
++++++++++++++++++++++++++++++++++++++++
If you want to **decorate a function argument** that is
provided elsewhere you can ask the request object
to provide the "next" value:
.. sourcecode:: python
def pytest_funcarg__myfile(self, request):
myfile = request.call_next_provider()
# do something extra
return myfile
This will raise a ``request.Error`` exception if there
is no next provider left. See the `decorator example`_
for a use of this method.
.. _`lookup order`:
Order of funcarg provider lookup
----------------------------------------
For any funcarg argument request here is the
lookup order for provider methods:
1. test class (if we are executing a method)
2. test module
3. local plugins
4. global plugins
Using multiple funcargs
----------------------------------------
A test function may receive more than one
function arguments. For each of the
function arguments a lookup of a
matching provider will be performed.
Funcarg Examples
=====================
Example: basic application specific setup
-----------------------------------------------------
Here is a basic useful example for handling application
specific setup. The goal is to have one place where
we have the glue code for bootstrapping and configuring
application objects and allow test modules and
test functions to stay ignorant of involved details.
Let's start with the using side and consider a simple
test function living in a test file ``test_sample.py``:
.. sourcecode:: python
def test_answer(mysetup):
app = mysetup.myapp()
answer = app.question()
assert answer == 42
To run this test py.test looks up and calls a provider to obtain the
required ``mysetup`` function argument. The test function simply
interacts with the provided application specific setup.
To provide the ``mysetup`` function argument we write down
a provider method in a `local plugin`_ by putting the
following code into a local ``conftest.py``:
.. sourcecode:: python
from myapp import MyApp
class ConftestPlugin:
def pytest_configure(self, config):
config.register_funcargs(mysetup=MySetup)
def pytest_funcarg__mysetup(self, request):
return MySetup()
class MySetup:
def myapp(self):
return MyApp()
The ``pytest_funcarg__mysetup`` method is called to
provide a value for the requested ``mysetup`` test function argument.
To complete the example we put a pseudo MyApp object
into ``myapp.py``:
.. sourcecode:: python
class MyApp:
def question(self):
return 6 * 9
You can now run the test with ``py.test test_sample.py`` which will
show this failure:
.. sourcecode:: python
def test_answer(mysetup):
app = mysetup.myapp()
answer = app.question()
> assert answer == 42
E assert 54 == 42
If you are confused as to that the question or answer is
here, please visit here_.
.. _here: http://uncyclopedia.wikia.com/wiki/The_Hitchhiker's_Guide_to_the_Galaxy
.. _`local plugin`: ext.html#local-plugin
Example: specifying funcargs in test modules or classes
---------------------------------------------------------
.. sourcecode:: python
def pytest_funcarg__mysetup(request):
result = request.call_next_provider()
result.extra = "..."
return result
You can also put such a function into a test class like this:
.. sourcecode:: python
class TestClass:
def pytest_funcarg__mysetup(self, request):
# ...
#
Example: command line option for providing SSH-host
-----------------------------------------------------------
If you provide a "funcarg" from a plugin you can
easily make methods depend on command line options
or environment settings. Here is a complete
example that allows to run tests involving
an SSH connection if an ssh host is specified::
an SSH connection if an ssh host is specified:
.. sourcecode:: python
class ConftestPlugin:
def pytest_addoption(self, parser):
parser.addoption("--ssh", action="store", default=None,
help="specify ssh host to run tests with")
def pytest_configure(self, config):
config.register_funcargs(mysetup=MySetup)
pytest_funcarg__mysetup = MySetupFuncarg
class MySetup:
def __init__(self, pyfuncitem):
self.pyfuncitem = pyfuncitem
class MySetupFuncarg:
def __init__(self, request):
self.request = request
def ssh_gateway(self):
host = pyfuncitem.config.option.ssh
host = self.request.config.option.ssh
if host is None:
py.test.skip("specify ssh host with --ssh to run this test")
return py.execnet.SshGateway(host)
Now any test functions can use the "mysetup" object, for example::
Now any test functions can use the "mysetup.ssh_gateway()" method like this:
.. sourcecode:: python
class TestClass:
def test_function(self, mysetup):
ssh_gw = mysetup.ssh_gateway()
# work with ssh_gw
Without specifying a command line option the output looks like this::
Running this without the command line will yield this run result::
...
Lookup rules
======================
.. _`accept example`:
In order to run this test function a value for the
``mysetup`` needs to be found. Here is how py.test
finds a matching provider function:
example: specifying and selecting acceptance tests
--------------------------------------------------------------
1. see if there is a ``pytest_funcargs`` dictionary
which maps ``mysetup`` to a provider function.
if so, call the provider function.
.. sourcecode:: python
XXX
example
=====================
You can run a test file ``test_some.py`` with this content:
pytest_funcargs = {'myarg': (lambda pyfuncitem: 42)}
def test_something(myarg):
assert myarg == 42
You can also put this on a class:
class TestClass:
pytest_funcargs = {'myarg': (lambda pyfuncitem: 42)}
def test_something(self, myarg):
assert myarg == 42
To separate funcarg setup you can also put a funcarg
definition into a conftest.py::
pytest_funcargs = {'myarg': decorate_myarg}
def decorate_myarg(pyfuncitem):
result = pyfuncitem.call_next_provider()
return result + 1
for registering funcargs from a plugin, talk to the
test configuration object like this::
class MyPlugin:
def pytest_configure(self, config):
config.register_funcargs(
myarg=decorate_myarg
)
a local helper funcarg for doing acceptance tests maybe
by running shell commands could look like this::
class MyPlugin:
class ConftestPlugin:
def pytest_option(self, parser):
group = parser.addgroup("myproject acceptance tests")
group = parser.getgroup("myproject")
group.addoption("-A", dest="acceptance", action="store_true",
help="run (slow) acceptance tests")
def pytest_configure(self, config):
config.register_funcargs(accept=AcceptFuncarg)
def pytest_funcarg__accept(self, request):
return AcceptFuncarg(request)
class AcceptFuncarg:
def __init__(self, pyfuncitem):
if not pyfuncitem.config.option.acceptance:
def __init__(self, request):
if not request.config.option.acceptance:
py.test.skip("specify -A to run acceptance tests")
self.tmpdir = pyfuncitem.config.maketempdir(pyfuncitem.name)
self.tmpdir = request.config.maketempdir(request.argname)
self._old = self.tmpdir.chdir()
pyfuncitem.addfinalizer(self.finalize)
request.addfinalizer(self.finalize)
def run(self):
return py.process.cmdexec("echo hello")
@ -144,17 +316,72 @@ by running shell commands could look like this::
def finalize(self):
self._old.chdir()
# cleanup any other resources
and the actual test function example:
.. sourcecode:: python
def test_some_acceptance_aspect(accept):
accept.tmpdir.mkdir("somesub")
result = accept.run()
assert result
for registering funcargs from a plugin, talk to the
test configuration object like this::
That's it! This test will get automatically skipped with
an appropriate message if you just run ``py.test``::
XXX
... OUTPUT of py.test on this example ...
.. _`decorator example`:
example: decorating/extending a funcarg in a TestClass
--------------------------------------------------------------
For larger scale setups it's sometimes useful to decorare
a funcarg just for a particular test module or even
a particular test class. We can extend the `accept example`_
by putting this in our test class:
.. sourcecode:: python
class TestSpecialAcceptance:
def pytest_funcarg__accept(self, request):
arg = request.call_next_provider()
# create a special layout in our tempdir
arg.tmpdir.mkdir("special")
return arg
def test_sometest(self, accept):
assert accept.tmpdir.join("special").check()
According to the `lookup order`_ our class-specific provider will
be invoked first. Here, we just ask our request object to
call the next provider and decorate its result. This simple
mechanism allows us to stay ignorant of how/where the
function argument is provided.
Note that we make use here of `py.path.local`_ objects
that provide uniform access to the local filesystem.
.. _`py.path.local`: ../path.html#local
Questions and Answers
==================================
.. _`why pytest_pyfuncarg__ methods?`:
Why ``pytest_funcarg__*`` methods?
------------------------------------
When experimenting with funcargs we also considered an explicit
registration mechanism, i.e. calling a register method e.g. on the
config object. But lacking a good use case for this indirection and
flexibility we decided to go for `Convention over Configuration`_
and allow to directly specify the provider. It has the
positive implication that you should be able to
"grep" for ``pytest_funcarg__MYARG`` and will find all
providing sites (usually exactly one).
.. _`Convention over Configuration`: http://en.wikipedia.org/wiki/Convention_over_Configuration

View File

@ -1,65 +0,0 @@
=================
Managing state
=================
funcargs: provding arguments for test functions
===========================================================
XXX write docs
Managing test state across test modules, classes and methods
============================================================
Often you want to create some files, database connections or other
state in order to run tests in a certain environment. With
``py.test`` there are three scopes for which you can provide hooks to
manage such state. Again, ``py.test`` will detect these hooks in
modules on a name basis. The following module-level hooks will
automatically be called by the session::
def setup_module(module):
""" setup up any state specific to the execution
of the given module.
"""
def teardown_module(module):
""" teardown any state that was previously setup
with a setup_module method.
"""
The following hooks are available for test classes::
def setup_class(cls):
""" setup up any state specific to the execution
of the given class (which usually contains tests).
"""
def teardown_class(cls):
""" teardown any state that was previously setup
with a call to setup_class.
"""
def setup_method(self, method):
""" setup up any state tied to the execution of the given
method in a class. setup_method is invoked for every
test method of a class.
"""
def teardown_method(self, method):
""" teardown any state that was previously setup
with a setup_method call.
"""
The last two hooks, ``setup_method`` and ``teardown_method``, are
equivalent to ``setUp`` and ``tearDown`` in the Python standard
library's ``unitest`` module.
All setup/teardown methods are optional. You could have a
``setup_module`` but no ``teardown_module`` and the other way round.
Note that while the test session guarantees that for every ``setup`` a
corresponding ``teardown`` will be invoked (if it exists) it does
*not* guarantee that any ``setup`` is called only happens once. For
example, the session might decide to call the ``setup_module`` /
``teardown_module`` pair more than once during the execution of a test
module.

73
doc/test/xunit_setup.txt Normal file
View File

@ -0,0 +1,73 @@
====================================
xUnit style setup
====================================
.. _`funcargs`: funcargs.html
.. _`xUnit`: http://en.wikipedia.org/wiki/XUnit
Note:
Since version 1.0 py.test offers funcargs_ for both
simple and complex test setup needs. Especially
for functional and integration, but also for unit testing, it is
highly recommended that you use this new method.
Python, Java and other languages have a tradition
of using xUnit_ style testing. This typically
involves the call of a ``setup`` method before
a test function is run and ``teardown`` after
it finishes. With ``py.test`` there are three
scopes for which you can provide setup/teardown
hooks to provide test fixtures: per-module, per-class
and per-method/function. ``py.test`` will
discover and call according methods automatically.
All setup/teardown methods are optional.
The following methods are called at module level if they exist:
.. sourcecode:: python
def setup_module(module):
""" setup up any state specific to the execution
of the given module.
"""
def teardown_module(module):
""" teardown any state that was previously setup
with a setup_module method.
"""
The following hooks are available for test classes:
.. sourcecode:: python
def setup_class(cls):
""" setup up any state specific to the execution
of the given class (which usually contains tests).
"""
def teardown_class(cls):
""" teardown any state that was previously setup
with a call to setup_class.
"""
def setup_method(self, method):
""" setup up any state tied to the execution of the given
method in a class. setup_method is invoked for every
test method of a class.
"""
def teardown_method(self, method):
""" teardown any state that was previously setup
with a setup_method call.
"""
The last two hooks, ``setup_method`` and ``teardown_method``, are
equivalent to ``setUp`` and ``tearDown`` in the Python standard
library's `unittest.py module`_.
Note that it possible that setup/teardown pairs are invoked multiple
times per testing process.
.. _`unittest.py module`: http://docs.python.org/library/unittest.html

View File

@ -0,0 +1,10 @@
from myapp import MyApp
class ConftestPlugin:
def pytest_funcarg__mysetup(self, request):
return MySetup()
class MySetup:
def myapp(self):
return MyApp()

View File

@ -0,0 +1,5 @@
class MyApp:
def question(self):
return 6 * 9

View File

@ -0,0 +1,5 @@
def test_answer(mysetup):
app = mysetup.myapp()
answer = app.question()
assert answer == 42

View File

@ -90,9 +90,7 @@ class Registry:
l = []
if plugins is None:
plugins = self._plugins
if extra:
plugins += list(extra)
for plugin in plugins:
for plugin in list(plugins) + list(extra):
try:
l.append(getattr(plugin, attrname))
except AttributeError:

View File

@ -5,10 +5,10 @@ rsyncignore = ['c-extension/greenlet/build']
import py
class PylibTestconfigPlugin:
def pytest_funcarg__specssh(self, pyfuncitem):
return getspecssh(pyfuncitem.config)
def pytest_funcarg__specsocket(self, pyfuncitem):
return getsocketspec(pyfuncitem.config)
def pytest_funcarg__specssh(self, request):
return getspecssh(request.config)
def pytest_funcarg__specsocket(self, request):
return getsocketspec(request.config)
def pytest_addoption(self, parser):
group = parser.addgroup("pylib", "py lib testing options")

View File

@ -135,6 +135,12 @@ class TestRegistry:
l = list(plugins.listattr('x', reverse=True))
assert l == [43, 42, 41]
class api4:
x = 44
l = list(plugins.listattr('x', extra=(api4,)))
assert l == range(41, 45)
assert len(list(plugins)) == 3 # otherwise extra added
def test_api_and_defaults():
assert isinstance(py._com.comregistry, Registry)

View File

@ -1,28 +1,25 @@
""" RSync filter test
"""
import py
from py.__.test.dist.nodemanage import NodeManager
def pytest_funcarg__source(pyfuncitem):
return py.test.ensuretemp(pyfuncitem.getmodpath()).mkdir("source")
def pytest_funcarg__dest(pyfuncitem):
dest = py.test.ensuretemp(pyfuncitem.getmodpath()).mkdir("dest")
return dest
class pytest_funcarg__mysetup:
def __init__(self, request):
basetemp = request.maketempdir()
basetemp = basetemp.mkdir(request.function.__name__)
self.source = basetemp.mkdir("source")
self.dest = basetemp.mkdir("dest")
class TestNodeManager:
@py.test.mark.xfail("consider / forbid implicit rsyncdirs?")
def test_rsync_roots_no_roots(self, source, dest):
source.ensure("dir1", "file1").write("hello")
def test_rsync_roots_no_roots(self, mysetup):
mysetup.source.ensure("dir1", "file1").write("hello")
config = py.test.config._reparse([source])
nodemanager = NodeManager(config, ["popen//chdir=%s" % dest])
nodemanager = NodeManager(config, ["popen//chdir=%s" % mysetup.dest])
assert nodemanager.config.topdir == source == config.topdir
nodemanager.rsync_roots()
p, = nodemanager.gwmanager.multi_exec("import os ; channel.send(os.getcwd())").receive_each()
p = py.path.local(p)
print "remote curdir", p
assert p == dest.join(config.topdir.basename)
assert p == mysetup.dest.join(config.topdir.basename)
assert p.join("dir1").check()
assert p.join("dir1", "file1").check()
@ -33,8 +30,9 @@ class TestNodeManager:
nodemanager.setup_nodes([].append)
nodemanager.wait_nodesready(timeout=2.0)
def test_popen_rsync_subdir(self, testdir, source, dest):
dir1 = source.mkdir("dir1")
def test_popen_rsync_subdir(self, testdir, mysetup):
source, dest = mysetup.source, mysetup.dest
dir1 = mysetup.source.mkdir("dir1")
dir2 = dir1.mkdir("dir2")
dir2.ensure("hello")
for rsyncroot in (dir1, source):
@ -53,7 +51,8 @@ class TestNodeManager:
assert dest.join("dir1", "dir2", 'hello').check()
nodemanager.gwmanager.exit()
def test_init_rsync_roots(self, source, dest):
def test_init_rsync_roots(self, mysetup):
source, dest = mysetup.source, mysetup.dest
dir2 = source.ensure("dir1", "dir2", dir=1)
source.ensure("dir1", "somefile", dir=1)
dir2.ensure("hello")
@ -68,7 +67,8 @@ class TestNodeManager:
assert not dest.join("dir1").check()
assert not dest.join("bogus").check()
def test_rsyncignore(self, source, dest):
def test_rsyncignore(self, mysetup):
source, dest = mysetup.source, mysetup.dest
dir2 = source.ensure("dir1", "dir2", dir=1)
dir5 = source.ensure("dir5", "dir6", "bogus")
dirf = source.ensure("dir5", "file")
@ -86,7 +86,8 @@ class TestNodeManager:
assert dest.join("dir5","file").check()
assert not dest.join("dir6").check()
def test_optimise_popen(self, source, dest):
def test_optimise_popen(self, mysetup):
source, dest = mysetup.source, mysetup.dest
specs = ["popen"] * 3
source.join("conftest.py").write("rsyncdirs = ['a']")
source.ensure('a', dir=1)
@ -97,7 +98,8 @@ class TestNodeManager:
assert gwspec._samefilesystem()
assert not gwspec.chdir
def test_setup_DEBUG(self, source, testdir):
def test_setup_DEBUG(self, mysetup, testdir):
source = mysetup.source
specs = ["popen"] * 2
source.join("conftest.py").write("rsyncdirs = ['a']")
source.ensure('a', dir=1)

View File

@ -31,8 +31,8 @@ class EventQueue:
print str(kwargs["excrepr"])
class MySetup:
def __init__(self, pyfuncitem):
self.pyfuncitem = pyfuncitem
def __init__(self, request):
self.request = request
def geteventargs(self, eventname, timeout=2.0):
eq = EventQueue(self.config.pluginmanager, self.queue)
@ -55,17 +55,11 @@ class MySetup:
print "exiting:", gw
gw.exit()
def pytest_funcarg__mysetup(pyfuncitem):
mysetup = MySetup(pyfuncitem)
def pytest_funcarg__mysetup(request):
mysetup = MySetup(request)
#pyfuncitem.addfinalizer(mysetup.finalize)
return mysetup
def pytest_funcarg__testdir(__call__, pyfuncitem):
# decorate to make us always change to testdir
testdir = __call__.execute(firstresult=True)
testdir.chdir()
return testdir
def test_node_hash_equality(mysetup):
node = mysetup.makenode()
node2 = mysetup.makenode()

View File

@ -1,19 +1,19 @@
import py
class _pytestPlugin:
def pytest_funcarg___pytest(self, pyfuncitem):
return PytestArg(pyfuncitem)
def pytest_funcarg___pytest(self, request):
return PytestArg(request)
class PytestArg:
def __init__(self, pyfuncitem):
self.pyfuncitem = pyfuncitem
def __init__(self, request):
self.request = request
def getcallrecorder(self, apiclass, comregistry=None):
if comregistry is None:
comregistry = self.pyfuncitem.config.pluginmanager.comregistry
comregistry = self.request.config.pluginmanager.comregistry
callrecorder = CallRecorder(comregistry)
callrecorder.start_recording(apiclass)
self.pyfuncitem.addfinalizer(callrecorder.finalize)
self.request.addfinalizer(callrecorder.finalize)
return callrecorder

View File

@ -22,7 +22,6 @@ class DefaultPlugin:
rep = runner.ItemTestReport(item, excinfo, "execute", outerr)
item.config.api.pytest_itemtestreport(rep=rep)
# XXX make this access pyfuncitem.args or funcargs
def pytest_pyfunc_call(self, pyfuncitem, args, kwargs):
pyfuncitem.obj(*args, **kwargs)

View File

@ -2,14 +2,14 @@ import py
class IocapturePlugin:
""" capture sys.stdout/sys.stderr / fd1/fd2. """
def pytest_funcarg__stdcapture(self, pyfuncitem):
def pytest_funcarg__stdcapture(self, request):
capture = Capture(py.io.StdCapture)
pyfuncitem.addfinalizer(capture.finalize)
request.addfinalizer(capture.finalize)
return capture
def pytest_funcarg__stdcapturefd(self, pyfuncitem):
def pytest_funcarg__stdcapturefd(self, request):
capture = Capture(py.io.StdCaptureFD)
pyfuncitem.addfinalizer(capture.finalize)
request.addfinalizer(capture.finalize)
return capture
class Capture:

View File

@ -2,9 +2,9 @@ import os
class MonkeypatchPlugin:
""" setattr-monkeypatching with automatical reversal after test. """
def pytest_funcarg__monkeypatch(self, pyfuncitem):
def pytest_funcarg__monkeypatch(self, request):
monkeypatch = MonkeyPatch()
pyfuncitem.addfinalizer(monkeypatch.finalize)
request.addfinalizer(monkeypatch.finalize)
return monkeypatch
notset = object()

View File

@ -6,37 +6,23 @@ from py.__.test.plugin import api
class PlugintesterPlugin:
""" test support code for testing pytest plugins. """
def pytest_funcarg__plugintester(self, pyfuncitem):
pt = PluginTester(pyfuncitem)
pyfuncitem.addfinalizer(pt.finalize)
return pt
def pytest_funcarg__plugintester(self, request):
return PluginTester(request)
class Support(object):
def __init__(self, pyfuncitem):
""" instantiated per function that requests it. """
self.pyfuncitem = pyfuncitem
class PluginTester:
def __init__(self, request):
self.request = request
def getmoditem(self):
for colitem in self.pyfuncitem.listchain():
if isinstance(colitem, colitem.Module):
return colitem
def finalize(self):
""" called after test function finished execution"""
class PluginTester(Support):
def testdir(self):
# XXX import differently, eg.
# FSTester = self.pyfuncitem.config.pluginmanager.getpluginattr("pytester", "FSTester")
from pytest_pytester import TmpTestdir
crunner = TmpTestdir(self.pyfuncitem)
self.pyfuncitem.addfinalizer(crunner.finalize)
crunner = TmpTestdir(self.request)
self.request.addfinalizer(crunner.finalize)
#
for colitem in self.pyfuncitem.listchain():
if isinstance(colitem, py.test.collect.Module) and \
colitem.name.startswith("pytest_"):
crunner.plugins.append(colitem.fspath.purebasename)
break
#for colitem in self.request.listchain():
# if isinstance(colitem, py.test.collect.Module) and \
# colitem.name.startswith("pytest_"):
# crunner.plugins.append(colitem.fspath.purebasename)
# break
return crunner
def apicheck(self, pluginclass):

View File

@ -11,19 +11,19 @@ import api
class PytesterPlugin:
def pytest_funcarg__linecomp(self, pyfuncitem):
def pytest_funcarg__linecomp(self, request):
return LineComp()
def pytest_funcarg__LineMatcher(self, pyfuncitem):
def pytest_funcarg__LineMatcher(self, request):
return LineMatcher
def pytest_funcarg__testdir(self, pyfuncitem):
tmptestdir = TmpTestdir(pyfuncitem)
def pytest_funcarg__testdir(self, request):
tmptestdir = TmpTestdir(request)
return tmptestdir
def pytest_funcarg__eventrecorder(self, pyfuncitem):
def pytest_funcarg__eventrecorder(self, request):
evrec = EventRecorder(py._com.comregistry)
pyfuncitem.addfinalizer(lambda: evrec.comregistry.unregister(evrec))
request.addfinalizer(lambda: evrec.comregistry.unregister(evrec))
return evrec
def test_generic(plugintester):
@ -38,11 +38,11 @@ class RunResult:
self.stderr = LineMatcher(errlines)
class TmpTestdir:
def __init__(self, pyfuncitem):
self.pyfuncitem = pyfuncitem
def __init__(self, request):
self.request = request
# XXX remove duplication with tmpdir plugin
basetmp = pyfuncitem.config.ensuretemp("testdir")
name = pyfuncitem.name
basetmp = request.config.ensuretemp("testdir")
name = request.function.__name__
for i in range(100):
try:
tmpdir = basetmp.mkdir(name + str(i))
@ -57,7 +57,7 @@ class TmpTestdir:
self._syspathremove = []
self.chdir() # always chdir
assert hasattr(self, '_olddir')
self.pyfuncitem.addfinalizer(self.finalize)
self.request.addfinalizer(self.finalize)
def __repr__(self):
return "<TmpTestdir %r>" % (self.tmpdir,)
@ -78,7 +78,7 @@ class TmpTestdir:
sorter.callrecorder = CallRecorder(registry)
sorter.callrecorder.start_recording(api.PluginHooks)
sorter.api = sorter.callrecorder.api
self.pyfuncitem.addfinalizer(sorter.callrecorder.finalize)
self.request.addfinalizer(sorter.callrecorder.finalize)
return sorter
def chdir(self):
@ -90,7 +90,7 @@ class TmpTestdir:
items = kwargs.items()
if args:
source = "\n".join(map(str, args))
basename = self.pyfuncitem.name
basename = self.request.function.__name__
items.insert(0, (basename, source))
ret = None
for name, value in items:
@ -139,7 +139,7 @@ class TmpTestdir:
# used from runner functional tests
item = self.getitem(source)
# the test class where we are called from wants to provide the runner
testclassinstance = self.pyfuncitem.obj.im_self
testclassinstance = self.request.function.im_self
runner = testclassinstance.getrunner()
return runner(item, **runnerargs)
@ -200,7 +200,7 @@ class TmpTestdir:
return self.config.getfsnode(path)
def getmodulecol(self, source, configargs=(), withinit=False):
kw = {self.pyfuncitem.name: py.code.Source(source).strip()}
kw = {self.request.function.__name__: py.code.Source(source).strip()}
path = self.makepyfile(**kw)
if withinit:
self.makepyfile(__init__ = "#")
@ -455,3 +455,10 @@ class LineMatcher:
return extralines
def test_parseconfig(testdir):
config1 = testdir.parseconfig()
config2 = testdir.parseconfig()
assert config2 != config1
assert config1 != py.test.config

View File

@ -382,10 +382,15 @@ class TestApigenLinkRole:
"resolve_linkrole('source', 'py/foo/bar.py')")
def pytest_funcarg__testdir(__call__, pyfuncitem):
testdir = __call__.execute(firstresult=True)
def pytest_funcarg__testdir(request):
testdir = request.call_next_provider()
testdir.makepyfile(confrest="from py.__.misc.rest import Project")
testdir.plugins.append(RestdocPlugin())
count = 0
for p in testdir.plugins:
if isinstance(p, RestdocPlugin):
count += 1
assert count < 2
return testdir
class TestDoctest:

View File

@ -1,4 +1,3 @@
import uuid
import py
from pytest_resultlog import ResultLog
@ -65,7 +64,7 @@ class JSONResultArchive(object):
self._flush()
def append_data(self, data):
runid = uuid.uuid4()
runid = py.std.uuid.uuid4()
for item in data:
item = item.copy()
item['runid'] = str(runid)
@ -100,7 +99,7 @@ class SQLiteResultArchive(object):
def append_data(self, data):
flat_data = []
runid = uuid.uuid4()
runid = py.std.uuid.uuid4()
for item in data:
item = item.copy()
item['runid'] = str(runid)

View File

@ -157,6 +157,7 @@ class TestWithFunctionIntegration:
# ignorant regarding formatting details.
def getresultlog(self, testdir, arg):
resultlog = testdir.tmpdir.join("resultlog")
testdir.plugins.append("resultlog")
args = ["--resultlog=%s" % resultlog] + [arg]
testdir.runpytest(*args)
return filter(None, resultlog.readlines(cr=0))
@ -166,7 +167,6 @@ class TestWithFunctionIntegration:
ok = testdir.makepyfile(test_collection_ok="")
skip = testdir.makepyfile(test_collection_skip="import py ; py.test.skip('hello')")
fail = testdir.makepyfile(test_collection_fail="XXX")
lines = self.getresultlog(testdir, ok)
assert not lines
@ -226,6 +226,7 @@ class TestWithFunctionIntegration:
def test_generic(plugintester, LineMatcher):
plugintester.apicheck(ResultlogPlugin)
testdir = plugintester.testdir()
testdir.plugins.append("resultlog")
testdir.makepyfile("""
import py
def test_pass():

View File

@ -13,9 +13,9 @@ class TmpdirPlugin:
""" provide temporary directories to test functions and methods.
"""
def pytest_funcarg__tmpdir(self, pyfuncitem):
name = pyfuncitem.name
return pyfuncitem.config.mktemp(name, numbered=True)
def pytest_funcarg__tmpdir(self, request):
name = request.function.__name__
return request.config.mktemp(name, numbered=True)
# ===============================================================================
#
@ -29,7 +29,7 @@ def test_generic(plugintester):
def test_funcarg(testdir):
item = testdir.getitem("def test_func(tmpdir): pass")
plugin = TmpdirPlugin()
p = plugin.pytest_funcarg__tmpdir(item)
p = plugin.pytest_funcarg__tmpdir(item.getrequest("tmpdir"))
assert p.check()
bn = p.basename.strip("0123456789-")
assert bn.endswith("test_func")

View File

@ -18,6 +18,7 @@ class PluginManager(object):
def register(self, plugin):
self.api.pytest_plugin_registered(plugin=plugin)
import types
self.comregistry.register(plugin)
def unregister(self, plugin):
@ -79,8 +80,8 @@ class PluginManager(object):
for x in self.comregistry.listattr(attrname):
return x
def listattr(self, attrname, plugins=None):
return self.comregistry.listattr(attrname, plugins=plugins)
def listattr(self, attrname, plugins=None, extra=()):
return self.comregistry.listattr(attrname, plugins=plugins, extra=extra)
def call_firstresult(self, *args, **kwargs):
return self.comregistry.call_firstresult(*args, **kwargs)

View File

@ -177,7 +177,7 @@ class Module(py.test.collect.File, PyCollectorMixin):
#print "*" * 20, "INVOKE assertion", self
py.magic.invoke(assertion=1)
mod = self.obj
self.config.pluginmanager.register(mod)
#self.config.pluginmanager.register(mod)
if hasattr(mod, 'setup_module'):
self.obj.setup_module(mod)
@ -187,7 +187,7 @@ class Module(py.test.collect.File, PyCollectorMixin):
if not self.config.option.nomagic:
#print "*" * 20, "revoke assertion", self
py.magic.revoke(assertion=1)
self.config.pluginmanager.unregister(self.obj)
#self.config.pluginmanager.unregister(self.obj)
class Class(PyCollectorMixin, py.test.collect.Collector):
@ -365,38 +365,15 @@ class Function(FunctionMixin, py.test.collect.Item):
for i, argname in py.builtin.enumerate(argnames):
if i < startindex:
continue
request = self.getrequest(argname)
try:
self.funcargs[argname] = self.lookup_onearg(argname)
except LookupError, e:
self.funcargs[argname] = request.call_next_provider()
except request.Error:
numdefaults = len(funcobj.func_defaults or ())
if i + numdefaults >= len(argnames):
continue # continue # seems that our args have defaults
continue # our args have defaults XXX issue warning?
else:
raise
def lookup_onearg(self, argname):
prefix = "pytest_funcarg__"
#makerlist = self.config.pluginmanager.listattr(prefix + argname)
value = self.config.pluginmanager.call_firstresult(prefix + argname, pyfuncitem=self)
if value is not None:
return value
else:
self._raisefuncargerror(argname, prefix)
def _raisefuncargerror(self, argname, prefix="pytest_funcarg__"):
metainfo = self.repr_metainfo()
available = []
plugins = list(self.config.pluginmanager.comregistry)
#plugins.extend(self.config.pluginmanager.registry.plugins)
for plugin in plugins:
for name in vars(plugin.__class__):
if name.startswith(prefix):
name = name[len(prefix):]
if name not in available:
available.append(name)
msg = "funcargument %r not found for: %s" %(argname,metainfo.verboseline())
msg += "\n available funcargs: %s" %(", ".join(available),)
raise LookupError(msg)
request._raiselookupfailed()
def __eq__(self, other):
try:
@ -410,6 +387,70 @@ class Function(FunctionMixin, py.test.collect.Item):
def __ne__(self, other):
return not self == other
def getrequest(self, argname):
return FuncargRequest(pyfuncitem=self, argname=argname)
# DEPRECATED
#from py.__.test.plugin.pytest_doctest import DoctestFile
class FuncargRequest:
_argprefix = "pytest_funcarg__"
class Error(LookupError):
""" error on performing funcarg request. """
def __init__(self, pyfuncitem, argname):
# XXX make pyfuncitem _pyfuncitem
self._pyfuncitem = pyfuncitem
self.argname = argname
self.function = pyfuncitem.obj
self.config = pyfuncitem.config
self.fspath = pyfuncitem.fspath
self._plugins = self._getplugins()
self._methods = self.config.pluginmanager.listattr(
plugins=self._plugins,
attrname=self._argprefix + str(argname)
)
def __repr__(self):
return "<FuncargRequest %r for %r>" %(self.argname, self._pyfuncitem)
def _getplugins(self):
plugins = []
current = self._pyfuncitem
while not isinstance(current, Module):
current = current.parent
if isinstance(current, (Instance, Module)):
plugins.insert(0, current.obj)
return self.config.pluginmanager.getplugins() + plugins
def call_next_provider(self):
if not self._methods:
raise self.Error("no provider methods left")
nextmethod = self._methods.pop()
return nextmethod(request=self)
def addfinalizer(self, finalizer):
self._pyfuncitem.addfinalizer(finalizer)
def maketempdir(self):
basetemp = self.config.getbasetemp()
tmp = py.path.local.make_numbered_dir(
prefix=self.function.__name__ + "_",
keep=0, rootdir=basetemp)
return tmp
def _raiselookupfailed(self):
available = []
for plugin in self._plugins:
for name in vars(plugin.__class__):
if name.startswith(self._argprefix):
name = name[len(self._argprefix):]
if name not in available:
available.append(name)
metainfo = self._pyfuncitem.repr_metainfo()
msg = "funcargument %r not found for: %s" %(self.argname,metainfo.verboseline())
msg += "\n available funcargs: %s" %(", ".join(available),)
raise LookupError(msg)

View File

@ -1,2 +1,3 @@
pytest_plugins = "pytest_xfail", "pytest_pytester", "pytest_tmpdir"

View File

@ -0,0 +1,130 @@
import py
class TestFuncargs:
def test_funcarg_lookupfails(self, testdir):
testdir.makeconftest("""
class ConftestPlugin:
def pytest_funcarg__xyzsomething(self, request):
return 42
""")
item = testdir.getitem("def test_func(some): pass")
exc = py.test.raises(LookupError, "item.setupargs()")
s = str(exc.value)
assert s.find("xyzsomething") != -1
def test_funcarg_lookup_default(self, testdir):
item = testdir.getitem("def test_func(some, other=42): pass")
class Provider:
def pytest_funcarg__some(self, request):
return request.function.__name__
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 1
def test_funcarg_lookup_default_gets_overriden(self, testdir):
item = testdir.getitem("def test_func(some=42, other=13): pass")
class Provider:
def pytest_funcarg__other(self, request):
return request.function.__name__
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 1
name, value = item.funcargs.popitem()
assert name == "other"
assert value == item.name
def test_funcarg_basic(self, testdir):
item = testdir.getitem("def test_func(some, other): pass")
class Provider:
def pytest_funcarg__some(self, request):
return request.function.__name__
def pytest_funcarg__other(self, request):
return 42
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 2
assert item.funcargs['some'] == "test_func"
assert item.funcargs['other'] == 42
def test_funcarg_lookup_modulelevel(self, testdir):
modcol = testdir.getmodulecol("""
def pytest_funcarg__something(request):
return request.function.__name__
class TestClass:
def test_method(self, something):
pass
def test_func(something):
pass
""")
item1, item2 = testdir.genitems([modcol])
item1.setupargs()
assert item1.funcargs['something'] == "test_method"
item2.setupargs()
assert item2.funcargs['something'] == "test_func"
class TestRequest:
def test_request_attributes(self, testdir):
item = testdir.getitem("""
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
req = item.getrequest("other")
assert req.argname == "other"
assert req.function == item.obj
assert req.function.__name__ == "test_func"
assert req.config == item.config
assert repr(req).find(req.function.__name__) != -1
def test_request_contains_funcargs_methods(self, testdir):
modcol = testdir.getmodulecol("""
def pytest_funcarg__something(request):
pass
class TestClass:
def pytest_funcarg__something(self, request):
pass
def test_method(self, something):
pass
""")
item1, = testdir.genitems([modcol])
assert item1.name == "test_method"
methods = item1.getrequest("something")._methods
assert len(methods) == 2
method1, method2 = methods
assert not hasattr(method1, 'im_self')
assert method2.im_self is not None
def test_request_call_next_provider(self, testdir):
item = testdir.getitem("""
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
req = item.getrequest("something")
val = req.call_next_provider()
assert val is None
py.test.raises(req.Error, "req.call_next_provider()")
def test_request_addfinalizer(self, testdir):
item = testdir.getitem("""
def pytest_funcarg__something(request): pass
def test_func(something): pass
""")
req = item.getrequest("something")
l = [1]
req.addfinalizer(l.pop)
item.teardown()
def test_request_maketemp(self, testdir):
item = testdir.getitem("def test_func(): pass")
req = item.getrequest("xxx")
tmpdir = req.maketempdir()
tmpdir2 = req.maketempdir()
assert tmpdir != tmpdir2
assert tmpdir.basename.startswith("test_func")
assert tmpdir2.basename.startswith("test_func")
def test_request_getmodulepath(self, testdir):
modcol = testdir.getmodulecol("def test_somefunc(): pass")
item, = testdir.genitems([modcol])
req = item.getrequest("hello")
assert req.fspath == modcol.fspath

View File

@ -1,31 +1,27 @@
import py
def pytest_funcarg__pickletransport(pyfuncitem):
return ImmutablePickleTransport()
def pytest_pyfunc_call(__call__, pyfuncitem, args, kwargs):
# for each function call we patch py._com.comregistry
# so that the unpickling of config objects
# (which bind to this mechanism) doesn't do harm
# usually config objects are no meant to be unpickled in
# the same system
def setglobals(request):
oldconfig = py.test.config
oldcom = py._com.comregistry
print "setting py.test.config to None"
py.test.config = None
py._com.comregistry = py._com.Registry()
try:
return __call__.execute(firstresult=True)
finally:
def resetglobals():
print "setting py.test.config to", oldconfig
py.test.config = oldconfig
py._com.comregistry = oldcom
request.addfinalizer(resetglobals)
def pytest_funcarg__testdir(request):
setglobals(request)
return request.call_next_provider()
class ImmutablePickleTransport:
def __init__(self):
def __init__(self, request):
from py.__.test.dist.mypickle import ImmutablePickler
self.p1 = ImmutablePickler(uneven=0)
self.p2 = ImmutablePickler(uneven=1)
setglobals(request)
def p1_to_p2(self, obj):
return self.p2.loads(self.p1.dumps(obj))
@ -39,6 +35,8 @@ class ImmutablePickleTransport:
return p2config
class TestImmutablePickling:
pytest_funcarg__pickletransport = ImmutablePickleTransport
def test_pickle_config(self, testdir, pickletransport):
config1 = testdir.parseconfig()
assert config1.topdir == testdir.tmpdir

View File

@ -34,13 +34,6 @@ class TestModule:
x = l.pop()
assert x is None
def test_module_participates_as_plugin(self, testdir):
modcol = testdir.getmodulecol("")
modcol.setup()
assert modcol.config.pluginmanager.isregistered(modcol.obj)
modcol.teardown()
assert not modcol.config.pluginmanager.isregistered(modcol.obj)
def test_module_considers_pluginmanager_at_import(self, testdir):
modcol = testdir.getmodulecol("pytest_plugins='xasdlkj',")
py.test.raises(ImportError, "modcol.obj")
@ -243,88 +236,6 @@ class TestFunction:
assert f1 == f1_b
assert not f1 != f1_b
def test_funcarg_lookupfails(self, testdir):
testdir.makeconftest("""
class ConftestPlugin:
def pytest_funcarg__something(self, pyfuncitem):
return 42
""")
item = testdir.getitem("def test_func(some): pass")
exc = py.test.raises(LookupError, "item.setupargs()")
s = str(exc.value)
assert s.find("something") != -1
def test_funcarg_lookup_default(self, testdir):
item = testdir.getitem("def test_func(some, other=42): pass")
class Provider:
def pytest_funcarg__some(self, pyfuncitem):
return pyfuncitem.name
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 1
def test_funcarg_lookup_default_gets_overriden(self, testdir):
item = testdir.getitem("def test_func(some=42, other=13): pass")
class Provider:
def pytest_funcarg__other(self, pyfuncitem):
return pyfuncitem.name
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 1
name, value = item.funcargs.popitem()
assert name == "other"
assert value == item.name
def test_funcarg_basic(self, testdir):
item = testdir.getitem("def test_func(some, other): pass")
class Provider:
def pytest_funcarg__some(self, pyfuncitem):
return pyfuncitem.name
def pytest_funcarg__other(self, pyfuncitem):
return 42
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 2
assert item.funcargs['some'] == "test_func"
assert item.funcargs['other'] == 42
def test_funcarg_addfinalizer(self, testdir):
item = testdir.getitem("def test_func(some): pass")
l = []
class Provider:
def pytest_funcarg__some(self, pyfuncitem):
pyfuncitem.addfinalizer(lambda: l.append(42))
return 3
item.config.pluginmanager.register(Provider())
item.setupargs()
assert len(item.funcargs) == 1
assert item.funcargs['some'] == 3
assert len(l) == 0
item.teardown()
assert len(l) == 1
assert l[0] == 42
def test_funcarg_lookup_modulelevel(self, testdir):
modcol = testdir.getmodulecol("""
def pytest_funcarg__something(pyfuncitem):
return pyfuncitem.name
class TestClass:
def test_method(self, something):
pass
def test_func(something):
pass
""")
item1, item2 = testdir.genitems([modcol])
modcol.setup()
assert modcol.config.pluginmanager.isregistered(modcol.obj)
item1.setupargs()
assert item1.funcargs['something'] == "test_method"
item2.setupargs()
assert item2.funcargs['something'] == "test_func"
modcol.teardown()
assert not modcol.config.pluginmanager.isregistered(modcol.obj)
class TestSorting:
def test_check_equality_and_cmp_basic(self, testdir):
modcol = testdir.getmodulecol("""

View File

@ -10,7 +10,7 @@ class TestTracebackCutting:
def test_traceback_argsetup(self, testdir):
testdir.makeconftest("""
class ConftestPlugin:
def pytest_funcarg__hello(self, pyfuncitem):
def pytest_funcarg__hello(self, request):
raise ValueError("xyz")
""")
p = testdir.makepyfile("def test(hello): pass")