holger krekel
commit
cbf05b325f
12
CHANGELOG
12
CHANGELOG
|
|
@ -1,7 +1,15 @@
|
|||
Changes between 1.0.0b7 and 1.0.0
|
||||
Changes between 1.0.0b7 and 1.0.0b8
|
||||
=====================================
|
||||
|
||||
* tweaked doctest output for docstrings in py modules
|
||||
* docs: refined funcargs doc, use the term
|
||||
"factory" instead of "provider", added a new
|
||||
talk/tutorial doc page
|
||||
|
||||
* fixed teardown problem related to partially failing funcarg setups
|
||||
(thanks MrTopf for reporting)
|
||||
|
||||
* tweaked doctest output for docstrings in py modules,
|
||||
thanks Radomir.
|
||||
|
||||
Changes between 1.0.0b3 and 1.0.0b7
|
||||
=============================================
|
||||
|
|
|
|||
|
|
@ -9,6 +9,11 @@ py.execnet
|
|||
* asynchronously send and receive data between processes through channels
|
||||
* completely avoid manual installation steps on remote places
|
||||
|
||||
There is a `EuroPython2009 talk`_ from July 2009 with
|
||||
examples and some pictures.
|
||||
|
||||
.. _`EuroPython2009 talk`: http://codespeak.net/download/py/ep2009-execnet.pdf
|
||||
|
||||
Gateways: immediately spawn local or remote process
|
||||
===================================================
|
||||
|
||||
|
|
|
|||
|
|
@ -68,25 +68,28 @@ home-directoray, per shell session or per test-run.
|
|||
|
||||
.. _`basetemp`:
|
||||
|
||||
per-testrun temporary directories
|
||||
Temporary directories
|
||||
-------------------------------------------
|
||||
|
||||
``py.test`` runs provide means to create per-test session
|
||||
temporary (sub) directories through the config object.
|
||||
You can create directories like this:
|
||||
You can create directories by calling a method
|
||||
on the config object:
|
||||
|
||||
.. XXX use a more local example, just with "config"
|
||||
- ``config.mktemp(basename)``: create and returns a new tempdir
|
||||
|
||||
.. sourcecode: python
|
||||
- ``config.ensuretemp(basename)``: create or return a new tempdir
|
||||
|
||||
import py
|
||||
basetemp = py.test.config.ensuretemp()
|
||||
basetemp_subdir = py.test.config.ensuretemp("subdir")
|
||||
|
||||
By default, ``py.test`` creates a ``pytest-NUMBER`` directory
|
||||
tempdirs are created as sub directories of a per-session testdir
|
||||
and will keep around the directories of the last three
|
||||
test runs. You can also set the base temporary directory
|
||||
with the `--basetemp`` option. When distributing
|
||||
tests on the same machine, ``py.test`` takes care to
|
||||
pass around the basetemp directory such that all temporary
|
||||
files land below the same basetemp directory.
|
||||
|
||||
The config object is available when implementing `function arguments`_
|
||||
or `extensions`_ and can otherwise be globally accessed as ``py.test.config``.
|
||||
|
||||
.. _`function arguments`: funcargs.html
|
||||
.. _`extensions`: extend.html
|
||||
|
|
|
|||
|
|
@ -1,45 +1,34 @@
|
|||
======================================================
|
||||
**funcargs**: test setup and parametrization
|
||||
======================================================
|
||||
==========================================================
|
||||
**funcargs**: test function arguments FTW
|
||||
==========================================================
|
||||
|
||||
Since version 1.0 py.test introduces test function arguments,
|
||||
in short "funcargs" for your Python test functions. The basic idea
|
||||
that your unit-, functional- or acceptance test functions can name
|
||||
arguments and py.test will discover a matching provider from your
|
||||
test configuration. The mechanism complements the automatic
|
||||
discovery of test files, classes and functions which follows
|
||||
the `Convention over Configuration`_ strategy. By discovering and
|
||||
calling functions ("funcarg providers") that provide values for your
|
||||
actual test functions it becomes easy to:
|
||||
Since version 1.0 py.test features the "funcarg" mechanism which
|
||||
allows a test function to take arguments which will be independently
|
||||
provided by factory functions. Factory functions are automatically
|
||||
discovered and allow to encapsulate all neccessary setup and glue code
|
||||
for running tests. Compared to `xUnit style`_ the new mechanism is
|
||||
meant to:
|
||||
|
||||
* separate test function code from test state setup/fixtures
|
||||
* manage test value setup and teardown depending on
|
||||
command line options or configuration
|
||||
* parametrize multiple runs of the same test functions
|
||||
* present useful debug info if setting up test state goes wrong
|
||||
* make test functions easier to write and to read
|
||||
* isolate test fixture creation to a single place
|
||||
* bring new flexibility and power to test state management
|
||||
* enable running of a test function with different values
|
||||
(superseding `old-style generative tests`_)
|
||||
* to enable creation of helper objects that interact with the execution
|
||||
of a test function, see the `blog post about the monkeypatch funcarg`_.
|
||||
|
||||
Using funcargs, test functions become more expressive,
|
||||
more "templaty" and more test-aspect oriented. In fact,
|
||||
funcarg mechanisms are meant to be complete and
|
||||
convenient enough to
|
||||
|
||||
* substitute and improve on most usages of `xUnit style`_ setup.
|
||||
For a simple example of how funcargs compare
|
||||
to xUnit setup, see the `blog post about
|
||||
the monkeypatch funcarg`_.
|
||||
|
||||
* substitute and improve on all usages of `old-style generative tests`_,
|
||||
i.e. test functions that use the "yield" statement.
|
||||
Using yield in test functions is deprecated since 1.0.
|
||||
If you find issues or have further suggestions for improving
|
||||
the mechanism you are welcome to checkout `contact possibilities`_ page.
|
||||
|
||||
.. _`contact possibilities`: ../contact.html
|
||||
|
||||
.. _`blog post about the monkeypatch funcarg`: http://tetamap.wordpress.com/2009/03/03/monkeypatching-in-unit-tests-done-right/
|
||||
.. _`xUnit style`: xunit_setup.html
|
||||
.. _`old-style generative tests`: features.html#generative-tests
|
||||
|
||||
.. _`funcarg provider`:
|
||||
.. _`funcarg factory`:
|
||||
|
||||
funcarg providers: setting up test function arguments
|
||||
funcarg factories: setting up test function arguments
|
||||
==============================================================
|
||||
|
||||
Test functions can specify one ore more arguments ("funcargs")
|
||||
|
|
@ -49,22 +38,22 @@ example that you can put into a test module:
|
|||
|
||||
.. sourcecode:: python
|
||||
|
||||
# ./test_simpleprovider.py
|
||||
# ./test_simplefactory.py
|
||||
def pytest_funcarg__myfuncarg(request):
|
||||
return 42
|
||||
|
||||
def test_function(myfuncarg):
|
||||
assert myfuncarg == 17
|
||||
|
||||
If you run this with ``py.test test_simpleprovider.py`` you see something like this:
|
||||
If you run this with ``py.test test_simplefactory.py`` you see something like this:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
============================ test session starts ============================
|
||||
python: platform linux2 -- Python 2.6.2
|
||||
test object 1: /home/hpk/hg/py/trunk/example/funcarg/test_simpleprovider.py
|
||||
test object 1: /home/hpk/hg/py/trunk/example/funcarg/test_simplefactory.py
|
||||
|
||||
test_simpleprovider.py F
|
||||
test_simplefactory.py F
|
||||
|
||||
================================= FAILURES ==================================
|
||||
_______________________________ test_function _______________________________
|
||||
|
|
@ -75,7 +64,7 @@ If you run this with ``py.test test_simpleprovider.py`` you see something like t
|
|||
> assert myfuncarg == 17
|
||||
E assert 42 == 17
|
||||
|
||||
test_simpleprovider.py:6: AssertionError
|
||||
test_simplefactory.py:6: AssertionError
|
||||
========================= 1 failed in 0.11 seconds ==========================
|
||||
|
||||
|
||||
|
|
@ -84,7 +73,7 @@ Here is how py.test comes to execute this test function:
|
|||
|
||||
1. py.test discovers the ``test_function`` because of the ``test_`` prefix.
|
||||
The test function needs a function argument named ``myfuncarg``.
|
||||
A matching provider function is discovered by looking for the special
|
||||
A matching factory function is discovered by looking for the special
|
||||
name ``pytest_funcarg__myfuncarg``.
|
||||
|
||||
2. ``pytest_funcarg__myfuncarg(request)`` is called and
|
||||
|
|
@ -96,18 +85,17 @@ Note that if you misspell a function argument or want
|
|||
to use one that isn't available, an error with a list of
|
||||
available function argument is provided.
|
||||
|
||||
For more interesting provider functions that make good use of the
|
||||
For more interesting factory functions that make good use of the
|
||||
`request object`_ please see the `application setup tutorial example`_.
|
||||
|
||||
.. _`request object`:
|
||||
|
||||
funcarg request objects
|
||||
funcarg factory request objects
|
||||
------------------------------------------
|
||||
|
||||
Request objects are passed to funcarg providers. They
|
||||
encapsulate a request for a function argument for a
|
||||
specific test function. Request objects allow providers
|
||||
to access test configuration and test context:
|
||||
Request objects are passed to funcarg factories and allow
|
||||
to access test configuration, test context and `useful caching
|
||||
and finalization helpers`_. Here is a list of attributes:
|
||||
|
||||
``request.function``: python function object requesting the argument
|
||||
|
||||
|
|
@ -119,9 +107,11 @@ to access test configuration and test context:
|
|||
|
||||
``request.param``: if exists was passed by a `parametrizing test generator`_
|
||||
|
||||
.. _`useful caching and finalization helpers`:
|
||||
|
||||
teardown/cleanup after test function execution
|
||||
------------------------------------------------
|
||||
|
||||
registering funcarg related finalizers/cleanup
|
||||
----------------------------------------------------
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
|
|
@ -130,7 +120,8 @@ teardown/cleanup after test function execution
|
|||
|
||||
Calling ``request.addfinalizer()`` is useful for scheduling teardown
|
||||
functions. Here is an example for providing a ``myfile``
|
||||
object that is to be closed when the test function finishes.
|
||||
object that is to be closed when the execution of a
|
||||
test function finishes.
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
|
|
@ -140,8 +131,8 @@ object that is to be closed when the test function finishes.
|
|||
return myfile
|
||||
|
||||
|
||||
perform scope-specific setup and cleanup
|
||||
---------------------------------------------
|
||||
managing fixtures across test modules and test runs
|
||||
----------------------------------------------------------
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
|
|
@ -156,38 +147,36 @@ perform scope-specific setup and cleanup
|
|||
scope == 'session': when tests of the session have run.
|
||||
"""
|
||||
|
||||
example for providing a value that is to be setup only once during a test run:
|
||||
Calling ``request.cached_setup()`` helps you to manage fixture
|
||||
objects across several scopes. For example, for creating a Database object
|
||||
that is to be setup only once during a test session you can use the helper
|
||||
like this:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
def pytest_funcarg__db(request):
|
||||
def pytest_funcarg__database(request):
|
||||
return request.cached_setup(
|
||||
lambda: ExpensiveSetup(request.config.option.db),
|
||||
lambda val: val.close(),
|
||||
scope="run"
|
||||
setup=lambda: Database("..."),
|
||||
teardown=lambda val: val.close(),
|
||||
scope="session"
|
||||
)
|
||||
|
||||
|
||||
requesting values of other funcargs
|
||||
---------------------------------------------
|
||||
|
||||
Inside a funcarg provider, you sometimes may want to use a
|
||||
different function argument which may be specified with
|
||||
the test function or not. For such purposes you can
|
||||
dynamically request a funcarg value:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
||||
def getfuncargvalue(name):
|
||||
""" Lookup and call function argument provider for the given name.
|
||||
Each function argument is only requested once per function setup.
|
||||
""" Lookup and call function argument factory for the given name.
|
||||
Each function argument is only created once per function setup.
|
||||
"""
|
||||
|
||||
You can also use this function if you want to `decorate a funcarg`_
|
||||
locally, i.e. you want to provide the normal value but add/do something
|
||||
extra. If a provider cannot be found a ``request.Error`` exception will be
|
||||
raised.
|
||||
|
||||
``request.getfuncargvalue(name)`` calls another funcarg factory function.
|
||||
You can use this function if you want to `decorate a funcarg`_, i.e.
|
||||
you want to provide the "normal" value but add something
|
||||
extra. If a factory cannot be found a ``request.Error``
|
||||
exception will be raised.
|
||||
|
||||
.. _`test generators`:
|
||||
.. _`parametrizing test generator`:
|
||||
|
|
@ -195,7 +184,7 @@ raised.
|
|||
generating parametrized tests with funcargs
|
||||
===========================================================
|
||||
|
||||
You can directly parametrize multiple runs of the same test
|
||||
You can parametrize multiple runs of the same test
|
||||
function by adding new test function calls with different
|
||||
function argument values. Let's look at a simple self-contained
|
||||
example:
|
||||
|
|
@ -280,7 +269,7 @@ the stringified counter of the list of added calls will be used.
|
|||
invocations for a given test function.
|
||||
|
||||
``param`` if specified will be seen by any
|
||||
`funcarg provider`_ as a ``request.param`` attribute.
|
||||
`funcarg factory`_ as a ``request.param`` attribute.
|
||||
Setting it is called *indirect parametrization*.
|
||||
|
||||
Indirect parametrization is preferable if test values are
|
||||
|
|
@ -322,12 +311,12 @@ specific setup.
|
|||
answer = app.question()
|
||||
assert answer == 42
|
||||
|
||||
To run this test py.test needs to find and call a provider to
|
||||
To run this test py.test needs to find and call a factory to
|
||||
obtain the required ``mysetup`` function argument. The test
|
||||
function interacts with the provided application specific setup.
|
||||
|
||||
To provide the ``mysetup`` function argument we write down
|
||||
a provider method in a `local plugin`_ by putting the
|
||||
a factory method in a `local plugin`_ by putting the
|
||||
following code into a local ``conftest.py``:
|
||||
|
||||
.. sourcecode:: python
|
||||
|
|
@ -448,7 +437,7 @@ Running ``py.test test_ssh.py`` without specifying a command line option will re
|
|||
conftest.py:23: [1] Skipped: 'specify ssh host with --ssh'
|
||||
====================== 1 skipped in 0.11 seconds ======================
|
||||
|
||||
Note especially how the test function could stay clear knowing about how to construct test state values or when to skip and with what message. The test function can concentrate on actual test code and test state providers can interact with execution of tests.
|
||||
Note especially how the test function could stay clear knowing about how to construct test state values or when to skip and with what message. The test function can concentrate on actual test code and test state factories can interact with execution of tests.
|
||||
|
||||
If you specify a command line option like ``py.test --ssh=python.org`` the test will get un-skipped and actually execute.
|
||||
|
||||
|
|
@ -508,7 +497,7 @@ extend the `accept example`_ by putting this in our test class:
|
|||
.. sourcecode:: python
|
||||
|
||||
def pytest_funcarg__accept(self, request):
|
||||
arg = request.getfuncargvalue("accept") # call the next provider
|
||||
arg = request.getfuncargvalue("accept") # call the next factory
|
||||
# create a special layout in our tempdir
|
||||
arg.tmpdir.mkdir("special")
|
||||
return arg
|
||||
|
|
@ -517,8 +506,8 @@ extend the `accept example`_ by putting this in our test class:
|
|||
def test_sometest(self, accept):
|
||||
assert accept.tmpdir.join("special").check()
|
||||
|
||||
Our module level provider will be invoked first and it can
|
||||
ask its request object to call the next provider and then
|
||||
Our module level factory will be invoked first and it can
|
||||
ask its request object to call the next factory and then
|
||||
decorate its result. This mechanism allows us to stay
|
||||
ignorant of how/where the function argument is provided -
|
||||
in our example from a `conftest plugin`_.
|
||||
|
|
@ -543,7 +532,7 @@ When experimenting with funcargs we also
|
|||
considered an explicit registration mechanism, i.e. calling a register
|
||||
method on the config object. But lacking a good use case for this
|
||||
indirection and flexibility we decided to go for `Convention over
|
||||
Configuration`_ and allow to directly specify the provider. It has the
|
||||
Configuration`_ and allow to directly specify the factory. It has the
|
||||
positive implication that you should be able to "grep" for
|
||||
``pytest_funcarg__MYARG`` and will find all providing sites (usually
|
||||
exactly one).
|
||||
|
|
|
|||
|
|
@ -50,10 +50,12 @@ Python test module is inspected for test methods starting with ``test_``.
|
|||
.. Organising your tests
|
||||
.. ---------------------------
|
||||
|
||||
Please refer to `features`_ for a walk through the basic features.
|
||||
|
||||
Please refer to `features`_ for a walk through the basic features
|
||||
or checkout the `tutorials`_ page for more introduction material.
|
||||
|
||||
.. _download: ../download.html
|
||||
.. _features: features.html
|
||||
.. _tutorials: talks.html
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,26 @@
|
|||
==========================
|
||||
Talks and Tutorials
|
||||
==========================
|
||||
|
||||
.. _`funcargs`: funcargs.html
|
||||
|
||||
a list of the latest talk and tutorial material:
|
||||
|
||||
- `ep2009-rapidtesting.pdf`_ tutorial slides (July 2009):
|
||||
|
||||
- testing terminology
|
||||
- basic py.test usage, file system layout
|
||||
- test function arguments (funcargs_) and test fixtures
|
||||
- existing plugins
|
||||
- distributed testing
|
||||
|
||||
- `ep2009-pytest.pdf`_ 60 minute py.test talk, highlighting unique features and a roadmap (July 2009)
|
||||
|
||||
- `pycon2009-pytest-introduction.zip`_ slides and files, extended version of py.test basic introduction, discusses more options, also introduces old-style xUnit setup, looponfailing and other features.
|
||||
|
||||
- `pycon2009-pytest-advanced.pdf`_ contain a slightly older version of funcargs and distributed testing, compared to the EuroPython 2009 slides.
|
||||
|
||||
.. _`ep2009-rapidtesting.pdf`: http://codespeak.net/download/py/ep2009-rapidtesting.pdf
|
||||
.. _`ep2009-pytest.pdf`: http://codespeak.net/download/py/ep2009-pytest.pdf
|
||||
.. _`pycon2009-pytest-introduction.zip`: http://codespeak.net/download/py/pycon2009-pytest-introduction.zip
|
||||
.. _`pycon2009-pytest-advanced.pdf`: http://codespeak.net/download/py/pycon2009-pytest-advanced.pdf
|
||||
|
|
@ -17,8 +17,11 @@ funcargs_: powerful parametrized test function setup
|
|||
|
||||
extend_: intro to extend and customize py.test runs
|
||||
|
||||
config_: ``conftest.py`` files and general configuration
|
||||
config_: ``conftest.py`` files and the config object
|
||||
|
||||
talks_: talk and tutorial slides
|
||||
|
||||
.. _talks: talks.html
|
||||
.. _quickstart: quickstart.html
|
||||
.. _features: features.html
|
||||
.. _funcargs: funcargs.html
|
||||
|
|
|
|||
|
|
@ -165,7 +165,7 @@ class FuncargRequest:
|
|||
line = "%s:%s" %(fspath, lineno)
|
||||
msg = "funcargument %r not found for: %s" %(argname, line)
|
||||
msg += "\n available funcargs: %s" %(", ".join(available),)
|
||||
raise LookupError(msg)
|
||||
raise self.Error(msg)
|
||||
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -254,5 +254,5 @@ class SetupState(object):
|
|||
break
|
||||
self._pop_and_teardown()
|
||||
for col in needed_collectors[len(self.stack):]:
|
||||
col.setup()
|
||||
self.stack.append(col)
|
||||
col.setup()
|
||||
|
|
|
|||
|
|
@ -152,6 +152,7 @@ class TestRequest:
|
|||
def test_func(something): pass
|
||||
""")
|
||||
req = funcargs.FuncargRequest(item)
|
||||
py.test.raises(req.Error, req.getfuncargvalue, "notexists")
|
||||
val = req.getfuncargvalue("something")
|
||||
assert val == 1
|
||||
val = req.getfuncargvalue("something")
|
||||
|
|
@ -181,6 +182,21 @@ class TestRequest:
|
|||
print ss.stack
|
||||
assert teardownlist == [1]
|
||||
|
||||
def test_request_addfinalizer_partial_setup_failure(self, testdir):
|
||||
p = testdir.makepyfile("""
|
||||
l = []
|
||||
def pytest_funcarg__something(request):
|
||||
request.addfinalizer(lambda: l.append(None))
|
||||
def test_func(something, missingarg):
|
||||
pass
|
||||
def test_second():
|
||||
assert len(l) == 1
|
||||
""")
|
||||
result = testdir.runpytest(p)
|
||||
assert result.stdout.fnmatch_lines([
|
||||
"*1 failed*1 passed*"
|
||||
])
|
||||
|
||||
def test_request_getmodulepath(self, testdir):
|
||||
modcol = testdir.getmodulecol("def test_somefunc(): pass")
|
||||
item, = testdir.genitems([modcol])
|
||||
|
|
|
|||
Loading…
Reference in New Issue