105 lines
4.3 KiB
Plaintext
105 lines
4.3 KiB
Plaintext
py.test/pylib 1.3.1: new py.test.xfail, --maxfail, better reporting
|
|
===========================================================================
|
|
|
|
The pylib/py.test 1.3.1 release brings:
|
|
|
|
- the new imperative ``py.test.xfail()`` helper in order to have a test or
|
|
setup function result in an "expected failure"
|
|
- a new option ``--maxfail=NUM`` to stop the test run after some failures
|
|
- markers/decorators are now applicable to test classes (>=Python2.6)
|
|
- improved reporting, shorter tracebacks in several cases
|
|
- some simplified internals, more compatibility with Jython and PyPy
|
|
- bug fixes and various refinements
|
|
|
|
See the below CHANGELOG entry below for more details and
|
|
http://pylib.org/install.html for installation instructions.
|
|
|
|
If you used older versions of py.test you should be able to upgrade
|
|
to 1.3.1 without changes to your test source code.
|
|
|
|
py.test is an automated testing tool working with Python2,
|
|
Python3, Jython and PyPy versions on all major operating systems. It
|
|
offers a no-boilerplate testing approach and has inspired other testing
|
|
tools and enhancements in the standard Python library for more than five
|
|
years. It has a simple and extensive plugin architecture, configurable
|
|
reporting and provides unique ways to make it fit to your testing
|
|
process and needs.
|
|
|
|
See http://pytest.org for more info.
|
|
|
|
cheers and have fun,
|
|
|
|
holger krekel
|
|
|
|
Changes between 1.3.0 and 1.3.1
|
|
==================================================
|
|
|
|
New features
|
|
++++++++++++++++++
|
|
|
|
- issue91: introduce new py.test.xfail(reason) helper
|
|
to imperatively mark a test as expected to fail. Can
|
|
be used from within setup and test functions. This is
|
|
useful especially for parametrized tests when certain
|
|
configurations are expected-to-fail. In this case the
|
|
declarative approach with the @py.test.mark.xfail cannot
|
|
be used as it would mark all configurations as xfail.
|
|
|
|
- issue102: introduce new --maxfail=NUM option to stop
|
|
test runs after NUM failures. This is a generalization
|
|
of the '-x' or '--exitfirst' option which is now equivalent
|
|
to '--maxfail=1'. Both '-x' and '--maxfail' will
|
|
now also print a line near the end indicating the Interruption.
|
|
|
|
- issue89: allow py.test.mark decorators to be used on classes
|
|
(class decorators were introduced with python2.6) and
|
|
also allow to have multiple markers applied at class/module level
|
|
by specifying a list.
|
|
|
|
- improve and refine letter reporting in the progress bar:
|
|
. pass
|
|
f failed test
|
|
s skipped tests (reminder: use for dependency/platform mismatch only)
|
|
x xfailed test (test that was expected to fail)
|
|
X xpassed test (test that was expected to fail but passed)
|
|
|
|
You can use any combination of 'fsxX' with the '-r' extended
|
|
reporting option. The xfail/xpass results will show up as
|
|
skipped tests in the junitxml output - which also fixes
|
|
issue99.
|
|
|
|
- make py.test.cmdline.main() return the exitstatus instead of raising
|
|
SystemExit and also allow it to be called multiple times. This of
|
|
course requires that your application and tests are properly teared
|
|
down and don't have global state.
|
|
|
|
Fixes / Maintenance
|
|
++++++++++++++++++++++
|
|
|
|
- improved traceback presentation:
|
|
- improved and unified reporting for "--tb=short" option
|
|
- Errors during test module imports are much shorter, (using --tb=short style)
|
|
- raises shows shorter more relevant tracebacks
|
|
- --fulltrace now more systematically makes traces longer / inhibits cutting
|
|
|
|
- improve support for raises and other dynamically compiled code by
|
|
manipulating python's linecache.cache instead of the previous
|
|
rather hacky way of creating custom code objects. This makes
|
|
it seemlessly work on Jython and PyPy where it previously didn't.
|
|
|
|
- fix issue96: make capturing more resilient against Control-C
|
|
interruptions (involved somewhat substantial refactoring
|
|
to the underlying capturing functionality to avoid race
|
|
conditions).
|
|
|
|
- fix chaining of conditional skipif/xfail decorators - so it works now
|
|
as expected to use multiple @py.test.mark.skipif(condition) decorators,
|
|
including specific reporting which of the conditions lead to skipping.
|
|
|
|
- fix issue95: late-import zlib so that it's not required
|
|
for general py.test startup.
|
|
|
|
- fix issue94: make reporting more robust against bogus source code
|
|
(and internally be more careful when presenting unexpected byte sequences)
|
|
|