p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Improve assertion failure reporting on iterables, by using ndiff and pprint. 

--HG--
branch : better-diff-on-verbose-2
This commit is contained in:
Anatoly Bubenkov 2014-09-27 01:29:47 +00:00
parent 49b7237581
commit 72e6f55b45
3 changed files with 94 additions and 17 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

27
CHANGELOG
View File

@ -1,3 +1,8 @@
Unreleased
----------
- Improve assertion failure reporting on iterables, by using ndiff and pprint.
2.6.3
-----------
@ -80,7 +85,7 @@
- fix issue with detecting conftest files if the arguments contain
"::" node id specifications (copy pasted from "-v" output)
- fix issue544 by only removing "@NUM" at the end of "::" separated parts
- fix issue544 by only removing "@NUM" at the end of "::" separated parts
and if the part has an ".py" extension
- don't use py.std import helper, rather import things directly.
@ -93,7 +98,7 @@
- fix issue537: Avoid importing old assertion reinterpretation code by default.
- fix issue364: shorten and enhance tracebacks representation by default.
- fix issue364: shorten and enhance tracebacks representation by default.
The new "--tb=auto" option (default) will only display long tracebacks
for the first and last entry. You can get the old behaviour of printing
all entries as long entries with "--tb=long". Also short entries by
@ -119,14 +124,14 @@
- fix issue473: work around mock putting an unbound method into a class
dict when double-patching.
- fix issue498: if a fixture finalizer fails, make sure that
- fix issue498: if a fixture finalizer fails, make sure that
the fixture is still invalidated.
- fix issue453: the result of the pytest_assertrepr_compare hook now gets
it's newlines escaped so that format_exception does not blow up.
- internal new warning system: pytest will now produce warnings when
it detects oddities in your test collection or execution.
it detects oddities in your test collection or execution.
Warnings are ultimately sent to a new pytest_logwarning hook which is
currently only implemented by the terminal plugin which displays
warnings in the summary line and shows more details when -rw (report on
@ -170,7 +175,7 @@
- fix issue492: avoid leak in test_writeorg. Thanks Marc Abramowitz.
- fix issue493: don't run tests in doc directory with ``python setup.py test``
- fix issue493: don't run tests in doc directory with ``python setup.py test``
(use tox -e doctesting for that)
- fix issue486: better reporting and handling of early conftest loading failures
@ -184,8 +189,8 @@
Groenholm.
- support nose-style ``__test__`` attribute on modules, classes and
functions, including unittest-style Classes. If set to False, the
test will not be collected.
functions, including unittest-style Classes. If set to False, the
test will not be collected.
- fix issue512: show "<notset>" for arguments which might not be set
in monkeypatch plugin. Improves output in documentation.
@ -195,11 +200,11 @@
-----------------------------------
- fix issue409 -- better interoperate with cx_freeze by not
trying to import from collections.abc which causes problems
trying to import from collections.abc which causes problems
for py27/cx_freeze. Thanks Wolfgang L. for reporting and tracking it down.
- fixed docs and code to use "pytest" instead of "py.test" almost everywhere.
Thanks Jurko Gospodnetic for the complete PR.
Thanks Jurko Gospodnetic for the complete PR.
- fix issue425: mention at end of "py.test -h" that --markers
and --fixtures work according to specified test path (or current dir)
@ -210,7 +215,7 @@
- copy, cleanup and integrate py.io capture
from pylib 1.4.20.dev2 (rev 13d9af95547e)
- address issue416: clarify docs as to conftest.py loading semantics
- fix issue429: comparing byte strings with non-ascii chars in assert
@ -230,7 +235,7 @@
- Allow parameterized fixtures to specify the ID of the parameters by
adding an ids argument to pytest.fixture() and pytest.yield_fixture().
Thanks Floris Bruynooghe.
Thanks Floris Bruynooghe.
- fix issue404 by always using the binary xml escape in the junitxml
plugin. Thanks Ronny Pfannschmidt.

39
_pytest/assertion/util.py
View File

@ -135,18 +135,32 @@ def assertrepr_compare(config, op, left, right):
isdict = lambda x: isinstance(x, dict)
isset = lambda x: isinstance(x, (set, frozenset))
def isiterable(obj):
try:
iter(obj)
return not istext(obj)
except TypeError:
return False
verbose = config.getoption('verbose')
explanation = None
try:
if op == '==':
if istext(left) and istext(right):
explanation = _diff_text(left, right, verbose)
elif issequence(left) and issequence(right):
explanation = _compare_eq_sequence(left, right, verbose)
elif isset(left) and isset(right):
explanation = _compare_eq_set(left, right, verbose)
elif isdict(left) and isdict(right):
explanation = _compare_eq_dict(left, right, verbose)
else:
if issequence(left) and issequence(right):
explanation = _compare_eq_sequence(left, right, verbose)
elif isset(left) and isset(right):
explanation = _compare_eq_set(left, right, verbose)
elif isdict(left) and isdict(right):
explanation = _compare_eq_dict(left, right, verbose)
if isiterable(left) and isiterable(right):
expl = _compare_eq_iterable(left, right, verbose)
if explanation is not None:
explanation.extend(expl)
else:
explanation = expl
elif op == 'not in':
if istext(left) and istext(right):
explanation = _notin_text(left, right, verbose)
@ -203,6 +217,19 @@ def _diff_text(left, right, verbose=False):
return explanation
def _compare_eq_iterable(left, right, verbose=False):
if not verbose:
return [u('Use -v to get the full diff')]
# dynamic import to speedup pytest
import difflib
left = pprint.pformat(left).splitlines()
right = pprint.pformat(right).splitlines()
explanation = [u('Full diff:')]
explanation.extend(line.strip() for line in difflib.ndiff(left, right))
return explanation
def _compare_eq_sequence(left, right, verbose=False):
explanation = []
for i in range(min(len(left), len(right))):

45
testing/test_assertion.py
View File

@ -1,11 +1,14 @@
# -*- coding: utf-8 -*-
import sys
import textwrap
import py, pytest
import _pytest.assertion as plugin
from _pytest.assertion import reinterpret
from _pytest.assertion import util
needsnewassert = pytest.mark.skipif("sys.version_info < (2,6)")
PY3 = sys.version_info >= (3, 0)
@pytest.fixture
@ -86,6 +89,48 @@ class TestAssert_reprcompare:
expl = callequal([0, 1], [0, 2])
assert len(expl) > 1
@pytest.mark.parametrize(
['left', 'right', 'expected'], [
([0, 1], [0, 2], """
Full diff:
- [0, 1]
? ^
+ [0, 2]
? ^
"""),
({0: 1}, {0: 2}, """
Full diff:
- {0: 1}
? ^
+ {0: 2}
? ^
"""),
(set([0, 1]), set([0, 2]), """
Full diff:
- set([0, 1])
? ^
+ set([0, 2])
? ^
""" if not PY3 else """
Full diff:
- {0, 1}
? ^
+ {0, 2}
? ^
""")
]
)
def test_iterable_full_diff(self, left, right, expected):
"""Test the full diff assertion failure explanation.
When verbose is False, then just a -v notice to get the diff is rendered,
when verbose is True, then ndiff of the pprint is returned.
"""
expl = callequal(left, right, verbose=False)
assert expl[-1] == 'Use -v to get the full diff'
expl = '\n'.join(callequal(left, right, verbose=True))
assert expl.endswith(textwrap.dedent(expected).strip())
def test_list_different_lenghts(self):
expl = callequal([0, 1], [0, 1, 2])
assert len(expl) > 1