No longer override existing warning filters during warnings capture

Fix #2430
This commit is contained in:
Bruno Oliveira 2017-05-29 18:59:34 -03:00
parent ee0844dbd8
commit 32e2642233
7 changed files with 92 additions and 48 deletions

View File

@ -53,7 +53,6 @@ def catch_warnings_for_item(item):
args = item.config.getoption('pythonwarnings') or [] args = item.config.getoption('pythonwarnings') or []
inifilters = item.config.getini("filterwarnings") inifilters = item.config.getini("filterwarnings")
with warnings.catch_warnings(record=True) as log: with warnings.catch_warnings(record=True) as log:
warnings.simplefilter('once')
for arg in args: for arg in args:
warnings._setoption(arg) warnings._setoption(arg)

View File

@ -1 +1 @@
initial addition of towncrier Addition of towncrier for changelog management.

4
changelog/2430.bugfix Normal file
View File

@ -0,0 +1,4 @@
pytest warning capture no longer overrides existing warning filters. The previous
behaviour would override all filters and caused regressions in test suites which configure warning
filters to match their needs. Note that as a side-effect of this is that ``DeprecationWarning``
and ``PendingDeprecationWarning`` are no longer shown by default.

View File

@ -5,32 +5,18 @@ Warnings Capture
.. versionadded:: 3.1 .. versionadded:: 3.1
.. warning:: Starting from version ``3.1``, pytest now automatically catches warnings during test execution
pytest captures all warnings between tests, which prevents custom warning
filters in existing test suites from working. If this causes problems to your test suite,
this plugin can be disabled in your ``pytest.ini`` file with:
.. code-block:: ini
[pytest]
addopts = -p no:warnings
There's an ongoing discussion about this on `#2430
<https://github.com/pytest-dev/pytest/issues/2430>`_.
Starting from version ``3.1``, pytest now automatically catches all warnings during test execution
and displays them at the end of the session:: and displays them at the end of the session::
# content of test_show_warnings.py # content of test_show_warnings.py
import warnings import warnings
def deprecated_function(): def api_v1():
warnings.warn("this function is deprecated, use another_function()", DeprecationWarning) warnings.warn(UserWarning("api v1, should use functions from v2"))
return 1 return 1
def test_one(): def test_one():
assert deprecated_function() == 1 assert api_v1() == 1
Running pytest now produces this output:: Running pytest now produces this output::
@ -50,10 +36,12 @@ Running pytest now produces this output::
-- Docs: http://doc.pytest.org/en/latest/warnings.html -- Docs: http://doc.pytest.org/en/latest/warnings.html
======= 1 passed, 1 warnings in 0.12 seconds ======== ======= 1 passed, 1 warnings in 0.12 seconds ========
Pytest by default catches all warnings except for ``DeprecationWarning`` and ``PendingDeprecationWarning``.
The ``-W`` flag can be passed to control which warnings will be displayed or even turn The ``-W`` flag can be passed to control which warnings will be displayed or even turn
them into errors:: them into errors::
$ pytest -q test_show_warnings.py -W error::DeprecationWarning $ pytest -q test_show_warnings.py -W error::UserWarning
F F
======= FAILURES ======== ======= FAILURES ========
_______ test_one ________ _______ test_one ________
@ -72,7 +60,7 @@ them into errors::
1 failed in 0.12 seconds 1 failed in 0.12 seconds
The same option can be set in the ``pytest.ini`` file using the ``filterwarnings`` ini option. The same option can be set in the ``pytest.ini`` file using the ``filterwarnings`` ini option.
For example, the configuration below will ignore all deprecation warnings, but will transform For example, the configuration below will ignore all user warnings, but will transform
all other warnings into errors. all other warnings into errors.
.. code-block:: ini .. code-block:: ini
@ -80,7 +68,7 @@ all other warnings into errors.
[pytest] [pytest]
filterwarnings = filterwarnings =
error error
ignore::DeprecationWarning ignore::UserWarning
When a warning matches more than one option in the list, the action for the last matching option When a warning matches more than one option in the list, the action for the last matching option
@ -90,6 +78,19 @@ Both ``-W`` command-line option and ``filterwarnings`` ini option are based on P
`-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python `-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python
documentation for other examples and advanced usage. documentation for other examples and advanced usage.
.. note::
``DeprecationWarning`` and ``PendingDeprecationWarning`` are hidden by the standard library
by default so you have to explicitly configure them to be displayed in your ``pytest.ini``:
.. code-block:: ini
[pytest]
filterwarnings =
once::DeprecationWarning
once::PendingDeprecationWarning
*Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_ *Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_
*plugin.* *plugin.*
@ -97,6 +98,19 @@ documentation for other examples and advanced usage.
.. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter .. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter
.. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings .. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings
Disabling warning capture
-------------------------
This feature is enabled by default but can be disabled entirely in your ``pytest.ini`` file with:
.. code-block:: ini
[pytest]
addopts = -p no:warnings
Or passing ``-p no:warnings`` in the command-line.
.. _`asserting warnings`: .. _`asserting warnings`:
.. _assertwarnings: .. _assertwarnings:

View File

@ -2,6 +2,8 @@ from __future__ import absolute_import, division, print_function
import warnings import warnings
import re import re
import py import py
import sys
import pytest import pytest
from _pytest.recwarn import WarningsRecorder from _pytest.recwarn import WarningsRecorder
@ -146,9 +148,12 @@ class TestDeprecatedCall(object):
pytest.deprecated_call(deprecated_function) pytest.deprecated_call(deprecated_function)
""") """)
result = testdir.runpytest() result = testdir.runpytest()
# the 2 tests must pass, but the call to test_one() will generate a warning # for some reason in py26 catch_warnings manages to catch the deprecation warning
# in pytest's summary # from deprecated_function(), even with default filters active (which ignore deprecation
result.stdout.fnmatch_lines('*=== 2 passed, 1 warnings in *===') # warnings)
py26 = sys.version_info[:2] == (2, 6)
expected = '*=== 2 passed in *===' if not py26 else '*=== 2 passed, 1 warnings in *==='
result.stdout.fnmatch_lines(expected)
class TestWarns(object): class TestWarns(object):

View File

@ -20,8 +20,8 @@ def pyfile_with_warnings(testdir, request):
module_name: ''' module_name: '''
import warnings import warnings
def foo(): def foo():
warnings.warn(PendingDeprecationWarning("functionality is pending deprecation")) warnings.warn(UserWarning("user warning"))
warnings.warn(DeprecationWarning("functionality is deprecated")) warnings.warn(RuntimeWarning("runtime warning"))
return 1 return 1
''', ''',
test_name: ''' test_name: '''
@ -43,11 +43,11 @@ def test_normal_flow(testdir, pyfile_with_warnings):
'*test_normal_flow.py::test_func', '*test_normal_flow.py::test_func',
'*normal_flow_module.py:3: PendingDeprecationWarning: functionality is pending deprecation', '*normal_flow_module.py:3: UserWarning: user warning',
'* warnings.warn(PendingDeprecationWarning("functionality is pending deprecation"))', '* warnings.warn(UserWarning("user warning"))',
'*normal_flow_module.py:4: DeprecationWarning: functionality is deprecated', '*normal_flow_module.py:4: RuntimeWarning: runtime warning',
'* warnings.warn(DeprecationWarning("functionality is deprecated"))', '* warnings.warn(RuntimeWarning("runtime warning"))',
'* 1 passed, 2 warnings*', '* 1 passed, 2 warnings*',
]) ])
assert result.stdout.str().count('test_normal_flow.py::test_func') == 1 assert result.stdout.str().count('test_normal_flow.py::test_func') == 1
@ -90,8 +90,8 @@ def test_as_errors(testdir, pyfile_with_warnings, method):
''') ''')
result = testdir.runpytest(*args) result = testdir.runpytest(*args)
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines([
'E PendingDeprecationWarning: functionality is pending deprecation', 'E UserWarning: user warning',
'as_errors_module.py:3: PendingDeprecationWarning', 'as_errors_module.py:3: UserWarning',
'* 1 failed in *', '* 1 failed in *',
]) ])
@ -133,9 +133,7 @@ def test_unicode(testdir, pyfile_with_warnings):
result = testdir.runpytest() result = testdir.runpytest()
result.stdout.fnmatch_lines([ result.stdout.fnmatch_lines([
'*== %s ==*' % WARNINGS_SUMMARY_HEADER, '*== %s ==*' % WARNINGS_SUMMARY_HEADER,
'*test_unicode.py:8: UserWarning: \u6d4b\u8bd5*',
'*test_unicode.py:8: UserWarning: \u6d4b\u8bd5',
'*warnings.warn(u"\u6d4b\u8bd5")',
'* 1 passed, 1 warnings*', '* 1 passed, 1 warnings*',
]) ])
@ -163,6 +161,30 @@ def test_py2_unicode(testdir, pyfile_with_warnings):
'*test_py2_unicode.py:8: UserWarning: \u6d4b\u8bd5', '*test_py2_unicode.py:8: UserWarning: \u6d4b\u8bd5',
'*warnings.warn(u"\u6d4b\u8bd5")', '*warnings.warn(u"\u6d4b\u8bd5")',
'*warnings.py:82: UnicodeWarning: This warning*\u6d4b\u8bd5', '*warnings.py:*: UnicodeWarning: This warning*\u6d4b\u8bd5',
'* 1 passed, 2 warnings*', '* 1 passed, 2 warnings*',
]) ])
def test_works_with_filterwarnings(testdir):
"""Ensure our warnings capture does not mess with pre-installed filters (#2430)."""
testdir.makepyfile('''
import warnings
class MyWarning(Warning):
pass
warnings.filterwarnings("error", category=MyWarning)
class TestWarnings(object):
def test_my_warning(self):
try:
warnings.warn(MyWarning("warn!"))
assert False
except MyWarning:
assert True
''')
result = testdir.runpytest()
result.stdout.fnmatch_lines([
'*== 1 passed in *',
])

View File

@ -184,7 +184,7 @@ python_files=test_*.py *_test.py testing/*/*.py
python_classes=Test Acceptance python_classes=Test Acceptance
python_functions=test python_functions=test
norecursedirs = .tox ja .hg cx_freeze_source norecursedirs = .tox ja .hg cx_freeze_source
filterwarnings= error filterwarnings=
# produced by path.local # produced by path.local
ignore:bad escape.*:DeprecationWarning:re ignore:bad escape.*:DeprecationWarning:re
# produced by path.readlines # produced by path.readlines