Merge remote-tracking branch 'upstream/master' into features

.github/PULL_REQUEST_TEMPLATE.md

@@ -2,13 +2,14 @@ Thanks for submitting a PR, your contribution is really appreciated!
 
 Here's a quick checklist that should be present in PRs:
 
-- [ ] Target: for bug or doc fixes, target `master`; for new features, target `features`;
+- [ ] Add a new news fragment into the changelog folder
+  * name it `$issue_id.$type` for example (588.bug)
+  * if you don't have an issue_id change it to the pr id after creating the pr
+  * ensure type is one of `removal`, `feature`, `bugfix`, `vendor`, `doc` or `trivial`
+  * Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."
+- [ ] Target: for `bugfix`, `vendor`, `doc` or `trivial` fixes, target `master`; for removals or features target `features`;
+- [ ] Make sure to include reasonable tests for your change if necessary
 
-Unless your change is trivial documentation fix (e.g.,  a typo or reword of a small section) please:
+Unless your change is a trivial or a documentation fix (e.g.,  a typo or reword of a small section) please:
 
-- [ ] Make sure to include one or more tests for your change;
 - [ ] Add yourself to `AUTHORS`;
-- [ ] Add a new entry to `CHANGELOG.rst`
-  * Choose any open position to avoid merge conflicts with other PRs.
-  * Add a link to the issue you are fixing (if any) using RST syntax.
-  * The pytest team likes to have people to acknowledged in the `CHANGELOG`, so please add a thank note to yourself ("Thanks @user for the PR") and a link to your GitHub profile. It may sound weird thanking yourself, but otherwise a maintainer would have to do it manually before or after merging instead of just using GitHub's merge button. This makes it easier on the maintainers to merge PRs. 

CHANGELOG.rst

@@ -1,9 +1,37 @@
-3.1.1 (unreleased)
-==================
+.. 
+    You should *NOT* be adding new change log entries to this file, this
+    file is managed by towncrier. You *may* edit previous change logs to
+    fix problems like typo corrections or such.
+    To add a new change log entry, please see
+    https://pip.pypa.io/en/latest/development/#adding-a-news-entry
+    we named the news folder changelog
 
-* Fix encoding errors for unicode warnings in Python 2. (towncrier: 2436.bugfix)
+.. towncrier release notes start
 
-* Fix issue with non-ascii contents in doctest text files. (towncrier: 2434.bugfix)
+Pytest 3.1.1 (2017-05-30)
+=========================
+
+Bug Fixes
+---------
+
+- pytest warning capture no longer overrides existing warning filters. The
+  previous behaviour would override all filters and caused regressions in test
+  suites which configure warning filters to match their needs. Note that as a
+  side-effect of this is that ``DeprecationWarning`` and
+  ``PendingDeprecationWarning`` are no longer shown by default. (#2430)
+
+- Fix issue with non-ascii contents in doctest text files. (#2434)
+
+- Fix encoding errors for unicode warnings in Python 2. (#2436)
+
+- ``pytest.deprecated_call`` now captures ``PendingDeprecationWarning`` in
+  context manager form. (#2441)
+
+
+Improved Documentation
+----------------------
+
+- Addition of towncrier for changelog management. (#2390)
 
 
 3.1.0 (2017-05-22)

HOWTORELEASE.rst

@@ -3,47 +3,59 @@ How to release pytest
 
 .. important::
 
-    pytest releases must be prepared on **linux** because the docs and examples expect
+    pytest releases must be prepared on **Linux** because the docs and examples expect
     to be executed in that platform.
 
 #. Install development dependencies in a virtual environment with::
 
     pip3 install -r tasks/requirements.txt
 
-#. Create a branch ``release-X.Y.Z`` with the version for the release. Make sure it is up to date
-   with the latest ``master`` (for patch releases) and with the latest ``features`` merged with
-   the latest ``master`` (for minor releases). Ensure your are in a clean work tree.
+#. Create a branch ``release-X.Y.Z`` with the version for the release.
 
-#. Check and finalize ``CHANGELOG.rst`` (will be automated soon).
+   * **patch releases**: from the latest ``master``;
 
-#. Execute to automatically generate docs, announcements and upload a package to
+   * **minor releases**: from the latest ``features``; then merge with the latest ``master``;
+
+   Ensure your are in a clean work tree.
+
+#. Generate docs, changelog, announcements and upload a package to
    your ``devpi`` staging server::
 
      invoke generate.pre_release <VERSION> <DEVPI USER> --password <DEVPI PASSWORD>
 
-   If ``--password`` is not given, it is assumed the user is already logged in. If you don't have
-   an account, please ask for one!
+   If ``--password`` is not given, it is assumed the user is already logged in ``devpi``.
+   If you don't have an account, please ask for one.
 
-#. Run from multiple machines::
+#. Open a PR for this branch targeting ``master``.
 
-     devpi use https://devpi.net/USER/dev
-     devpi test pytest==VERSION
+#. Test the package
 
-   Alternatively, you can use `devpi-cloud-tester <https://github.com/nicoddemus/devpi-cloud-tester>`_ to test
-   the package on AppVeyor and Travis (follow instructions on the ``README``).
+   * **Manual method**
 
-#. Check that tests pass for relevant combinations with::
+     Run from multiple machines::
+
+       devpi use https://devpi.net/USER/dev
+       devpi test pytest==VERSION
+
+     Check that tests pass for relevant combinations with::
 
        devpi list pytest
 
-   or look at failures with "devpi list -f pytest".
+   * **CI servers**
 
-#. Feeling confident? Publish to PyPI::
+     Configure a repository as per-instructions on
+     devpi-cloud-test_ to test the package on Travis_ and AppVeyor_.
+     All test environments should pass.
+
+#. Publish to PyPI::
 
       invoke generate.publish_release <VERSION> <DEVPI USER> <PYPI_NAME>
 
    where PYPI_NAME is the name of pypi.python.org as configured in your ``~/.pypirc``
    file `for devpi <http://doc.devpi.net/latest/quickstart-releaseprocess.html?highlight=pypirc#devpi-push-releasing-to-an-external-index>`_.
 
-
 #. After a minor/major release, merge ``features`` into ``master`` and push (or open a PR).
+
+.. _devpi-cloud-test: https://github.com/obestwalter/devpi-cloud-test
+.. _AppVeyor: https://www.appveyor.com/
+.. _Travis: https://travis-ci.org

MANIFEST.in

@@ -1,6 +1,7 @@
 include CHANGELOG.rst
 include LICENSE
 include AUTHORS
+include pyproject.toml
 
 include README.rst
 include CONTRIBUTING.rst
@@ -9,6 +10,7 @@ include HOWTORELEASE.rst
 include tox.ini
 include setup.py
 
+recursive-include changelog *
 recursive-include scripts *.py
 recursive-include scripts *.bat
 

_pytest/recwarn.py

@@ -45,7 +45,7 @@ def deprecated_call(func=None, *args, **kwargs):
     triggered twice for the same module. See #1190.
     """
     if not func:
-        return WarningsChecker(expected_warning=DeprecationWarning)
+        return WarningsChecker(expected_warning=(DeprecationWarning, PendingDeprecationWarning))
 
     categories = []
 

_pytest/warnings.py

@@ -53,7 +53,6 @@ def catch_warnings_for_item(item):
     args = item.config.getoption('pythonwarnings') or []
     inifilters = item.config.getini("filterwarnings")
     with warnings.catch_warnings(record=True) as log:
-        warnings.simplefilter('once')
         for arg in args:
             warnings._setoption(arg)
 

changelog/_template.rst

@@ -0,0 +1,39 @@
+{% for section in sections %}
+{% set underline = "-" %}
+{% if section %}
+{{section}}
+{{ underline * section|length }}{% set underline = "~" %}
+
+{% endif %}
+{% if sections[section] %}
+{% for category, val in definitions.items() if category in sections[section] and category != 'trivial' %}
+
+{{ definitions[category]['name'] }}
+{{ underline * definitions[category]['name']|length }}
+
+{% if definitions[category]['showcontent'] %}
+{% for text, values in sections[section][category]|dictsort(by='value') %}
+- {{ text }}{% if category != 'vendor' %} ({{ values|sort|join(', ') }}){% endif %}
+
+
+{% endfor %}
+{% else %}
+- {{ sections[section][category]['']|sort|join(', ') }}
+
+
+{% endif %}
+{% if sections[section][category]|length == 0 %}
+
+No significant changes.
+
+
+{% else %}
+{% endif %}
+{% endfor %}
+{% else %}
+
+No significant changes.
+
+
+{% endif %}
+{% endfor %}

doc/en/announce/index.rst

@@ -6,6 +6,7 @@ Release announcements
    :maxdepth: 2
 
    
+   release-3.1.1
    release-3.1.0
    release-3.0.7
    release-3.0.6

doc/en/announce/release-3.1.1.rst

@@ -0,0 +1,23 @@
+pytest-3.1.1
+=======================================
+
+pytest 3.1.1 has just been released to PyPI.
+
+This is a bug-fix release, being a drop-in replacement. To upgrade::
+
+  pip install --upgrade pytest
+  
+The full changelog is available at http://doc.pytest.org/en/latest/changelog.html.
+
+Thanks to all who contributed to this release, among them:
+
+* Bruno Oliveira
+* Florian Bruhin
+* Floris Bruynooghe
+* Jason R. Coombs
+* Ronny Pfannschmidt
+* wanghui
+
+
+Happy testing,
+The pytest Development Team

doc/en/warnings.rst

@@ -5,32 +5,18 @@ Warnings Capture
 
 .. versionadded:: 3.1
 
-.. warning::
-    pytest captures all warnings between tests, which prevents custom warning
-    filters in existing test suites from working. If this causes problems to your test suite,
-    this plugin can be disabled in your ``pytest.ini`` file with:
-
-    .. code-block:: ini
-
-        [pytest]
-        addopts = -p no:warnings
-
-    There's an ongoing discussion about this on `#2430
-    <https://github.com/pytest-dev/pytest/issues/2430>`_.
-
-
-Starting from version ``3.1``, pytest now automatically catches all warnings during test execution
+Starting from version ``3.1``, pytest now automatically catches warnings during test execution
 and displays them at the end of the session::
 
     # content of test_show_warnings.py
     import warnings
 
-    def deprecated_function():
-        warnings.warn("this function is deprecated, use another_function()", DeprecationWarning)
+    def api_v1():
+        warnings.warn(UserWarning("api v1, should use functions from v2"))
         return 1
 
     def test_one():
-        assert deprecated_function() == 1
+        assert api_v1() == 1
 
 Running pytest now produces this output::
 
@@ -44,35 +30,37 @@ Running pytest now produces this output::
     
     ======= warnings summary ========
     test_show_warnings.py::test_one
-      $REGENDOC_TMPDIR/test_show_warnings.py:4: DeprecationWarning: this function is deprecated, use another_function()
-        warnings.warn("this function is deprecated, use another_function()", DeprecationWarning)
+      $REGENDOC_TMPDIR/test_show_warnings.py:4: UserWarning: api v1, should use functions from v2
+        warnings.warn(UserWarning("api v1, should use functions from v2"))
     
     -- Docs: http://doc.pytest.org/en/latest/warnings.html
     ======= 1 passed, 1 warnings in 0.12 seconds ========
 
+Pytest by default catches all warnings except for ``DeprecationWarning`` and ``PendingDeprecationWarning``.
+
 The ``-W`` flag can be passed to control which warnings will be displayed or even turn
 them into errors::
 
-    $ pytest -q test_show_warnings.py -W error::DeprecationWarning
+    $ pytest -q test_show_warnings.py -W error::UserWarning
     F
     ======= FAILURES ========
     _______ test_one ________
     
         def test_one():
-    >       assert deprecated_function() == 1
+    >       assert api_v1() == 1
     
     test_show_warnings.py:8: 
     _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
     
-        def deprecated_function():
-    >       warnings.warn("this function is deprecated, use another_function()", DeprecationWarning)
-    E       DeprecationWarning: this function is deprecated, use another_function()
+        def api_v1():
+    >       warnings.warn(UserWarning("api v1, should use functions from v2"))
+    E       UserWarning: api v1, should use functions from v2
     
-    test_show_warnings.py:4: DeprecationWarning
+    test_show_warnings.py:4: UserWarning
     1 failed in 0.12 seconds
 
 The same option can be set in the ``pytest.ini`` file using the ``filterwarnings`` ini option.
-For example, the configuration below will ignore all deprecation warnings, but will transform
+For example, the configuration below will ignore all user warnings, but will transform
 all other warnings into errors.
 
 .. code-block:: ini
@@ -80,7 +68,7 @@ all other warnings into errors.
     [pytest]
     filterwarnings =
         error
-        ignore::DeprecationWarning
+        ignore::UserWarning
 
 
 When a warning matches more than one option in the list, the action for the last matching option
@@ -90,6 +78,19 @@ Both ``-W`` command-line option and ``filterwarnings`` ini option are based on P
 `-W option`_ and `warnings.simplefilter`_, so please refer to those sections in the Python
 documentation for other examples and advanced usage.
 
+.. note::
+
+    ``DeprecationWarning`` and ``PendingDeprecationWarning`` are hidden by the standard library
+    by default so you have to explicitly configure them to be displayed in your ``pytest.ini``:
+
+    .. code-block:: ini
+
+        [pytest]
+        filterwarnings =
+            once::DeprecationWarning
+            once::PendingDeprecationWarning
+
+
 *Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_
 *plugin.*
 
@@ -97,6 +98,19 @@ documentation for other examples and advanced usage.
 .. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter
 .. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings
 
+
+Disabling warning capture
+-------------------------
+
+This feature is enabled by default but can be disabled entirely in your ``pytest.ini`` file with:
+
+    .. code-block:: ini
+
+        [pytest]
+        addopts = -p no:warnings
+
+Or passing ``-p no:warnings`` in the command-line.
+
 .. _`asserting warnings`:
 
 .. _assertwarnings:

pyproject.toml

@@ -0,0 +1,35 @@
+[tool.towncrier]
+package = "pytest"
+filename = "CHANGELOG.rst"
+directory = "changelog/"
+template = "changelog/_template.rst"
+
+  [[tool.towncrier.type]]
+  directory = "removal"
+  name = "Deprecations and Removals"
+  showcontent = true
+
+  [[tool.towncrier.type]]
+  directory = "feature"
+  name = "Features"
+  showcontent = true
+
+  [[tool.towncrier.type]]
+  directory = "bugfix"
+  name = "Bug Fixes"
+  showcontent = true
+
+  [[tool.towncrier.type]]
+  directory = "vendor"
+  name = "Vendored Libraries"
+  showcontent = true
+
+  [[tool.towncrier.type]]
+  directory = "doc"
+  name = "Improved Documentation"
+  showcontent = true
+
+  [[tool.towncrier.type]]
+  directory = "trivial"
+  name = "Trivial Changes"
+  showcontent = false

scripts/check-manifest.py

@@ -11,10 +11,10 @@ from __future__ import print_function
 import os
 import subprocess
 import sys
-
+from check_manifest import main
 
 if os.path.isdir('.git'):
-    sys.exit(subprocess.call('check-manifest', shell=True))
+    sys.exit(main())
 else:
     print('No .git directory found, skipping checking the manifest file')
     sys.exit(0)

scripts/check-rst.py

@@ -0,0 +1,11 @@
+
+from __future__ import print_function
+
+import subprocess
+import glob
+import sys
+
+sys.exit(subprocess.call([
+    'rst-lint', '--encoding', 'utf-8',
+    'CHANGELOG.rst', 'HOWTORELEASE.rst', 'README.rst',
+] + glob.glob('changelog/[0-9]*.*')))

tasks/generate.py

@@ -96,9 +96,10 @@ def devpi_upload(ctx, version, user, password=None):
                 '(if not given assumed logged in)',
 })
 def pre_release(ctx, version, user, password=None):
-    """Generates new docs, release announcements and uploads a new release to devpi for testing."""    
+    """Generates new docs, release announcements and uploads a new release to devpi for testing."""
     announce(ctx, version)
     regen(ctx)
+    changelog(ctx, version, write_out=True)
 
     msg = 'Preparing release version {}'.format(version)
     check_call(['git', 'commit', '-a', '-m', msg])
@@ -146,3 +147,16 @@ def publish_release(ctx, version, user, pypi_name):
     print(' ', ','.join(emails))
     print()
     print('And announce it on twitter adding the #pytest hash tag.')
+
+
+@invoke.task(help={
+    'version': 'version being released',
+    'write_out': 'write changes to the actial changelog'
+})
+def changelog(ctx, version, write_out=False):
+    if write_out:
+        addopts = []
+    else:
+        addopts = ['--draft']
+    check_call(['towncrier', '--version', version] + addopts)
+

tasks/requirements.txt

@@ -1,3 +1,4 @@
 invoke
 tox
 gitpython
+towncrier

testing/test_recwarn.py

@@ -2,6 +2,8 @@ from __future__ import absolute_import, division, print_function
 import warnings
 import re
 import py
+import sys
+
 import pytest
 from _pytest.recwarn import WarningsRecorder
 
@@ -109,14 +111,17 @@ class TestDeprecatedCall(object):
             with pytest.deprecated_call():
                 self.dep(1)
 
-    def test_deprecated_call_as_context_manager(self):
-        with pytest.deprecated_call():
-            self.dep(0)
-
-    def test_deprecated_call_pending(self):
+    @pytest.mark.parametrize('warning_type', [PendingDeprecationWarning, DeprecationWarning])
+    @pytest.mark.parametrize('mode', ['context_manager', 'call'])
+    def test_deprecated_call_modes(self, warning_type, mode):
         def f():
-            py.std.warnings.warn(PendingDeprecationWarning("hi"))
-        pytest.deprecated_call(f)
+            warnings.warn(warning_type("hi"))
+            
+        if mode == 'call':
+            pytest.deprecated_call(f)
+        else:
+            with pytest.deprecated_call():
+                f()
 
     def test_deprecated_call_specificity(self):
         other_warnings = [Warning, UserWarning, SyntaxWarning, RuntimeWarning,
@@ -146,9 +151,12 @@ class TestDeprecatedCall(object):
                 pytest.deprecated_call(deprecated_function)
         """)
         result = testdir.runpytest()
-        # the 2 tests must pass, but the call to test_one() will generate a warning
-        # in pytest's summary
-        result.stdout.fnmatch_lines('*=== 2 passed, 1 warnings in *===')
+        # for some reason in py26 catch_warnings manages to catch the deprecation warning
+        # from deprecated_function(), even with default filters active (which ignore deprecation
+        # warnings)
+        py26 = sys.version_info[:2] == (2, 6)
+        expected = '*=== 2 passed in *===' if not py26 else '*=== 2 passed, 1 warnings in *==='
+        result.stdout.fnmatch_lines(expected)
 
 
 class TestWarns(object):

testing/test_warnings.py

@@ -20,8 +20,8 @@ def pyfile_with_warnings(testdir, request):
         module_name: '''
             import warnings
             def foo():
-                warnings.warn(PendingDeprecationWarning("functionality is pending deprecation"))
-                warnings.warn(DeprecationWarning("functionality is deprecated"))
+                warnings.warn(UserWarning("user warning"))
+                warnings.warn(RuntimeWarning("runtime warning"))
                 return 1
         ''',
         test_name: '''
@@ -43,11 +43,11 @@ def test_normal_flow(testdir, pyfile_with_warnings):
 
         '*test_normal_flow.py::test_func',
 
-        '*normal_flow_module.py:3: PendingDeprecationWarning: functionality is pending deprecation',
-        '*  warnings.warn(PendingDeprecationWarning("functionality is pending deprecation"))',
+        '*normal_flow_module.py:3: UserWarning: user warning',
+        '*  warnings.warn(UserWarning("user warning"))',
 
-        '*normal_flow_module.py:4: DeprecationWarning: functionality is deprecated',
-        '*  warnings.warn(DeprecationWarning("functionality is deprecated"))',
+        '*normal_flow_module.py:4: RuntimeWarning: runtime warning',
+        '*  warnings.warn(RuntimeWarning("runtime warning"))',
         '* 1 passed, 2 warnings*',
     ])
     assert result.stdout.str().count('test_normal_flow.py::test_func') == 1
@@ -90,8 +90,8 @@ def test_as_errors(testdir, pyfile_with_warnings, method):
             ''')
     result = testdir.runpytest(*args)
     result.stdout.fnmatch_lines([
-        'E       PendingDeprecationWarning: functionality is pending deprecation',
-        'as_errors_module.py:3: PendingDeprecationWarning',
+        'E       UserWarning: user warning',
+        'as_errors_module.py:3: UserWarning',
         '* 1 failed in *',
     ])
 
@@ -133,9 +133,7 @@ def test_unicode(testdir, pyfile_with_warnings):
     result = testdir.runpytest()
     result.stdout.fnmatch_lines([
         '*== %s ==*' % WARNINGS_SUMMARY_HEADER,
-
-        '*test_unicode.py:8: UserWarning: \u6d4b\u8bd5',
-        '*warnings.warn(u"\u6d4b\u8bd5")',
+        '*test_unicode.py:8: UserWarning: \u6d4b\u8bd5*',
         '* 1 passed, 1 warnings*',
     ])
 
@@ -163,6 +161,30 @@ def test_py2_unicode(testdir, pyfile_with_warnings):
 
         '*test_py2_unicode.py:8: UserWarning: \u6d4b\u8bd5',
         '*warnings.warn(u"\u6d4b\u8bd5")',
-        '*warnings.py:82: UnicodeWarning: This warning*\u6d4b\u8bd5',
+        '*warnings.py:*: UnicodeWarning: This warning*\u6d4b\u8bd5',
         '* 1 passed, 2 warnings*',
     ])
+
+
+def test_works_with_filterwarnings(testdir):
+    """Ensure our warnings capture does not mess with pre-installed filters (#2430)."""
+    testdir.makepyfile('''
+        import warnings
+
+        class MyWarning(Warning):
+            pass
+        
+        warnings.filterwarnings("error", category=MyWarning)
+        
+        class TestWarnings(object):
+            def test_my_warning(self):
+                try:
+                    warnings.warn(MyWarning("warn!"))
+                    assert False
+                except MyWarning:
+                    assert True
+    ''')
+    result = testdir.runpytest()
+    result.stdout.fnmatch_lines([
+        '*== 1 passed in *',
+    ])

tox.ini

@@ -61,7 +61,7 @@ deps =
 commands =
     {envpython} scripts/check-manifest.py
     flake8 pytest.py _pytest testing
-    rst-lint CHANGELOG.rst HOWTORELEASE.rst README.rst --encoding utf-8
+    {envpython} scripts/check-rst.py
 
 [testenv:py27-xdist]
 deps=pytest-xdist>=1.13
@@ -184,7 +184,7 @@ python_files=test_*.py *_test.py testing/*/*.py
 python_classes=Test Acceptance
 python_functions=test
 norecursedirs = .tox ja .hg cx_freeze_source
-filterwarnings= error
+filterwarnings=
                 # produced by path.local
                 ignore:bad escape.*:DeprecationWarning:re
                 # produced by path.readlines