Merge master into features

Several conflicts, mostly due to 2c402f4bd.

Conflicts:
	.pre-commit-config.yaml
	src/_pytest/outcomes.py
	src/_pytest/python_api.py
	tox.ini

.pre-commit-config.yaml

@@ -5,13 +5,11 @@ repos:
     hooks:
     -   id: black
         args: [--safe, --quiet]
-        language_version: python3
 -   repo: https://github.com/asottile/blacken-docs
     rev: v1.0.0
     hooks:
     -   id: blacken-docs
         additional_dependencies: [black==19.3b0]
-        language_version: python3
 -   repo: https://github.com/pre-commit/pre-commit-hooks
     rev: v2.2.3
     hooks:

.travis.yml

@@ -13,6 +13,10 @@ env:
   global:
     - PYTEST_ADDOPTS=-vv
 
+# setuptools-scm needs all tags in order to obtain a proper version
+git:
+  depth: false
+
 install:
   - python -m pip install --upgrade --pre tox
 

AUTHORS

@@ -239,6 +239,7 @@ Tareq Alayan
 Ted Xiao
 Thomas Grainger
 Thomas Hisch
+Tim Hoffmann
 Tim Strazny
 Tom Dalton
 Tom Viner
@@ -258,6 +259,7 @@ Wil Cooley
 William Lee
 Wim Glenn
 Wouter van Ackooy
+Xixi Zhao
 Xuan Luong
 Xuecong Liao
 Zac Hatfield-Dodds

CHANGELOG.rst

@@ -1,6 +1,6 @@
-=================
+=========
-Changelog history
+Changelog
-=================
+=========
 
 Versions follow `Semantic Versioning <https://semver.org/>`_ (``<major>.<minor>.<patch>``).
 
@@ -90,6 +90,24 @@ Removals
 - `#5412 <https://github.com/pytest-dev/pytest/issues/5412>`_: ``ExceptionInfo`` objects (returned by ``pytest.raises``) now have the same ``str`` representation as ``repr``, which
   avoids some confusion when users use ``print(e)`` to inspect the object.
 
+  This means code like:
+
+  .. code-block:: python
+
+        with pytest.raises(SomeException) as e:
+            ...
+        assert "some message" in str(e)
+
+
+  Needs to be changed to:
+
+  .. code-block:: python
+
+        with pytest.raises(SomeException) as e:
+            ...
+        assert "some message" in str(e.value)
+
+
 
 
 Deprecations
@@ -2173,10 +2191,10 @@ Features
   design. This introduces new ``Node.iter_markers(name)`` and
   ``Node.get_closest_marker(name)`` APIs. Users are **strongly encouraged** to
   read the `reasons for the revamp in the docs
-  <https://docs.pytest.org/en/latest/mark.html#marker-revamp-and-iteration>`_,
+  <https://docs.pytest.org/en/latest/historical-notes.html#marker-revamp-and-iteration>`_,
   or jump over to details about `updating existing code to use the new APIs
-  <https://docs.pytest.org/en/latest/mark.html#updating-code>`_. (`#3317
+  <https://docs.pytest.org/en/latest/historical-notes.html#updating-code>`_.
-  <https://github.com/pytest-dev/pytest/issues/3317>`_)
+  (`#3317 <https://github.com/pytest-dev/pytest/issues/3317>`_)
 
 - Now when ``@pytest.fixture`` is applied more than once to the same function a
   ``ValueError`` is raised. This buggy behavior would cause surprising problems
@@ -2582,10 +2600,10 @@ Features
   <https://github.com/pytest-dev/pytest/issues/3038>`_)
 
 - New `pytest_runtest_logfinish
-  <https://docs.pytest.org/en/latest/writing_plugins.html#_pytest.hookspec.pytest_runtest_logfinish>`_
+  <https://docs.pytest.org/en/latest/reference.html#_pytest.hookspec.pytest_runtest_logfinish>`_
   hook which is called when a test item has finished executing, analogous to
   `pytest_runtest_logstart
-  <https://docs.pytest.org/en/latest/writing_plugins.html#_pytest.hookspec.pytest_runtest_start>`_.
+  <https://docs.pytest.org/en/latest/reference.html#_pytest.hookspec.pytest_runtest_logstart>`_.
   (`#3101 <https://github.com/pytest-dev/pytest/issues/3101>`_)
 
 - Improve performance when collecting tests using many fixtures. (`#3107
@@ -3575,7 +3593,7 @@ Bug Fixes
   Thanks `@sirex`_ for the report and `@nicoddemus`_ for the PR.
 
 * Replace ``raise StopIteration`` usages in the code by simple ``returns`` to finish generators, in accordance to `PEP-479`_ (`#2160`_).
-  Thanks `@tgoodlet`_ for the report and `@nicoddemus`_ for the PR.
+  Thanks to `@nicoddemus`_ for the PR.
 
 * Fix internal errors when an unprintable ``AssertionError`` is raised inside a test.
   Thanks `@omerhadari`_ for the PR.
@@ -3706,7 +3724,7 @@ Bug Fixes
 
 .. _@syre: https://github.com/syre
 .. _@adler-j: https://github.com/adler-j
-.. _@d-b-w: https://bitbucket.org/d-b-w/
+.. _@d-b-w: https://github.com/d-b-w
 .. _@DuncanBetts: https://github.com/DuncanBetts
 .. _@dupuy: https://bitbucket.org/dupuy/
 .. _@kerrick-lyft: https://github.com/kerrick-lyft
@@ -3766,7 +3784,7 @@ Bug Fixes
 
 .. _@adborden: https://github.com/adborden
 .. _@cwitty: https://github.com/cwitty
-.. _@d_b_w: https://github.com/d_b_w
+.. _@d_b_w: https://github.com/d-b-w
 .. _@gdyuldin: https://github.com/gdyuldin
 .. _@matclab: https://github.com/matclab
 .. _@MSeifert04: https://github.com/MSeifert04
@@ -3801,7 +3819,7 @@ Bug Fixes
   Thanks `@axil`_ for the PR.
 
 * Explain a bad scope value passed to ``@fixture`` declarations or
-  a ``MetaFunc.parametrize()`` call. Thanks `@tgoodlet`_ for the PR.
+  a ``MetaFunc.parametrize()`` call.
 
 * This version includes ``pluggy-0.4.0``, which correctly handles
   ``VersionConflict`` errors in plugins (`#704`_).
@@ -3811,7 +3829,6 @@ Bug Fixes
 .. _@philpep: https://github.com/philpep
 .. _@raquel-ucl: https://github.com/raquel-ucl
 .. _@axil: https://github.com/axil
-.. _@tgoodlet: https://github.com/tgoodlet
 .. _@vlad-dragos: https://github.com/vlad-dragos
 
 .. _#1853: https://github.com/pytest-dev/pytest/issues/1853
@@ -4157,7 +4174,7 @@ time or change existing behaviors in order to make them less surprising/more use
 * Updated docstrings with a more uniform style.
 
 * Add stderr write for ``pytest.exit(msg)`` during startup. Previously the message was never shown.
-  Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@JonathonSonesen`_ and
+  Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@jgsonesen`_ and
   `@tomviner`_ for the PR.
 
 * No longer display the incorrect test deselection reason (`#1372`_).
@@ -4205,7 +4222,7 @@ time or change existing behaviors in order to make them less surprising/more use
   Thanks to `@Stranger6667`_ for the PR.
 
 * Fixed the total tests tally in junit xml output (`#1798`_).
-  Thanks to `@cryporchild`_ for the PR.
+  Thanks to `@cboelsen`_ for the PR.
 
 * Fixed off-by-one error with lines from ``request.node.warn``.
   Thanks to `@blueyed`_ for the PR.
@@ -4278,7 +4295,7 @@ time or change existing behaviors in order to make them less surprising/more use
 .. _@BeyondEvil: https://github.com/BeyondEvil
 .. _@blueyed: https://github.com/blueyed
 .. _@ceridwen: https://github.com/ceridwen
-.. _@cryporchild: https://github.com/cryporchild
+.. _@cboelsen: https://github.com/cboelsen
 .. _@csaftoiu: https://github.com/csaftoiu
 .. _@d6e: https://github.com/d6e
 .. _@davehunt: https://github.com/davehunt
@@ -4289,7 +4306,7 @@ time or change existing behaviors in order to make them less surprising/more use
 .. _@gprasad84: https://github.com/gprasad84
 .. _@graingert: https://github.com/graingert
 .. _@hartym: https://github.com/hartym
-.. _@JonathonSonesen: https://github.com/JonathonSonesen
+.. _@jgsonesen: https://github.com/jgsonesen
 .. _@kalekundert: https://github.com/kalekundert
 .. _@kvas-it: https://github.com/kvas-it
 .. _@marscher: https://github.com/marscher
@@ -4426,7 +4443,7 @@ time or change existing behaviors in order to make them less surprising/more use
 
 **Changes**
 
-* **Important**: `py.code <https://pylib.readthedocs.io/en/latest/code.html>`_ has been
+* **Important**: `py.code <https://pylib.readthedocs.io/en/stable/code.html>`_ has been
   merged into the ``pytest`` repository as ``pytest._code``. This decision
   was made because ``py.code`` had very few uses outside ``pytest`` and the
   fact that it was in a different repository made it difficult to fix bugs on

CONTRIBUTING.rst

@@ -5,8 +5,9 @@ Contribution getting started
 Contributions are highly welcomed and appreciated.  Every little help counts,
 so do not hesitate!
 
-.. contents:: Contribution links
+.. contents::
    :depth: 2
+   :backlinks: none
 
 
 .. _submitfeedback:

OPENCOLLECTIVE.rst

@@ -9,7 +9,7 @@ What is it
 ==========
 
 Open Collective is an online funding platform for open and transparent communities.
-It provide tools to raise money and share your finances in full transparency.
+It provides tools to raise money and share your finances in full transparency.
 
 It is the platform of choice for individuals and companies that want to make one-time or
 monthly donations directly to the project.
@@ -19,7 +19,7 @@ Funds
 
 The OpenCollective funds donated to pytest will be used to fund overall maintenance,
 local sprints, merchandising (stickers to distribute in conferences for example), and future
-gatherings of pytest developers (Sprints).
+gatherings of pytest developers (sprints).
 
 `Core contributors`_ which are contributing on a continuous basis are free to submit invoices
 to bill maintenance hours using the platform. How much each contributor should request is still an

README.rst

@@ -111,13 +111,13 @@ Consult the `Changelog <https://docs.pytest.org/en/latest/changelog.html>`__ pag
 Support pytest
 --------------
 
-You can support pytest by obtaining a `Tideflift subscription`_.
+You can support pytest by obtaining a `Tidelift subscription`_.
 
 Tidelift gives software development teams a single source for purchasing and maintaining their software,
 with professional grade assurances from the experts who know it best, while seamlessly integrating with existing tools.
 
 
-.. _`Tideflift subscription`: https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=readme
+.. _`Tidelift subscription`: https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=readme
 
 
 Security

changelog/5516.trivial.rst

@@ -0,0 +1 @@
+Cache node splitting function which can improve collection performance in very large test suites.

changelog/5524.bugfix.rst

@@ -0,0 +1,2 @@
+Fix issue where ``tmp_path`` and ``tmpdir`` would not remove directories containing files marked as read-only,
+which could lead to pytest crashing when executed a second time with the ``--basetemp`` option.

changelog/5578.bugfix.rst

@@ -0,0 +1,3 @@
+Improve type checking for some exception-raising functions (``pytest.xfail``, ``pytest.skip``, etc)
+so they provide better error messages when users meant to use marks (for example ``@pytest.xfail``
+instead of ``@pytest.mark.xfail``).

changelog/5606.bugfix.rst

@@ -0,0 +1,2 @@
+Fixed internal error when test functions were patched with objects that cannot be compared
+for truth values against others, like ``numpy`` arrays.

changelog/5634.bugfix.rst

@@ -0,0 +1,2 @@
+``pytest.exit`` is now correctly handled in ``unittest`` cases.
+This makes ``unittest`` cases handle ``quit`` from pytest's pdb correctly.

changelog/5650.bugfix.rst

@@ -0,0 +1 @@
+Improved output when parsing an ini configuration file fails.

changelog/5664.trivial.rst

@@ -0,0 +1,2 @@
+When invoking pytest's own testsuite with ``PYTHONDONTWRITEBYTECODE=1``,
+the ``test_xfail_handling`` test no longer fails.

doc/en/_templates/globaltoc.html

@@ -4,7 +4,7 @@
   <li><a href="{{ pathto('index') }}">Home</a></li>
   <li><a href="{{ pathto('getting-started') }}">Install</a></li>
   <li><a href="{{ pathto('contents') }}">Contents</a></li>
-  <li><a href="{{ pathto('reference') }}">Reference</a></li>
+  <li><a href="{{ pathto('reference') }}">API Reference</a></li>
   <li><a href="{{ pathto('example/index') }}">Examples</a></li>
   <li><a href="{{ pathto('customize') }}">Customize</a></li>
   <li><a href="{{ pathto('changelog') }}">Changelog</a></li>

doc/en/_themes/flask/layout.html

@@ -16,7 +16,7 @@
 {%- block footer %}
   <div class="footer">
     &copy; Copyright {{ copyright }}.
-    Created using <a href="http://sphinx.pocoo.org/">Sphinx</a>.
+    Created using <a href="https://www.sphinx-doc.org/">Sphinx</a> {{ sphinx_version }}.
   </div>
   {% if pagename == 'index' %}
   </div>

doc/en/_themes/flask/slim_searchbox.html

@@ -0,0 +1,15 @@
+{#
+    basic/searchbox.html with heading removed.
+#}
+{%- if pagename != "search" and builder != "singlehtml" %}
+<div id="searchbox" style="display: none" role="search">
+  <div class="searchformwrapper">
+    <form class="search" action="{{ pathto('search') }}" method="get">
+      <input type="text" name="q" aria-labelledby="searchlabel"
+        placeholder="Search"/>
+      <input type="submit" value="{{ _('Go') }}" />
+    </form>
+  </div>
+</div>
+<script type="text/javascript">$('#searchbox').show(0);</script>
+{%- endif %}

doc/en/_themes/flask/static/flasky.css_t

@@ -8,11 +8,12 @@
 
 {% set page_width = '1020px' %}
 {% set sidebar_width = '220px' %}
-/* orange of logo is #d67c29 but we use black for links for now */
+/* muted version of green logo color #C9D22A */
-{% set link_color = '#000' %}
+{% set link_color = '#606413' %}
-{% set link_hover_color = '#000' %}
+/* blue logo color */
+{% set link_hover_color = '#009de0' %}
 {% set base_font = 'sans-serif' %}
-{% set header_font = 'serif' %}
+{% set header_font = 'sans-serif' %}
 
 @import url("basic.css");
 
@@ -20,7 +21,7 @@
 
 body {
     font-family: {{ base_font }};
-    font-size: 17px;
+    font-size: 16px;
     background-color: white;
     color: #000;
     margin: 0;
@@ -78,13 +79,13 @@ div.related {
 }
 
 div.sphinxsidebar a {
-    color: #444;
     text-decoration: none;
-    border-bottom: 1px dotted #999;
+    border-bottom: none;
 }
 
 div.sphinxsidebar a:hover {
-    border-bottom: 1px solid #999;
+    color: {{ link_hover_color }};
+    border-bottom: 1px solid {{ link_hover_color }};
 }
 
 div.sphinxsidebar {
@@ -106,14 +107,14 @@ div.sphinxsidebar h3,
 div.sphinxsidebar h4 {
     font-family: {{ header_font }};
     color: #444;
-    font-size: 24px;
+    font-size: 21px;
     font-weight: normal;
-    margin: 0 0 5px 0;
+    margin: 16px 0 0 0;
     padding: 0;
 }
 
 div.sphinxsidebar h4 {
-    font-size: 20px;
+    font-size: 18px;
 }
 
 div.sphinxsidebar h3 a {
@@ -205,10 +206,22 @@ div.body p, div.body dd, div.body li {
     line-height: 1.4em;
 }
 
+ul.simple li {
+    margin-bottom: 0.5em;
+}
+
+div.topic ul.simple li {
+    margin-bottom: 0;
+}
+
+div.topic li > p:first-child {
+    margin-top: 0;
+    margin-bottom: 0;
+}
+
 div.admonition {
     background: #fafafa;
-    margin: 20px -30px;
+    padding: 10px 20px;
-    padding: 10px 30px;
     border-top: 1px solid #ccc;
     border-bottom: 1px solid #ccc;
 }
@@ -217,11 +230,6 @@ div.admonition tt.xref, div.admonition a tt {
     border-bottom: 1px solid #fafafa;
 }
 
-dd div.admonition {
-    margin-left: -60px;
-    padding-left: 60px;
-}
-
 div.admonition p.admonition-title {
     font-family: {{ header_font }};
     font-weight: normal;
@@ -231,7 +239,7 @@ div.admonition p.admonition-title {
     line-height: 1;
 }
 
-div.admonition p.last {
+div.admonition :last-child {
     margin-bottom: 0;
 }
 
@@ -243,7 +251,7 @@ dt:target, .highlight {
     background: #FAF3E8;
 }
 
-div.note {
+div.note, div.warning {
     background-color: #eee;
     border: 1px solid #ccc;
 }
@@ -257,6 +265,11 @@ div.topic {
     background-color: #eee;
 }
 
+div.topic a {
+    text-decoration: none;
+    border-bottom: none;
+}
+
 p.admonition-title {
     display: inline;
 }
@@ -358,21 +371,10 @@ ul, ol {
 
 pre {
     background: #eee;
-    padding: 7px 30px;
+    padding: 7px 12px;
-    margin: 15px -30px;
     line-height: 1.3em;
 }
 
-dl pre, blockquote pre, li pre {
-    margin-left: -60px;
-    padding-left: 60px;
-}
-
-dl dl pre {
-    margin-left: -90px;
-    padding-left: 90px;
-}
-
 tt {
     background-color: #ecf0f3;
     color: #222;
@@ -393,6 +395,20 @@ a.reference:hover {
     border-bottom: 1px solid {{ link_hover_color }};
 }
 
+li.toctree-l1 a.reference,
+li.toctree-l2 a.reference,
+li.toctree-l3 a.reference,
+li.toctree-l4 a.reference {
+    border-bottom: none;
+}
+
+li.toctree-l1 a.reference:hover,
+li.toctree-l2 a.reference:hover,
+li.toctree-l3 a.reference:hover,
+li.toctree-l4 a.reference:hover {
+    border-bottom: 1px solid {{ link_hover_color }};
+}
+
 a.footnote-reference {
     text-decoration: none;
     font-size: 0.7em;
@@ -408,6 +424,56 @@ a:hover tt {
     background: #EEE;
 }
 
+#reference div.section h2 {
+    /* separate code elements in the reference section */
+    border-top: 2px solid #ccc;
+    padding-top: 0.5em;
+}
+
+#reference div.section h3 {
+    /* separate code elements in the reference section */
+    border-top: 1px solid #ccc;
+    padding-top: 0.5em;
+}
+
+dl.class, dl.function {
+    margin-top: 1em;
+    margin-bottom: 1em;
+}
+
+dl.class > dd {
+    border-left: 3px solid #ccc;
+    margin-left: 0px;
+    padding-left: 30px;
+}
+
+dl.field-list {
+    flex-direction: column;
+}
+
+dl.field-list dd {
+    padding-left: 4em;
+    border-left: 3px solid #ccc;
+    margin-bottom: 0.5em;
+}
+
+dl.field-list dd > ul {
+    list-style: none;
+    padding-left: 0px;
+}
+
+dl.field-list dd > ul > li li :first-child {
+    text-indent: 0;
+}
+
+dl.field-list dd > ul > li :first-child {
+    text-indent: -2em;
+    padding-left: 0px;
+}
+
+dl.field-list dd > p:first-child {
+    text-indent: -2em;
+}
 
 @media screen and (max-width: 870px) {
 

doc/en/adopt.rst

@@ -24,11 +24,9 @@ The ideal pytest helper
  - feels confident in using pytest (e.g. has explored command line options, knows how to write parametrized tests, has an idea about conftest contents)
  - does not need to be an expert in every aspect!
 
-`Pytest helpers, sign up here`_! (preferably in February, hard deadline 22 March)
+Pytest helpers, sign up here! (preferably in February, hard deadline 22 March)
 
 
-.. _`Pytest helpers, sign up here`: http://goo.gl/forms/nxqAhqWt1P
-
 
 The ideal partner project
 -----------------------------------------
@@ -40,11 +38,9 @@ The ideal partner project
  - has the support of the core development team, in trying out pytest adoption
  - has no tests... or 100% test coverage... or somewhere in between!
 
-`Partner projects, sign up here`_! (by 22 March)
+Partner projects, sign up here! (by 22 March)
 
 
-.. _`Partner projects, sign up here`:  http://goo.gl/forms/ZGyqlHiwk3
-
 
 What does it mean to "adopt pytest"?
 -----------------------------------------
@@ -68,11 +64,11 @@ Progressive success might look like:
 It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies.
 
 .. _`nose and unittest`: faq.html#how-does-pytest-relate-to-nose-and-unittest
-.. _assert: asserts.html
+.. _assert: assert.html
 .. _pycmd: https://bitbucket.org/hpk42/pycmd/overview
 .. _`setUp/tearDown methods`: xunit_setup.html
 .. _fixtures: fixture.html
-.. _markers: markers.html
+.. _markers: mark.html
 .. _distributed: xdist.html
 
 

doc/en/announce/release-2.1.0.rst

@@ -12,7 +12,7 @@ courtesy of Benjamin Peterson.  You can now safely use ``assert``
 statements in test modules without having to worry about side effects
 or python optimization ("-OO") options.  This is achieved by rewriting
 assert statements in test modules upon import, using a PEP302 hook.
-See http://pytest.org/assert.html#advanced-assertion-introspection for
+See https://docs.pytest.org/en/latest/assert.html for
 detailed information.  The work has been partly sponsored by my company,
 merlinux GmbH.
 

doc/en/announce/release-2.9.0.rst

@@ -75,7 +75,7 @@ The py.test Development Team
 
 **Changes**
 
-* **Important**: `py.code <https://pylib.readthedocs.io/en/latest/code.html>`_ has been
+* **Important**: `py.code <https://pylib.readthedocs.io/en/stable/code.html>`_ has been
   merged into the ``pytest`` repository as ``pytest._code``. This decision
   was made because ``py.code`` had very few uses outside ``pytest`` and the
   fact that it was in a different repository made it difficult to fix bugs on
@@ -88,7 +88,7 @@ The py.test Development Team
   **experimental**, so you definitely should not import it explicitly!
 
   Please note that the original ``py.code`` is still available in
-  `pylib <https://pylib.readthedocs.io>`_.
+  `pylib <https://pylib.readthedocs.io/en/stable/>`_.
 
 * ``pytest_enter_pdb`` now optionally receives the pytest config object.
   Thanks `@nicoddemus`_ for the PR.

doc/en/announce/release-2.9.2.rst

@@ -66,8 +66,8 @@ The py.test Development Team
 
 .. _#510: https://github.com/pytest-dev/pytest/issues/510
 .. _#1506: https://github.com/pytest-dev/pytest/pull/1506
-.. _#1496: https://github.com/pytest-dev/pytest/issue/1496
+.. _#1496: https://github.com/pytest-dev/pytest/issues/1496
-.. _#1524: https://github.com/pytest-dev/pytest/issue/1524
+.. _#1524: https://github.com/pytest-dev/pytest/pull/1524
 
 .. _@astraw38: https://github.com/astraw38
 .. _@hackebrot: https://github.com/hackebrot

doc/en/conf.py

@@ -15,7 +15,6 @@
 #
 # The full version, including alpha/beta/rc tags.
 # The short X.Y version.
-import datetime
 import os
 import sys
 
@@ -63,7 +62,6 @@ master_doc = "contents"
 
 # General information about the project.
 project = "pytest"
-year = datetime.datetime.utcnow().year
 copyright = "2015–2019, holger krekel and pytest-dev team"
 
 
@@ -167,18 +165,18 @@ html_favicon = "img/pytest1favi.ico"
 
 html_sidebars = {
     "index": [
+        "slim_searchbox.html",
         "sidebarintro.html",
         "globaltoc.html",
         "links.html",
         "sourcelink.html",
-        "searchbox.html",
     ],
     "**": [
+        "slim_searchbox.html",
         "globaltoc.html",
         "relations.html",
         "links.html",
         "sourcelink.html",
-        "searchbox.html",
     ],
 }
 

doc/en/example/fixtures/test_fixtures_order.py

@@ -0,0 +1,38 @@
+import pytest
+
+# fixtures documentation order example
+order = []
+
+
+@pytest.fixture(scope="session")
+def s1():
+    order.append("s1")
+
+
+@pytest.fixture(scope="module")
+def m1():
+    order.append("m1")
+
+
+@pytest.fixture
+def f1(f3):
+    order.append("f1")
+
+
+@pytest.fixture
+def f3():
+    order.append("f3")
+
+
+@pytest.fixture(autouse=True)
+def a1():
+    order.append("a1")
+
+
+@pytest.fixture
+def f2():
+    order.append("f2")
+
+
+def test_order(f1, m1, f2, s1):
+    assert order == ["s1", "m1", "a1", "f3", "f1", "f2"]

doc/en/example/markers.rst

@@ -336,7 +336,7 @@ apply a marker to an individual test instance:
 
     @pytest.mark.foo
     @pytest.mark.parametrize(
-        ("n", "expected"), [(1, 2), pytest.param((1, 3), marks=pytest.mark.bar), (2, 3)]
+        ("n", "expected"), [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)]
     )
     def test_increment(n, expected):
         assert n + 1 == expected

doc/en/example/parametrize.rst

@@ -552,13 +552,13 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker:
     platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python
     cachedir: $PYTHON_PREFIX/.pytest_cache
     rootdir: $REGENDOC_TMPDIR
-    collecting ... collected 17 items / 14 deselected / 3 selected
+    collecting ... collected 18 items / 15 deselected / 3 selected
 
     test_pytest_param_example.py::test_eval[1+7-8] PASSED                [ 33%]
     test_pytest_param_example.py::test_eval[basic_2+4] PASSED            [ 66%]
     test_pytest_param_example.py::test_eval[basic_6*9] XFAIL             [100%]
 
-    ============ 2 passed, 14 deselected, 1 xfailed in 0.12 seconds ============
+    ============ 2 passed, 15 deselected, 1 xfailed in 0.12 seconds ============
 
 As the result:
 

doc/en/fixture.rst

@@ -289,51 +289,29 @@ are finalized when the last test of a *package* finishes.
     Use this new feature sparingly and please make sure to report any issues you find.
 
 
-Higher-scoped fixtures are instantiated first
+Order: Higher-scoped fixtures are instantiated first
----------------------------------------------
+----------------------------------------------------
 
 
 
 Within a function request for features, fixture of higher-scopes (such as ``session``) are instantiated first than
 lower-scoped fixtures (such as ``function`` or ``class``). The relative order of fixtures of same scope follows
-the declared order in the test function and honours dependencies between fixtures.
+the declared order in the test function and honours dependencies between fixtures. Autouse fixtures will be
+instantiated before explicitly used fixtures.
 
 Consider the code below:
 
-.. code-block:: python
+.. literalinclude:: example/fixtures/test_fixtures_order.py
-
-    @pytest.fixture(scope="session")
-    def s1():
-        pass
-
-
-    @pytest.fixture(scope="module")
-    def m1():
-        pass
-
-
-    @pytest.fixture
-    def f1(tmpdir):
-        pass
-
-
-    @pytest.fixture
-    def f2():
-        pass
-
-
-    def test_foo(f1, m1, f2, s1):
-        ...
-
 
 The fixtures requested by ``test_foo`` will be instantiated in the following order:
 
 1. ``s1``: is the highest-scoped fixture (``session``).
 2. ``m1``: is the second highest-scoped fixture (``module``).
-3. ``tmpdir``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point
+3. ``a1``: is a ``function``-scoped ``autouse`` fixture: it will be instantiated before other fixtures
-   because it is a dependency of ``f1``.
+   within the same scope.
-4. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list.
+4. ``f3``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point
-5. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list.
+5. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list.
+6. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list.
 
 
 .. _`finalization`:
@@ -400,6 +378,34 @@ The ``smtp_connection`` connection will be closed after the test finished
 execution because the ``smtp_connection`` object automatically closes when
 the ``with`` statement ends.
 
+Using the contextlib.ExitStack context manager finalizers will always be called
+regardless if the fixture *setup* code raises an exception. This is handy to properly
+close all resources created by a fixture even if one of them fails to be created/acquired:
+
+.. code-block:: python
+
+    # content of test_yield3.py
+
+    import contextlib
+
+    import pytest
+
+
+    @contextlib.contextmanager
+    def connect(port):
+        ...  # create connection
+        yield
+        ...  # close connection
+
+
+    @pytest.fixture
+    def equipments():
+        with contextlib.ExitStack() as stack:
+            yield [stack.enter_context(connect(port)) for port in ("C1", "C3", "C28")]
+
+In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still
+be properly closed.
+
 Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the
 *teardown* code (after the ``yield``) will not be called.
 
@@ -428,27 +434,39 @@ Here's the ``smtp_connection`` fixture changed to use ``addfinalizer`` for clean
         return smtp_connection  # provide the fixture value
 
 
-Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test
+Here's the ``equipments`` fixture changed to use ``addfinalizer`` for cleanup:
-ends, but ``addfinalizer`` has two key differences over ``yield``:
 
-1. It is possible to register multiple finalizer functions.
+.. code-block:: python
+
+    # content of test_yield3.py
+
+    import contextlib
+    import functools
+
+    import pytest
+
+
+    @contextlib.contextmanager
+    def connect(port):
+        ...  # create connection
+        yield
+        ...  # close connection
 
-2. Finalizers will always be called regardless if the fixture *setup* code raises an exception.
-   This is handy to properly close all resources created by a fixture even if one of them
-   fails to be created/acquired::
 
     @pytest.fixture
     def equipments(request):
         r = []
-            for port in ('C1', 'C3', 'C28'):
+        for port in ("C1", "C3", "C28"):
-                equip = connect(port)
+            cm = connect(port)
-                request.addfinalizer(equip.disconnect)
+            equip = cm.__enter__()
+            request.addfinalizer(functools.partial(cm.__exit__, None, None, None))
             r.append(equip)
         return r
 
-   In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still
+
-   be properly closed. Of course, if an exception happens before the finalize function is
+Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test
-   registered then it will not be executed.
+ends. Of course, if an exception happens before the finalize function is registered then it
+will not be executed.
 
 
 .. _`request-context`:
@@ -522,7 +540,7 @@ of a fixture is needed multiple times in a single test. Instead of returning
 data directly, the fixture instead returns a function which generates the data.
 This function can then be called multiple times in the test.
 
-Factories can have have parameters as needed::
+Factories can have parameters as needed::
 
     @pytest.fixture
     def make_customer_record():

doc/en/flaky.rst

@@ -122,4 +122,4 @@ Resources
 * Google:
 
   * `Flaky Tests at Google and How We Mitigate Them <https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html>`_ by John Micco, 2016
-  * `Where do Google's flaky tests come from? <https://docs.google.com/document/d/1mZ0-Kc97DI_F3tf_GBW_NB_aqka-P1jVOsFfufxqUUM/edit#heading=h.ec0r4fypsleh>`_  by Jeff Listfield, 2017
+  * `Where do Google's flaky tests come from? <https://testing.googleblog.com/2017/04/where-do-our-flaky-tests-come-from.html>`_  by Jeff Listfield, 2017

doc/en/getting-started.rst

@@ -142,7 +142,7 @@ The first test passed and the second failed. You can easily see the intermediate
 Request a unique temporary directory for functional tests
 --------------------------------------------------------------
 
-``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html#builtinfixtures>`_ to request arbitrary resources, like a unique temporary directory::
+``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory::
 
     # content of test_tmpdir.py
     def test_needsfiles(tmpdir):

doc/en/index.rst

@@ -61,7 +61,7 @@ Features
 
 - Can run :ref:`unittest <unittest>` (including trial) and :ref:`nose <noseintegration>` test suites out of the box;
 
-- Python Python 3.5+ and PyPy 3;
+- Python 3.5+ and PyPy 3;
 
 - Rich plugin architecture, with over 315+ `external plugins <http://plugincompat.herokuapp.com>`_ and thriving community;
 

doc/en/license.rst

@@ -9,7 +9,7 @@ Distributed under the terms of the `MIT`_ license, pytest is free and open sourc
 
     The MIT License (MIT)
 
-    Copyright (c) 2004-2017 Holger Krekel and others
+    Copyright (c) 2004-2019 Holger Krekel and others
 
     Permission is hereby granted, free of charge, to any person obtaining a copy of
     this software and associated documentation files (the "Software"), to deal in

doc/en/links.inc

@@ -14,7 +14,7 @@
 .. _`distribute docs`:
 .. _`distribute`: https://pypi.org/project/distribute/
 .. _`pip`: https://pypi.org/project/pip/
-.. _`venv`: https://docs.python.org/3/library/venv.html/
+.. _`venv`: https://docs.python.org/3/library/venv.html
 .. _`virtualenv`: https://pypi.org/project/virtualenv/
 .. _hudson: http://hudson-ci.org/
 .. _jenkins: http://jenkins-ci.org/

doc/en/mark.rst

@@ -40,7 +40,7 @@ You can register custom marks in your ``pytest.ini`` file like this:
 
 Note that everything after the ``:`` is an optional description.
 
-Alternatively, you can register new markers programatically in a
+Alternatively, you can register new markers programmatically in a
 :ref:`pytest_configure <initialization-hooks>` hook:
 
 .. code-block:: python

doc/en/monkeypatch.rst

@@ -46,10 +46,13 @@ environment variable is missing, or to set multiple values to a known variable.
 :py:meth:`monkeypatch.setenv` and :py:meth:`monkeypatch.delenv` can be used for
 these patches.
 
-4. Use :py:meth:`monkeypatch.syspath_prepend` to modify the system ``$PATH`` safely, and
+4. Use ``monkeypatch.setenv("PATH", value, prepend=os.pathsep)`` to modify ``$PATH``, and
 :py:meth:`monkeypatch.chdir` to change the context of the current working directory
 during a test.
 
+5. Use py:meth:`monkeypatch.syspath_prepend` to modify ``sys.path`` which will also
+call :py:meth:`pkg_resources.fixup_namespace_packages` and :py:meth:`importlib.invalidate_caches`.
+
 See the `monkeypatch blog post`_ for some introduction material
 and a discussion of its motivation.
 

doc/en/projects.rst

@@ -28,7 +28,6 @@ Here are some examples of projects using ``pytest`` (please send notes via :ref:
 * `sentry <https://getsentry.com/welcome/>`_, realtime app-maintenance and exception tracking
 * `Astropy <http://www.astropy.org/>`_ and `affiliated packages <http://www.astropy.org/affiliated/index.html>`_
 * `tox <http://testrun.org/tox>`_, virtualenv/Hudson integration tool
-* `PIDA <http://pida.co.uk>`_ framework for integrated development
 * `PyPM <http://code.activestate.com/pypm/>`_ ActiveState's package manager
 * `Fom <http://packages.python.org/Fom/>`_ a fluid object mapper for FluidDB
 * `applib <https://github.com/ActiveState/applib>`_ cross-platform utilities
@@ -37,8 +36,7 @@ Here are some examples of projects using ``pytest`` (please send notes via :ref:
 * `mwlib <https://pypi.org/project/mwlib/>`_ mediawiki parser and utility library
 * `The Translate Toolkit <http://translate.sourceforge.net/wiki/toolkit/index>`_ for localization and conversion
 * `execnet <http://codespeak.net/execnet>`_ rapid multi-Python deployment
-* `pylib <https://py.readthedocs.io>`_ cross-platform path, IO, dynamic code library
+* `pylib <https://pylib.readthedocs.io/en/stable/>`_ cross-platform path, IO, dynamic code library
-* `Pacha <http://pacha.cafepais.com/>`_ configuration management in five minutes
 * `bbfreeze <https://pypi.org/project/bbfreeze/>`_ create standalone executables from Python scripts
 * `pdb++ <http://bitbucket.org/antocuni/pdb>`_ a fancier version of PDB
 * `py-s3fuse <http://code.google.com/p/py-s3fuse/>`_ Amazon S3 FUSE based filesystem
@@ -77,7 +75,7 @@ Some organisations using pytest
 * `Tandberg <http://www.tandberg.com/>`_
 * `Shootq <http://web.shootq.com/>`_
 * `Stups department of Heinrich Heine University Duesseldorf <http://www.stups.uni-duesseldorf.de/projects.php>`_
-* `cellzome <http://www.cellzome.com/>`_
+* cellzome
 * `Open End, Gothenborg <http://www.openend.se>`_
 * `Laboratory of Bioinformatics, Warsaw <http://genesilico.pl/>`_
 * `merlinux, Germany <http://merlinux.eu>`_

doc/en/reference.rst

@@ -1,5 +1,5 @@
-Reference
+API Reference
-=========
+=============
 
 This page contains the full reference to pytest's API.
 

doc/en/talks.rst

@@ -4,9 +4,8 @@ Talks and Tutorials
 
 .. sidebar:: Next Open Trainings
 
-   - `Training at Europython 2019 <https://ep2019.europython.eu/talks/94WEnsY-introduction-to-pytest/>`_, 8th July 2019, Basel, Switzerland.
-
    - `Training at Workshoptage 2019 <https://workshoptage.ch/workshops/2019/test-driven-development-fuer-python-mit-pytest/>`_ (German), 10th September 2019, Rapperswil, Switzerland.
+   - `3 day hands-on workshop covering pytest, tox and devpi: "Professional Testing with Python" <https://python-academy.com/courses/specialtopics/python_course_testing.html>`_ (English), October 21 - 23, 2019, Leipzig, Germany.
 
 .. _`funcargs`: funcargs.html
 

doc/en/warnings.rst

@@ -127,7 +127,7 @@ decorator or to all tests in a module by setting the ``pytestmark`` variable:
 *Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_
 *plugin.*
 
-.. _`-W option`: https://docs.python.org/3/using/cmdline.html?highlight=#cmdoption-W
+.. _`-W option`: https://docs.python.org/3/using/cmdline.html#cmdoption-w
 .. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter
 .. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings
 

doc/en/writing_plugins.rst

@@ -164,7 +164,7 @@ If a package is installed this way, ``pytest`` will load
 .. note::
 
     Make sure to include ``Framework :: Pytest`` in your list of
-    `PyPI classifiers <https://python-packaging-user-guide.readthedocs.io/distributing/#classifiers>`_
+    `PyPI classifiers <https://pypi.org/classifiers/>`_
     to make it easy for users to find your plugin.
 
 

src/_pytest/cacheprovider.py

@@ -14,7 +14,7 @@ import py
 import pytest
 from .pathlib import Path
 from .pathlib import resolve_from_str
-from .pathlib import rmtree
+from .pathlib import rm_rf
 
 README_CONTENT = """\
 # pytest cache directory #
@@ -44,7 +44,7 @@ class Cache:
     def for_config(cls, config):
         cachedir = cls.cache_dir_from_config(config)
         if config.getoption("cacheclear") and cachedir.exists():
-            rmtree(cachedir, force=True)
+            rm_rf(cachedir)
             cachedir.mkdir()
         return cls(cachedir, config)
 

src/_pytest/compat.py

@@ -70,13 +70,18 @@ def num_mock_patch_args(function):
     patchings = getattr(function, "patchings", None)
     if not patchings:
         return 0
-    mock_modules = [sys.modules.get("mock"), sys.modules.get("unittest.mock")]
+
-    if any(mock_modules):
+    mock_sentinel = getattr(sys.modules.get("mock"), "DEFAULT", object())
-        sentinels = [m.DEFAULT for m in mock_modules if m is not None]
+    ut_mock_sentinel = getattr(sys.modules.get("unittest.mock"), "DEFAULT", object())
+
     return len(
-            [p for p in patchings if not p.attribute_name and p.new in sentinels]
+        [
+            p
+            for p in patchings
+            if not p.attribute_name
+            and (p.new is mock_sentinel or p.new is ut_mock_sentinel)
+        ]
     )
-    return len(patchings)
 
 
 def getfuncargnames(function, is_method=False, cls=None):

src/_pytest/config/argparsing.py

@@ -399,7 +399,7 @@ class DropShorterLongHelpFormatter(argparse.HelpFormatter):
     """shorten help for long options that differ only in extra hyphens
 
     - collapse **long** options that are the same except for extra hyphens
-    - special action attribute map_long_option allows surpressing additional
+    - special action attribute map_long_option allows suppressing additional
       long options
     - shortcut if there are only two options and one of them is a short one
     - cache result on action object as this is called at least 2 times

src/_pytest/config/findpaths.py

@@ -30,7 +30,11 @@ def getcfg(args, config=None):
             for inibasename in inibasenames:
                 p = base.join(inibasename)
                 if exists(p):
+                    try:
                         iniconfig = py.iniconfig.IniConfig(p)
+                    except py.iniconfig.ParseError as exc:
+                        raise UsageError(str(exc))
+
                     if (
                         inibasename == "setup.cfg"
                         and "tool:pytest" in iniconfig.sections

src/_pytest/mark/structures.py

@@ -171,9 +171,7 @@ class Mark:
 @attr.s
 class MarkDecorator:
     """ A decorator for test functions and test classes.  When applied
-    it will create :class:`MarkInfo` objects which may be
+    it will create :class:`Mark` objects which are often created like this::
-    :ref:`retrieved by hooks as item keywords <excontrolskip>`.
-    MarkDecorator instances are often created like this::
 
         mark1 = pytest.mark.NAME              # simple MarkDecorator
         mark2 = pytest.mark.NAME(name1=value) # parametrized MarkDecorator
@@ -185,6 +183,7 @@ class MarkDecorator:
             pass
 
     When a MarkDecorator instance is called it does the following:
+
     1. If called with a single class as its only positional argument and no
        additional keyword arguments, it attaches itself to the class so it
        gets applied automatically to all test cases found in that class.

src/_pytest/nodes.py

@@ -1,5 +1,6 @@
 import os
 import warnings
+from functools import lru_cache
 
 import py
 
@@ -13,6 +14,7 @@ SEP = "/"
 tracebackcutdir = py.path.local(_pytest.__file__).dirpath()
 
 
+@lru_cache(maxsize=None)
 def _splitnode(nodeid):
     """Split a nodeid into constituent 'parts'.
 
@@ -30,11 +32,12 @@ def _splitnode(nodeid):
     """
     if nodeid == "":
         # If there is no root node at all, return an empty list so the caller's logic can remain sane
-        return []
+        return ()
     parts = nodeid.split(SEP)
     # Replace single last element 'test_foo.py::Bar' with multiple elements 'test_foo.py', 'Bar'
     parts[-1:] = parts[-1].split("::")
-    return parts
+    # Convert parts into a tuple to avoid possible errors with caching of a mutable type
+    return tuple(parts)
 
 
 def ischildnode(baseid, nodeid):

src/_pytest/outcomes.py

@@ -18,6 +18,12 @@ class OutcomeException(BaseException):
     """
 
     def __init__(self, msg: Optional[str] = None, pytrace: bool = True) -> None:
+        if msg is not None and not isinstance(msg, str):
+            error_msg = (
+                "{} expected string as 'msg' parameter, got '{}' instead.\n"
+                "Perhaps you meant to use a mark?"
+            )
+            raise TypeError(error_msg.format(type(self).__name__, type(msg).__name__))
         BaseException.__init__(self, msg)
         self.msg = msg
         self.pytrace = pytrace

src/_pytest/pathlib.py

@@ -7,12 +7,15 @@ import os
 import shutil
 import sys
 import uuid
+import warnings
+from functools import partial
 from os.path import expanduser
 from os.path import expandvars
 from os.path import isabs
 from os.path import sep
 from posixpath import sep as posix_sep
 
+from _pytest.warning_types import PytestWarning
 
 if sys.version_info[:2] >= (3, 6):
     from pathlib import Path, PurePath
@@ -32,17 +35,53 @@ def ensure_reset_dir(path):
     ensures the given path is an empty directory
     """
     if path.exists():
-        rmtree(path, force=True)
+        rm_rf(path)
     path.mkdir()
 
 
-def rmtree(path, force=False):
+def on_rm_rf_error(func, path: str, exc, *, start_path):
-    if force:
+    """Handles known read-only errors during rmtree."""
-        # NOTE: ignore_errors might leave dead folders around.
+    excvalue = exc[1]
-        #       Python needs a rm -rf as a followup.
+
-        shutil.rmtree(str(path), ignore_errors=True)
+    if not isinstance(excvalue, PermissionError):
-    else:
+        warnings.warn(
-        shutil.rmtree(str(path))
+            PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue))
+        )
+        return
+
+    if func not in (os.rmdir, os.remove, os.unlink):
+        warnings.warn(
+            PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue))
+        )
+        return
+
+    # Chmod + retry.
+    import stat
+
+    def chmod_rw(p: str):
+        mode = os.stat(p).st_mode
+        os.chmod(p, mode | stat.S_IRUSR | stat.S_IWUSR)
+
+    # For files, we need to recursively go upwards in the directories to
+    # ensure they all are also writable.
+    p = Path(path)
+    if p.is_file():
+        for parent in p.parents:
+            chmod_rw(str(parent))
+            # stop when we reach the original path passed to rm_rf
+            if parent == start_path:
+                break
+    chmod_rw(str(path))
+
+    func(path)
+
+
+def rm_rf(path: Path):
+    """Remove the path contents recursively, even if some elements
+    are read-only.
+    """
+    onerror = partial(on_rm_rf_error, start_path=path)
+    shutil.rmtree(str(path), onerror=onerror)
 
 
 def find_prefixed(root, prefix):
@@ -168,7 +207,7 @@ def maybe_delete_a_numbered_dir(path):
 
         garbage = parent.joinpath("garbage-{}".format(uuid.uuid4()))
         path.rename(garbage)
-        rmtree(garbage, force=True)
+        rm_rf(garbage)
     except (OSError, EnvironmentError):
         #  known races:
         #  * other process did a cleanup at the same time

src/_pytest/stepwise.py

@@ -71,7 +71,6 @@ class StepwisePlugin:
         config.hook.pytest_deselected(items=already_passed)
 
     def pytest_runtest_logreport(self, report):
-        # Skip this hook if plugin is not active or the test is xfailed.
         if not self.active:
             return
 

src/_pytest/terminal.py

@@ -692,7 +692,7 @@ class TerminalReporter:
             else:
                 excrepr.reprcrash.toterminal(self._tw)
                 self._tw.line(
-                    "(to show a full traceback on KeyboardInterrupt use --fulltrace)",
+                    "(to show a full traceback on KeyboardInterrupt use --full-trace)",
                     yellow=True,
                 )
 

src/_pytest/unittest.py

@@ -6,6 +6,7 @@ import _pytest._code
 import pytest
 from _pytest.compat import getimfunc
 from _pytest.config import hookimpl
+from _pytest.outcomes import exit
 from _pytest.outcomes import fail
 from _pytest.outcomes import skip
 from _pytest.outcomes import xfail
@@ -154,6 +155,11 @@ class TestCaseFunction(Function):
         self.__dict__.setdefault("_excinfo", []).append(excinfo)
 
     def addError(self, testcase, rawexcinfo):
+        try:
+            if isinstance(rawexcinfo[1], exit.Exception):
+                exit(rawexcinfo[1].msg)
+        except TypeError:
+            pass
         self._addexcinfo(rawexcinfo)
 
     def addFailure(self, testcase, rawexcinfo):

testing/code/test_excinfo.py

@@ -378,7 +378,7 @@ def test_excinfo_no_python_sourcecode(tmpdir):
     excinfo = pytest.raises(ValueError, template.render, h=h)
     for item in excinfo.traceback:
         print(item)  # XXX: for some reason jinja.Template.render is printed in full
-        item.source  # shouldnt fail
+        item.source  # shouldn't fail
         if item.path.basename == "test.txt":
             assert str(item.source) == "{{ h()}}:"
 

testing/deprecated_test.py

@@ -36,7 +36,7 @@ def test_terminal_reporter_writer_attr(pytestconfig):
     assert terminal_reporter.writer is terminal_reporter._tw
 
 
-@pytest.mark.parametrize("plugin", deprecated.DEPRECATED_EXTERNAL_PLUGINS)
+@pytest.mark.parametrize("plugin", sorted(deprecated.DEPRECATED_EXTERNAL_PLUGINS))
 @pytest.mark.filterwarnings("default")
 def test_external_plugins_integrated(testdir, plugin):
     testdir.syspathinsert()

testing/python/integration.py

@@ -104,21 +104,15 @@ class TestMockDecoration:
         values = getfuncargnames(f)
         assert values == ("x",)
 
-    @pytest.mark.xfail(
+    def test_getfuncargnames_patching(self):
-        strict=False, reason="getfuncargnames breaks if mock is imported"
-    )
-    def test_wrapped_getfuncargnames_patching(self):
         from _pytest.compat import getfuncargnames
+        from unittest.mock import patch
 
-        def wrap(f):
+        class T:
-            def func():
+            def original(self, x, y, z):
                 pass
 
-            func.__wrapped__ = f
+        @patch.object(T, "original")
-            func.patchings = ["qwe"]
-            return func
-
-        @wrap
         def f(x, y, z):
             pass
 
@@ -126,7 +120,6 @@ class TestMockDecoration:
         assert values == ("y", "z")
 
     def test_unittest_mock(self, testdir):
-        pytest.importorskip("unittest.mock")
         testdir.makepyfile(
             """
             import unittest.mock
@@ -142,7 +135,6 @@ class TestMockDecoration:
         reprec.assertoutcome(passed=1)
 
     def test_unittest_mock_and_fixture(self, testdir):
-        pytest.importorskip("unittest.mock")
         testdir.makepyfile(
             """
             import os.path
@@ -164,7 +156,6 @@ class TestMockDecoration:
         reprec.assertoutcome(passed=1)
 
     def test_unittest_mock_and_pypi_mock(self, testdir):
-        pytest.importorskip("unittest.mock")
         pytest.importorskip("mock", "1.0.1")
         testdir.makepyfile(
             """
@@ -187,6 +178,34 @@ class TestMockDecoration:
         reprec = testdir.inline_run()
         reprec.assertoutcome(passed=2)
 
+    def test_mock_sentinel_check_against_numpy_like(self, testdir):
+        """Ensure our function that detects mock arguments compares against sentinels using
+        identity to circumvent objects which can't be compared with equality against others
+        in a truth context, like with numpy arrays (#5606).
+        """
+        testdir.makepyfile(
+            dummy="""
+            class NumpyLike:
+                def __init__(self, value):
+                    self.value = value
+                def __eq__(self, other):
+                    raise ValueError("like numpy, cannot compare against others for truth")
+            FOO = NumpyLike(10)
+        """
+        )
+        testdir.makepyfile(
+            """
+            from unittest.mock import patch
+            import dummy
+            class Test(object):
+                @patch("dummy.FOO", new=dummy.NumpyLike(50))
+                def test_hello(self):
+                    assert dummy.FOO.value == 50
+        """
+        )
+        reprec = testdir.inline_run()
+        reprec.assertoutcome(passed=1)
+
     def test_mock(self, testdir):
         pytest.importorskip("mock", "1.0.1")
         testdir.makepyfile(

testing/test_config.py

@@ -123,6 +123,12 @@ class TestParseIni:
         config = testdir.parseconfigure(sub)
         assert config.getini("minversion") == "2.0"
 
+    def test_ini_parse_error(self, testdir):
+        testdir.tmpdir.join("pytest.ini").write("addopts = -x")
+        result = testdir.runpytest()
+        assert result.ret != 0
+        result.stderr.fnmatch_lines(["ERROR: *pytest.ini:1: no section header defined"])
+
     @pytest.mark.xfail(reason="probably not needed")
     def test_confcutdir(self, testdir):
         sub = testdir.mkdir("sub")

testing/test_junitxml.py

@@ -592,7 +592,7 @@ class TestPython:
         assert "hx" in fnode.toxml()
 
     def test_assertion_binchars(self, testdir):
-        """this test did fail when the escaping wasnt strict"""
+        """this test did fail when the escaping wasn't strict"""
         testdir.makepyfile(
             """
 
@@ -715,7 +715,7 @@ def test_dont_configure_on_slaves(tmpdir):
             return "pytest"
 
         junitprefix = None
-        # XXX: shouldnt need tmpdir ?
+        # XXX: shouldn't need tmpdir ?
         xmlpath = str(tmpdir.join("junix.xml"))
         register = gotten.append
 

testing/test_mark.py

@@ -430,7 +430,7 @@ class TestFunctional:
                 def test_b(self):
                     assert True
                 class TestC(object):
-                    # this one didnt get marked
+                    # this one didn't get marked
                     def test_d(self):
                         assert True
         """

testing/test_nose.py

@@ -253,7 +253,7 @@ def test_apiwrapper_problem_issue260(testdir):
 
 
 def test_setup_teardown_linking_issue265(testdir):
-    # we accidentally didnt integrate nose setupstate with normal setupstate
+    # we accidentally didn't integrate nose setupstate with normal setupstate
     # this test ensures that won't happen again
     testdir.makepyfile(
         '''

testing/test_pytester.py

@@ -230,8 +230,8 @@ class TestInlineRunModulesCleanup:
     ):
         spy_factory = self.spy_factory()
         monkeypatch.setattr(pytester, "SysModulesSnapshot", spy_factory)
-        original = dict(sys.modules)
         testdir.syspathinsert()
+        original = dict(sys.modules)
         testdir.makepyfile(import1="# you son of a silly person")
         testdir.makepyfile(import2="# my hovercraft is full of eels")
         test_mod = testdir.makepyfile(

testing/test_runner.py

@@ -11,6 +11,7 @@ from _pytest import main
 from _pytest import outcomes
 from _pytest import reports
 from _pytest import runner
+from _pytest.outcomes import OutcomeException
 
 
 class TestSetupState:
@@ -990,3 +991,18 @@ class TestReportContents:
         rep = reports[1]
         assert rep.capstdout == ""
         assert rep.capstderr == ""
+
+
+def test_outcome_exception_bad_msg():
+    """Check that OutcomeExceptions validate their input to prevent confusing errors (#5578)"""
+
+    def func():
+        pass
+
+    expected = (
+        "OutcomeException expected string as 'msg' parameter, got 'function' instead.\n"
+        "Perhaps you meant to use a mark?"
+    )
+    with pytest.raises(TypeError) as excinfo:
+        OutcomeException(func)
+    assert str(excinfo.value) == expected

testing/test_skipping.py

@@ -1066,7 +1066,8 @@ def test_module_level_skip_error(testdir):
     testdir.makepyfile(
         """
         import pytest
-        @pytest.skip
+        pytest.skip("skip_module_level")
+
         def test_func():
             assert True
     """

testing/test_stepwise.py

@@ -207,6 +207,7 @@ def test_xfail_handling(testdir):
 
     # because we are writing to the same file, mtime might not be affected enough to
     # invalidate the cache, making this next run flaky
+    if testdir.tmpdir.join("__pycache__").exists():
         testdir.tmpdir.join("__pycache__").remove()
     testdir.makepyfile(contents.format(assert_value="0", strict="True"))
     result = testdir.runpytest("--sw", "-v")

testing/test_terminal.py

@@ -233,7 +233,7 @@ class TestTerminal:
             )
         else:
             result.stdout.fnmatch_lines(
-                ["(to show a full traceback on KeyboardInterrupt use --fulltrace)"]
+                ["(to show a full traceback on KeyboardInterrupt use --full-trace)"]
             )
         result.stdout.fnmatch_lines(["*KeyboardInterrupt*"])
 

testing/test_tmpdir.py

@@ -1,3 +1,5 @@
+import os
+import stat
 import sys
 
 import attr
@@ -303,22 +305,6 @@ class TestNumberedDir:
             p, consider_lock_dead_if_created_before=p.stat().st_mtime + 1
         )
 
-    def test_rmtree(self, tmp_path):
-        from _pytest.pathlib import rmtree
-
-        adir = tmp_path / "adir"
-        adir.mkdir()
-        rmtree(adir)
-
-        assert not adir.exists()
-
-        adir.mkdir()
-        afile = adir / "afile"
-        afile.write_bytes(b"aa")
-
-        rmtree(adir, force=True)
-        assert not adir.exists()
-
     def test_cleanup_ignores_symlink(self, tmp_path):
         the_symlink = tmp_path / (self.PREFIX + "current")
         attempt_symlink_to(the_symlink, tmp_path / (self.PREFIX + "5"))
@@ -331,6 +317,83 @@ class TestNumberedDir:
         assert folder.is_dir()
 
 
+class TestRmRf:
+    def test_rm_rf(self, tmp_path):
+        from _pytest.pathlib import rm_rf
+
+        adir = tmp_path / "adir"
+        adir.mkdir()
+        rm_rf(adir)
+
+        assert not adir.exists()
+
+        adir.mkdir()
+        afile = adir / "afile"
+        afile.write_bytes(b"aa")
+
+        rm_rf(adir)
+        assert not adir.exists()
+
+    def test_rm_rf_with_read_only_file(self, tmp_path):
+        """Ensure rm_rf can remove directories with read-only files in them (#5524)"""
+        from _pytest.pathlib import rm_rf
+
+        fn = tmp_path / "dir/foo.txt"
+        fn.parent.mkdir()
+
+        fn.touch()
+
+        self.chmod_r(fn)
+
+        rm_rf(fn.parent)
+
+        assert not fn.parent.is_dir()
+
+    def chmod_r(self, path):
+        mode = os.stat(str(path)).st_mode
+        os.chmod(str(path), mode & ~stat.S_IWRITE)
+
+    def test_rm_rf_with_read_only_directory(self, tmp_path):
+        """Ensure rm_rf can remove read-only directories (#5524)"""
+        from _pytest.pathlib import rm_rf
+
+        adir = tmp_path / "dir"
+        adir.mkdir()
+
+        (adir / "foo.txt").touch()
+        self.chmod_r(adir)
+
+        rm_rf(adir)
+
+        assert not adir.is_dir()
+
+    def test_on_rm_rf_error(self, tmp_path):
+        from _pytest.pathlib import on_rm_rf_error
+
+        adir = tmp_path / "dir"
+        adir.mkdir()
+
+        fn = adir / "foo.txt"
+        fn.touch()
+        self.chmod_r(fn)
+
+        # unknown exception
+        with pytest.warns(pytest.PytestWarning):
+            exc_info = (None, RuntimeError(), None)
+            on_rm_rf_error(os.unlink, str(fn), exc_info, start_path=tmp_path)
+            assert fn.is_file()
+
+        # unknown function
+        with pytest.warns(pytest.PytestWarning):
+            exc_info = (None, PermissionError(), None)
+            on_rm_rf_error(None, str(fn), exc_info, start_path=tmp_path)
+            assert fn.is_file()
+
+        exc_info = (None, PermissionError(), None)
+        on_rm_rf_error(os.unlink, str(fn), exc_info, start_path=tmp_path)
+        assert not fn.is_file()
+
+
 def attempt_symlink_to(path, to_path):
     """Try to make a symlink from "path" to "to_path", skipping in case this platform
     does not support it or we don't have sufficient privileges (common on Windows)."""
@@ -342,3 +405,24 @@ def attempt_symlink_to(path, to_path):
 
 def test_tmpdir_equals_tmp_path(tmpdir, tmp_path):
     assert Path(tmpdir) == tmp_path
+
+
+def test_basetemp_with_read_only_files(testdir):
+    """Integration test for #5524"""
+    testdir.makepyfile(
+        """
+        import os
+        import stat
+
+        def test(tmp_path):
+            fn = tmp_path / 'foo.txt'
+            fn.write_text('hello')
+            mode = os.stat(str(fn)).st_mode
+            os.chmod(str(fn), mode & ~stat.S_IREAD)
+    """
+    )
+    result = testdir.runpytest("--basetemp=tmp")
+    assert result.ret == 0
+    # running a second time and ensure we don't crash
+    result = testdir.runpytest("--basetemp=tmp")
+    assert result.ret == 0

testing/test_unittest.py

@@ -1048,3 +1048,39 @@ def test_setup_inheritance_skipping(testdir, test_name, expected_outcome):
     testdir.copy_example("unittest/{}".format(test_name))
     result = testdir.runpytest()
     result.stdout.fnmatch_lines(["* {} in *".format(expected_outcome)])
+
+
+def test_BdbQuit(testdir):
+    testdir.makepyfile(
+        test_foo="""
+        import unittest
+
+        class MyTestCase(unittest.TestCase):
+            def test_bdbquit(self):
+                import bdb
+                raise bdb.BdbQuit()
+
+            def test_should_not_run(self):
+                pass
+    """
+    )
+    reprec = testdir.inline_run()
+    reprec.assertoutcome(failed=1, passed=1)
+
+
+def test_exit_outcome(testdir):
+    testdir.makepyfile(
+        test_foo="""
+        import pytest
+        import unittest
+
+        class MyTestCase(unittest.TestCase):
+            def test_exit_outcome(self):
+                pytest.exit("pytest_exit called")
+
+            def test_should_not_run(self):
+                pass
+    """
+    )
+    result = testdir.runpytest()
+    result.stdout.fnmatch_lines(["*Exit: pytest_exit called*", "*= no tests ran in *"])

tox.ini

@@ -127,11 +127,10 @@ norecursedirs = testing/example_scripts
 xfail_strict=true
 filterwarnings =
     error
+    default:Using or importing the ABCs:DeprecationWarning:unittest2.*
     ignore:Module already imported so cannot be rewritten:pytest.PytestWarning
-    # produced by path.local
+    # produced by python3.6/site.py itself (3.6.7 on Travis, could not trigger it with 3.6.8).
-    ignore:bad escape.*:DeprecationWarning:re
+    ignore:.*U.*mode is deprecated:DeprecationWarning:(?!(pytest|_pytest))
-    # produced by path.readlines
-    ignore:.*U.*mode is deprecated:DeprecationWarning
     # produced by pytest-xdist
     ignore:.*type argument to addoption.*:DeprecationWarning
     # produced by python >=3.5 on execnet (pytest-xdist)
@@ -139,6 +138,8 @@ filterwarnings =
     # pytest's own futurewarnings
     ignore::pytest.PytestExperimentalApiWarning
     # Do not cause SyntaxError for invalid escape sequences in py37.
+    # Those are caught/handled by pyupgrade, and not easy to filter with the
+    # module being the filename (with .py removed).
     default:invalid escape sequence:DeprecationWarning
     # ignore use of unregistered marks, because we use many to test the implementation
     ignore::_pytest.warning_types.PytestUnknownMarkWarning