diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index 59128a4fb..b82693c18 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -5,13 +5,11 @@ repos: hooks: - id: black args: [--safe, --quiet] - language_version: python3 - repo: https://github.com/asottile/blacken-docs rev: v1.0.0 hooks: - id: blacken-docs additional_dependencies: [black==19.3b0] - language_version: python3 - repo: https://github.com/pre-commit/pre-commit-hooks rev: v2.2.3 hooks: diff --git a/.travis.yml b/.travis.yml index 8053eed65..af33d672e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -13,6 +13,10 @@ env: global: - PYTEST_ADDOPTS=-vv +# setuptools-scm needs all tags in order to obtain a proper version +git: + depth: false + install: - python -m pip install --upgrade --pre tox diff --git a/AUTHORS b/AUTHORS index 140f4de3d..88bbfe352 100644 --- a/AUTHORS +++ b/AUTHORS @@ -239,6 +239,7 @@ Tareq Alayan Ted Xiao Thomas Grainger Thomas Hisch +Tim Hoffmann Tim Strazny Tom Dalton Tom Viner @@ -258,6 +259,7 @@ Wil Cooley William Lee Wim Glenn Wouter van Ackooy +Xixi Zhao Xuan Luong Xuecong Liao Zac Hatfield-Dodds diff --git a/CHANGELOG.rst b/CHANGELOG.rst index c4ab6f1bd..811f7475d 100644 --- a/CHANGELOG.rst +++ b/CHANGELOG.rst @@ -1,6 +1,6 @@ -================= -Changelog history -================= +========= +Changelog +========= Versions follow `Semantic Versioning `_ (``..``). @@ -90,6 +90,24 @@ Removals - `#5412 `_: ``ExceptionInfo`` objects (returned by ``pytest.raises``) now have the same ``str`` representation as ``repr``, which avoids some confusion when users use ``print(e)`` to inspect the object. + This means code like: + + .. code-block:: python + + with pytest.raises(SomeException) as e: + ... + assert "some message" in str(e) + + + Needs to be changed to: + + .. code-block:: python + + with pytest.raises(SomeException) as e: + ... + assert "some message" in str(e.value) + + Deprecations @@ -2173,10 +2191,10 @@ Features design. This introduces new ``Node.iter_markers(name)`` and ``Node.get_closest_marker(name)`` APIs. Users are **strongly encouraged** to read the `reasons for the revamp in the docs - `_, + `_, or jump over to details about `updating existing code to use the new APIs - `_. (`#3317 - `_) + `_. + (`#3317 `_) - Now when ``@pytest.fixture`` is applied more than once to the same function a ``ValueError`` is raised. This buggy behavior would cause surprising problems @@ -2582,10 +2600,10 @@ Features `_) - New `pytest_runtest_logfinish - `_ + `_ hook which is called when a test item has finished executing, analogous to `pytest_runtest_logstart - `_. + `_. (`#3101 `_) - Improve performance when collecting tests using many fixtures. (`#3107 @@ -3575,7 +3593,7 @@ Bug Fixes Thanks `@sirex`_ for the report and `@nicoddemus`_ for the PR. * Replace ``raise StopIteration`` usages in the code by simple ``returns`` to finish generators, in accordance to `PEP-479`_ (`#2160`_). - Thanks `@tgoodlet`_ for the report and `@nicoddemus`_ for the PR. + Thanks to `@nicoddemus`_ for the PR. * Fix internal errors when an unprintable ``AssertionError`` is raised inside a test. Thanks `@omerhadari`_ for the PR. @@ -3706,7 +3724,7 @@ Bug Fixes .. _@syre: https://github.com/syre .. _@adler-j: https://github.com/adler-j -.. _@d-b-w: https://bitbucket.org/d-b-w/ +.. _@d-b-w: https://github.com/d-b-w .. _@DuncanBetts: https://github.com/DuncanBetts .. _@dupuy: https://bitbucket.org/dupuy/ .. _@kerrick-lyft: https://github.com/kerrick-lyft @@ -3766,7 +3784,7 @@ Bug Fixes .. _@adborden: https://github.com/adborden .. _@cwitty: https://github.com/cwitty -.. _@d_b_w: https://github.com/d_b_w +.. _@d_b_w: https://github.com/d-b-w .. _@gdyuldin: https://github.com/gdyuldin .. _@matclab: https://github.com/matclab .. _@MSeifert04: https://github.com/MSeifert04 @@ -3801,7 +3819,7 @@ Bug Fixes Thanks `@axil`_ for the PR. * Explain a bad scope value passed to ``@fixture`` declarations or - a ``MetaFunc.parametrize()`` call. Thanks `@tgoodlet`_ for the PR. + a ``MetaFunc.parametrize()`` call. * This version includes ``pluggy-0.4.0``, which correctly handles ``VersionConflict`` errors in plugins (`#704`_). @@ -3811,7 +3829,6 @@ Bug Fixes .. _@philpep: https://github.com/philpep .. _@raquel-ucl: https://github.com/raquel-ucl .. _@axil: https://github.com/axil -.. _@tgoodlet: https://github.com/tgoodlet .. _@vlad-dragos: https://github.com/vlad-dragos .. _#1853: https://github.com/pytest-dev/pytest/issues/1853 @@ -4157,7 +4174,7 @@ time or change existing behaviors in order to make them less surprising/more use * Updated docstrings with a more uniform style. * Add stderr write for ``pytest.exit(msg)`` during startup. Previously the message was never shown. - Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@JonathonSonesen`_ and + Thanks `@BeyondEvil`_ for reporting `#1210`_. Thanks to `@jgsonesen`_ and `@tomviner`_ for the PR. * No longer display the incorrect test deselection reason (`#1372`_). @@ -4205,7 +4222,7 @@ time or change existing behaviors in order to make them less surprising/more use Thanks to `@Stranger6667`_ for the PR. * Fixed the total tests tally in junit xml output (`#1798`_). - Thanks to `@cryporchild`_ for the PR. + Thanks to `@cboelsen`_ for the PR. * Fixed off-by-one error with lines from ``request.node.warn``. Thanks to `@blueyed`_ for the PR. @@ -4278,7 +4295,7 @@ time or change existing behaviors in order to make them less surprising/more use .. _@BeyondEvil: https://github.com/BeyondEvil .. _@blueyed: https://github.com/blueyed .. _@ceridwen: https://github.com/ceridwen -.. _@cryporchild: https://github.com/cryporchild +.. _@cboelsen: https://github.com/cboelsen .. _@csaftoiu: https://github.com/csaftoiu .. _@d6e: https://github.com/d6e .. _@davehunt: https://github.com/davehunt @@ -4289,7 +4306,7 @@ time or change existing behaviors in order to make them less surprising/more use .. _@gprasad84: https://github.com/gprasad84 .. _@graingert: https://github.com/graingert .. _@hartym: https://github.com/hartym -.. _@JonathonSonesen: https://github.com/JonathonSonesen +.. _@jgsonesen: https://github.com/jgsonesen .. _@kalekundert: https://github.com/kalekundert .. _@kvas-it: https://github.com/kvas-it .. _@marscher: https://github.com/marscher @@ -4426,7 +4443,7 @@ time or change existing behaviors in order to make them less surprising/more use **Changes** -* **Important**: `py.code `_ has been +* **Important**: `py.code `_ has been merged into the ``pytest`` repository as ``pytest._code``. This decision was made because ``py.code`` had very few uses outside ``pytest`` and the fact that it was in a different repository made it difficult to fix bugs on diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 128df6661..5ef418e0b 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -5,8 +5,9 @@ Contribution getting started Contributions are highly welcomed and appreciated. Every little help counts, so do not hesitate! -.. contents:: Contribution links +.. contents:: :depth: 2 + :backlinks: none .. _submitfeedback: diff --git a/OPENCOLLECTIVE.rst b/OPENCOLLECTIVE.rst index 9b7ddcd06..8c1c90281 100644 --- a/OPENCOLLECTIVE.rst +++ b/OPENCOLLECTIVE.rst @@ -9,7 +9,7 @@ What is it ========== Open Collective is an online funding platform for open and transparent communities. -It provide tools to raise money and share your finances in full transparency. +It provides tools to raise money and share your finances in full transparency. It is the platform of choice for individuals and companies that want to make one-time or monthly donations directly to the project. @@ -19,7 +19,7 @@ Funds The OpenCollective funds donated to pytest will be used to fund overall maintenance, local sprints, merchandising (stickers to distribute in conferences for example), and future -gatherings of pytest developers (Sprints). +gatherings of pytest developers (sprints). `Core contributors`_ which are contributing on a continuous basis are free to submit invoices to bill maintenance hours using the platform. How much each contributor should request is still an diff --git a/README.rst b/README.rst index 9739a1bda..301e49538 100644 --- a/README.rst +++ b/README.rst @@ -111,13 +111,13 @@ Consult the `Changelog `__ pag Support pytest -------------- -You can support pytest by obtaining a `Tideflift subscription`_. +You can support pytest by obtaining a `Tidelift subscription`_. Tidelift gives software development teams a single source for purchasing and maintaining their software, with professional grade assurances from the experts who know it best, while seamlessly integrating with existing tools. -.. _`Tideflift subscription`: https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=readme +.. _`Tidelift subscription`: https://tidelift.com/subscription/pkg/pypi-pytest?utm_source=pypi-pytest&utm_medium=referral&utm_campaign=readme Security diff --git a/changelog/5516.trivial.rst b/changelog/5516.trivial.rst new file mode 100644 index 000000000..2f6b4e35e --- /dev/null +++ b/changelog/5516.trivial.rst @@ -0,0 +1 @@ +Cache node splitting function which can improve collection performance in very large test suites. diff --git a/changelog/5524.bugfix.rst b/changelog/5524.bugfix.rst new file mode 100644 index 000000000..96ebbd43e --- /dev/null +++ b/changelog/5524.bugfix.rst @@ -0,0 +1,2 @@ +Fix issue where ``tmp_path`` and ``tmpdir`` would not remove directories containing files marked as read-only, +which could lead to pytest crashing when executed a second time with the ``--basetemp`` option. diff --git a/changelog/5578.bugfix.rst b/changelog/5578.bugfix.rst new file mode 100644 index 000000000..5f6c39185 --- /dev/null +++ b/changelog/5578.bugfix.rst @@ -0,0 +1,3 @@ +Improve type checking for some exception-raising functions (``pytest.xfail``, ``pytest.skip``, etc) +so they provide better error messages when users meant to use marks (for example ``@pytest.xfail`` +instead of ``@pytest.mark.xfail``). diff --git a/changelog/5606.bugfix.rst b/changelog/5606.bugfix.rst new file mode 100644 index 000000000..82332ba99 --- /dev/null +++ b/changelog/5606.bugfix.rst @@ -0,0 +1,2 @@ +Fixed internal error when test functions were patched with objects that cannot be compared +for truth values against others, like ``numpy`` arrays. diff --git a/changelog/5634.bugfix.rst b/changelog/5634.bugfix.rst new file mode 100644 index 000000000..a2a282f93 --- /dev/null +++ b/changelog/5634.bugfix.rst @@ -0,0 +1,2 @@ +``pytest.exit`` is now correctly handled in ``unittest`` cases. +This makes ``unittest`` cases handle ``quit`` from pytest's pdb correctly. diff --git a/changelog/5650.bugfix.rst b/changelog/5650.bugfix.rst new file mode 100644 index 000000000..db57a40b9 --- /dev/null +++ b/changelog/5650.bugfix.rst @@ -0,0 +1 @@ +Improved output when parsing an ini configuration file fails. diff --git a/changelog/5664.trivial.rst b/changelog/5664.trivial.rst new file mode 100644 index 000000000..3928454ef --- /dev/null +++ b/changelog/5664.trivial.rst @@ -0,0 +1,2 @@ +When invoking pytest's own testsuite with ``PYTHONDONTWRITEBYTECODE=1``, +the ``test_xfail_handling`` test no longer fails. diff --git a/doc/en/_templates/globaltoc.html b/doc/en/_templates/globaltoc.html index 39cebb968..50c2239e5 100644 --- a/doc/en/_templates/globaltoc.html +++ b/doc/en/_templates/globaltoc.html @@ -4,7 +4,7 @@
  • Home
  • Install
  • Contents
    • -
  • Reference
    • +
  • API Reference
  • Examples
  • Customize
  • Changelog
    • diff --git a/doc/en/_themes/flask/layout.html b/doc/en/_themes/flask/layout.html index 19c43fbbe..f2fa8e6aa 100644 --- a/doc/en/_themes/flask/layout.html +++ b/doc/en/_themes/flask/layout.html @@ -16,7 +16,7 @@ {%- block footer %}
    © Copyright {{ copyright }}. - Created using Sphinx. + Created using Sphinx {{ sphinx_version }}.
    {% if pagename == 'index' %} diff --git a/doc/en/_themes/flask/slim_searchbox.html b/doc/en/_themes/flask/slim_searchbox.html new file mode 100644 index 000000000..e98ad4ed9 --- /dev/null +++ b/doc/en/_themes/flask/slim_searchbox.html @@ -0,0 +1,15 @@ +{# + basic/searchbox.html with heading removed. +#} +{%- if pagename != "search" and builder != "singlehtml" %} + + +{%- endif %} diff --git a/doc/en/_themes/flask/static/flasky.css_t b/doc/en/_themes/flask/static/flasky.css_t index 6b593da29..108c85401 100644 --- a/doc/en/_themes/flask/static/flasky.css_t +++ b/doc/en/_themes/flask/static/flasky.css_t @@ -8,11 +8,12 @@ {% set page_width = '1020px' %} {% set sidebar_width = '220px' %} -/* orange of logo is #d67c29 but we use black for links for now */ -{% set link_color = '#000' %} -{% set link_hover_color = '#000' %} +/* muted version of green logo color #C9D22A */ +{% set link_color = '#606413' %} +/* blue logo color */ +{% set link_hover_color = '#009de0' %} {% set base_font = 'sans-serif' %} -{% set header_font = 'serif' %} +{% set header_font = 'sans-serif' %} @import url("basic.css"); @@ -20,7 +21,7 @@ body { font-family: {{ base_font }}; - font-size: 17px; + font-size: 16px; background-color: white; color: #000; margin: 0; @@ -78,13 +79,13 @@ div.related { } div.sphinxsidebar a { - color: #444; text-decoration: none; - border-bottom: 1px dotted #999; + border-bottom: none; } div.sphinxsidebar a:hover { - border-bottom: 1px solid #999; + color: {{ link_hover_color }}; + border-bottom: 1px solid {{ link_hover_color }}; } div.sphinxsidebar { @@ -106,14 +107,14 @@ div.sphinxsidebar h3, div.sphinxsidebar h4 { font-family: {{ header_font }}; color: #444; - font-size: 24px; + font-size: 21px; font-weight: normal; - margin: 0 0 5px 0; + margin: 16px 0 0 0; padding: 0; } div.sphinxsidebar h4 { - font-size: 20px; + font-size: 18px; } div.sphinxsidebar h3 a { @@ -205,10 +206,22 @@ div.body p, div.body dd, div.body li { line-height: 1.4em; } +ul.simple li { + margin-bottom: 0.5em; +} + +div.topic ul.simple li { + margin-bottom: 0; +} + +div.topic li > p:first-child { + margin-top: 0; + margin-bottom: 0; +} + div.admonition { background: #fafafa; - margin: 20px -30px; - padding: 10px 30px; + padding: 10px 20px; border-top: 1px solid #ccc; border-bottom: 1px solid #ccc; } @@ -217,11 +230,6 @@ div.admonition tt.xref, div.admonition a tt { border-bottom: 1px solid #fafafa; } -dd div.admonition { - margin-left: -60px; - padding-left: 60px; -} - div.admonition p.admonition-title { font-family: {{ header_font }}; font-weight: normal; @@ -231,7 +239,7 @@ div.admonition p.admonition-title { line-height: 1; } -div.admonition p.last { +div.admonition :last-child { margin-bottom: 0; } @@ -243,7 +251,7 @@ dt:target, .highlight { background: #FAF3E8; } -div.note { +div.note, div.warning { background-color: #eee; border: 1px solid #ccc; } @@ -257,6 +265,11 @@ div.topic { background-color: #eee; } +div.topic a { + text-decoration: none; + border-bottom: none; +} + p.admonition-title { display: inline; } @@ -358,21 +371,10 @@ ul, ol { pre { background: #eee; - padding: 7px 30px; - margin: 15px -30px; + padding: 7px 12px; line-height: 1.3em; } -dl pre, blockquote pre, li pre { - margin-left: -60px; - padding-left: 60px; -} - -dl dl pre { - margin-left: -90px; - padding-left: 90px; -} - tt { background-color: #ecf0f3; color: #222; @@ -393,6 +395,20 @@ a.reference:hover { border-bottom: 1px solid {{ link_hover_color }}; } +li.toctree-l1 a.reference, +li.toctree-l2 a.reference, +li.toctree-l3 a.reference, +li.toctree-l4 a.reference { + border-bottom: none; +} + +li.toctree-l1 a.reference:hover, +li.toctree-l2 a.reference:hover, +li.toctree-l3 a.reference:hover, +li.toctree-l4 a.reference:hover { + border-bottom: 1px solid {{ link_hover_color }}; +} + a.footnote-reference { text-decoration: none; font-size: 0.7em; @@ -408,6 +424,56 @@ a:hover tt { background: #EEE; } +#reference div.section h2 { + /* separate code elements in the reference section */ + border-top: 2px solid #ccc; + padding-top: 0.5em; +} + +#reference div.section h3 { + /* separate code elements in the reference section */ + border-top: 1px solid #ccc; + padding-top: 0.5em; +} + +dl.class, dl.function { + margin-top: 1em; + margin-bottom: 1em; +} + +dl.class > dd { + border-left: 3px solid #ccc; + margin-left: 0px; + padding-left: 30px; +} + +dl.field-list { + flex-direction: column; +} + +dl.field-list dd { + padding-left: 4em; + border-left: 3px solid #ccc; + margin-bottom: 0.5em; +} + +dl.field-list dd > ul { + list-style: none; + padding-left: 0px; +} + +dl.field-list dd > ul > li li :first-child { + text-indent: 0; +} + +dl.field-list dd > ul > li :first-child { + text-indent: -2em; + padding-left: 0px; +} + +dl.field-list dd > p:first-child { + text-indent: -2em; +} @media screen and (max-width: 870px) { diff --git a/doc/en/adopt.rst b/doc/en/adopt.rst index 710f431be..e3c0477bc 100644 --- a/doc/en/adopt.rst +++ b/doc/en/adopt.rst @@ -24,11 +24,9 @@ The ideal pytest helper - feels confident in using pytest (e.g. has explored command line options, knows how to write parametrized tests, has an idea about conftest contents) - does not need to be an expert in every aspect! -`Pytest helpers, sign up here`_! (preferably in February, hard deadline 22 March) +Pytest helpers, sign up here! (preferably in February, hard deadline 22 March) -.. _`Pytest helpers, sign up here`: http://goo.gl/forms/nxqAhqWt1P - The ideal partner project ----------------------------------------- @@ -40,11 +38,9 @@ The ideal partner project - has the support of the core development team, in trying out pytest adoption - has no tests... or 100% test coverage... or somewhere in between! -`Partner projects, sign up here`_! (by 22 March) +Partner projects, sign up here! (by 22 March) -.. _`Partner projects, sign up here`: http://goo.gl/forms/ZGyqlHiwk3 - What does it mean to "adopt pytest"? ----------------------------------------- @@ -68,11 +64,11 @@ Progressive success might look like: It may be after the month is up, the partner project decides that pytest is not right for it. That's okay - hopefully the pytest team will also learn something about its weaknesses or deficiencies. .. _`nose and unittest`: faq.html#how-does-pytest-relate-to-nose-and-unittest -.. _assert: asserts.html +.. _assert: assert.html .. _pycmd: https://bitbucket.org/hpk42/pycmd/overview .. _`setUp/tearDown methods`: xunit_setup.html .. _fixtures: fixture.html -.. _markers: markers.html +.. _markers: mark.html .. _distributed: xdist.html diff --git a/doc/en/announce/release-2.1.0.rst b/doc/en/announce/release-2.1.0.rst index 831548ac2..2a2181d97 100644 --- a/doc/en/announce/release-2.1.0.rst +++ b/doc/en/announce/release-2.1.0.rst @@ -12,7 +12,7 @@ courtesy of Benjamin Peterson. You can now safely use ``assert`` statements in test modules without having to worry about side effects or python optimization ("-OO") options. This is achieved by rewriting assert statements in test modules upon import, using a PEP302 hook. -See http://pytest.org/assert.html#advanced-assertion-introspection for +See https://docs.pytest.org/en/latest/assert.html for detailed information. The work has been partly sponsored by my company, merlinux GmbH. diff --git a/doc/en/announce/release-2.9.0.rst b/doc/en/announce/release-2.9.0.rst index c079fdf6b..05d9a394f 100644 --- a/doc/en/announce/release-2.9.0.rst +++ b/doc/en/announce/release-2.9.0.rst @@ -75,7 +75,7 @@ The py.test Development Team **Changes** -* **Important**: `py.code `_ has been +* **Important**: `py.code `_ has been merged into the ``pytest`` repository as ``pytest._code``. This decision was made because ``py.code`` had very few uses outside ``pytest`` and the fact that it was in a different repository made it difficult to fix bugs on @@ -88,7 +88,7 @@ The py.test Development Team **experimental**, so you definitely should not import it explicitly! Please note that the original ``py.code`` is still available in - `pylib `_. + `pylib `_. * ``pytest_enter_pdb`` now optionally receives the pytest config object. Thanks `@nicoddemus`_ for the PR. diff --git a/doc/en/announce/release-2.9.2.rst b/doc/en/announce/release-2.9.2.rst index 8f274cdf3..b007a6d99 100644 --- a/doc/en/announce/release-2.9.2.rst +++ b/doc/en/announce/release-2.9.2.rst @@ -66,8 +66,8 @@ The py.test Development Team .. _#510: https://github.com/pytest-dev/pytest/issues/510 .. _#1506: https://github.com/pytest-dev/pytest/pull/1506 -.. _#1496: https://github.com/pytest-dev/pytest/issue/1496 -.. _#1524: https://github.com/pytest-dev/pytest/issue/1524 +.. _#1496: https://github.com/pytest-dev/pytest/issues/1496 +.. _#1524: https://github.com/pytest-dev/pytest/pull/1524 .. _@astraw38: https://github.com/astraw38 .. _@hackebrot: https://github.com/hackebrot diff --git a/doc/en/conf.py b/doc/en/conf.py index 42dc18fd8..1a6ef7ca8 100644 --- a/doc/en/conf.py +++ b/doc/en/conf.py @@ -15,7 +15,6 @@ # # The full version, including alpha/beta/rc tags. # The short X.Y version. -import datetime import os import sys @@ -63,8 +62,7 @@ master_doc = "contents" # General information about the project. project = "pytest" -year = datetime.datetime.utcnow().year -copyright = "2015–2019 , holger krekel and pytest-dev team" +copyright = "2015–2019, holger krekel and pytest-dev team" # The language for content autogenerated by Sphinx. Refer to documentation @@ -167,18 +165,18 @@ html_favicon = "img/pytest1favi.ico" html_sidebars = { "index": [ + "slim_searchbox.html", "sidebarintro.html", "globaltoc.html", "links.html", "sourcelink.html", - "searchbox.html", ], "**": [ + "slim_searchbox.html", "globaltoc.html", "relations.html", "links.html", "sourcelink.html", - "searchbox.html", ], } diff --git a/doc/en/example/fixtures/test_fixtures_order.py b/doc/en/example/fixtures/test_fixtures_order.py new file mode 100644 index 000000000..97b3e8005 --- /dev/null +++ b/doc/en/example/fixtures/test_fixtures_order.py @@ -0,0 +1,38 @@ +import pytest + +# fixtures documentation order example +order = [] + + +@pytest.fixture(scope="session") +def s1(): + order.append("s1") + + +@pytest.fixture(scope="module") +def m1(): + order.append("m1") + + +@pytest.fixture +def f1(f3): + order.append("f1") + + +@pytest.fixture +def f3(): + order.append("f3") + + +@pytest.fixture(autouse=True) +def a1(): + order.append("a1") + + +@pytest.fixture +def f2(): + order.append("f2") + + +def test_order(f1, m1, f2, s1): + assert order == ["s1", "m1", "a1", "f3", "f1", "f2"] diff --git a/doc/en/example/markers.rst b/doc/en/example/markers.rst index 004e77add..218ef2707 100644 --- a/doc/en/example/markers.rst +++ b/doc/en/example/markers.rst @@ -336,7 +336,7 @@ apply a marker to an individual test instance: @pytest.mark.foo @pytest.mark.parametrize( - ("n", "expected"), [(1, 2), pytest.param((1, 3), marks=pytest.mark.bar), (2, 3)] + ("n", "expected"), [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)] ) def test_increment(n, expected): assert n + 1 == expected diff --git a/doc/en/example/parametrize.rst b/doc/en/example/parametrize.rst index 0a5ba8358..3dff2c559 100644 --- a/doc/en/example/parametrize.rst +++ b/doc/en/example/parametrize.rst @@ -552,13 +552,13 @@ Then run ``pytest`` with verbose mode and with only the ``basic`` marker: platform linux -- Python 3.x.y, pytest-5.x.y, py-1.x.y, pluggy-0.x.y -- $PYTHON_PREFIX/bin/python cachedir: $PYTHON_PREFIX/.pytest_cache rootdir: $REGENDOC_TMPDIR - collecting ... collected 17 items / 14 deselected / 3 selected + collecting ... collected 18 items / 15 deselected / 3 selected test_pytest_param_example.py::test_eval[1+7-8] PASSED [ 33%] test_pytest_param_example.py::test_eval[basic_2+4] PASSED [ 66%] test_pytest_param_example.py::test_eval[basic_6*9] XFAIL [100%] - ============ 2 passed, 14 deselected, 1 xfailed in 0.12 seconds ============ + ============ 2 passed, 15 deselected, 1 xfailed in 0.12 seconds ============ As the result: diff --git a/doc/en/fixture.rst b/doc/en/fixture.rst index 852069731..40493e66f 100644 --- a/doc/en/fixture.rst +++ b/doc/en/fixture.rst @@ -289,51 +289,29 @@ are finalized when the last test of a *package* finishes. Use this new feature sparingly and please make sure to report any issues you find. -Higher-scoped fixtures are instantiated first ---------------------------------------------- +Order: Higher-scoped fixtures are instantiated first +---------------------------------------------------- Within a function request for features, fixture of higher-scopes (such as ``session``) are instantiated first than lower-scoped fixtures (such as ``function`` or ``class``). The relative order of fixtures of same scope follows -the declared order in the test function and honours dependencies between fixtures. +the declared order in the test function and honours dependencies between fixtures. Autouse fixtures will be +instantiated before explicitly used fixtures. Consider the code below: -.. code-block:: python - - @pytest.fixture(scope="session") - def s1(): - pass - - - @pytest.fixture(scope="module") - def m1(): - pass - - - @pytest.fixture - def f1(tmpdir): - pass - - - @pytest.fixture - def f2(): - pass - - - def test_foo(f1, m1, f2, s1): - ... - +.. literalinclude:: example/fixtures/test_fixtures_order.py The fixtures requested by ``test_foo`` will be instantiated in the following order: 1. ``s1``: is the highest-scoped fixture (``session``). 2. ``m1``: is the second highest-scoped fixture (``module``). -3. ``tmpdir``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point - because it is a dependency of ``f1``. -4. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list. -5. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list. +3. ``a1``: is a ``function``-scoped ``autouse`` fixture: it will be instantiated before other fixtures + within the same scope. +4. ``f3``: is a ``function``-scoped fixture, required by ``f1``: it needs to be instantiated at this point +5. ``f1``: is the first ``function``-scoped fixture in ``test_foo`` parameter list. +6. ``f2``: is the last ``function``-scoped fixture in ``test_foo`` parameter list. .. _`finalization`: @@ -400,6 +378,34 @@ The ``smtp_connection`` connection will be closed after the test finished execution because the ``smtp_connection`` object automatically closes when the ``with`` statement ends. +Using the contextlib.ExitStack context manager finalizers will always be called +regardless if the fixture *setup* code raises an exception. This is handy to properly +close all resources created by a fixture even if one of them fails to be created/acquired: + +.. code-block:: python + + # content of test_yield3.py + + import contextlib + + import pytest + + + @contextlib.contextmanager + def connect(port): + ... # create connection + yield + ... # close connection + + + @pytest.fixture + def equipments(): + with contextlib.ExitStack() as stack: + yield [stack.enter_context(connect(port)) for port in ("C1", "C3", "C28")] + +In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still +be properly closed. + Note that if an exception happens during the *setup* code (before the ``yield`` keyword), the *teardown* code (after the ``yield``) will not be called. @@ -428,27 +434,39 @@ Here's the ``smtp_connection`` fixture changed to use ``addfinalizer`` for clean return smtp_connection # provide the fixture value +Here's the ``equipments`` fixture changed to use ``addfinalizer`` for cleanup: + +.. code-block:: python + + # content of test_yield3.py + + import contextlib + import functools + + import pytest + + + @contextlib.contextmanager + def connect(port): + ... # create connection + yield + ... # close connection + + + @pytest.fixture + def equipments(request): + r = [] + for port in ("C1", "C3", "C28"): + cm = connect(port) + equip = cm.__enter__() + request.addfinalizer(functools.partial(cm.__exit__, None, None, None)) + r.append(equip) + return r + + Both ``yield`` and ``addfinalizer`` methods work similarly by calling their code after the test -ends, but ``addfinalizer`` has two key differences over ``yield``: - -1. It is possible to register multiple finalizer functions. - -2. Finalizers will always be called regardless if the fixture *setup* code raises an exception. - This is handy to properly close all resources created by a fixture even if one of them - fails to be created/acquired:: - - @pytest.fixture - def equipments(request): - r = [] - for port in ('C1', 'C3', 'C28'): - equip = connect(port) - request.addfinalizer(equip.disconnect) - r.append(equip) - return r - - In the example above, if ``"C28"`` fails with an exception, ``"C1"`` and ``"C3"`` will still - be properly closed. Of course, if an exception happens before the finalize function is - registered then it will not be executed. +ends. Of course, if an exception happens before the finalize function is registered then it +will not be executed. .. _`request-context`: @@ -522,7 +540,7 @@ of a fixture is needed multiple times in a single test. Instead of returning data directly, the fixture instead returns a function which generates the data. This function can then be called multiple times in the test. -Factories can have have parameters as needed:: +Factories can have parameters as needed:: @pytest.fixture def make_customer_record(): diff --git a/doc/en/flaky.rst b/doc/en/flaky.rst index 8e340316e..0f0eecab0 100644 --- a/doc/en/flaky.rst +++ b/doc/en/flaky.rst @@ -122,4 +122,4 @@ Resources * Google: * `Flaky Tests at Google and How We Mitigate Them `_ by John Micco, 2016 - * `Where do Google's flaky tests come from? `_ by Jeff Listfield, 2017 + * `Where do Google's flaky tests come from? `_ by Jeff Listfield, 2017 diff --git a/doc/en/getting-started.rst b/doc/en/getting-started.rst index 750de086e..c313f3849 100644 --- a/doc/en/getting-started.rst +++ b/doc/en/getting-started.rst @@ -142,7 +142,7 @@ The first test passed and the second failed. You can easily see the intermediate Request a unique temporary directory for functional tests -------------------------------------------------------------- -``pytest`` provides `Builtin fixtures/function arguments `_ to request arbitrary resources, like a unique temporary directory:: +``pytest`` provides `Builtin fixtures/function arguments `_ to request arbitrary resources, like a unique temporary directory:: # content of test_tmpdir.py def test_needsfiles(tmpdir): diff --git a/doc/en/index.rst b/doc/en/index.rst index 2d6ea620f..6c7c84865 100644 --- a/doc/en/index.rst +++ b/doc/en/index.rst @@ -61,7 +61,7 @@ Features - Can run :ref:`unittest ` (including trial) and :ref:`nose ` test suites out of the box; -- Python Python 3.5+ and PyPy 3; +- Python 3.5+ and PyPy 3; - Rich plugin architecture, with over 315+ `external plugins `_ and thriving community; diff --git a/doc/en/license.rst b/doc/en/license.rst index 5ee55cf96..d94b34a99 100644 --- a/doc/en/license.rst +++ b/doc/en/license.rst @@ -9,7 +9,7 @@ Distributed under the terms of the `MIT`_ license, pytest is free and open sourc The MIT License (MIT) - Copyright (c) 2004-2017 Holger Krekel and others + Copyright (c) 2004-2019 Holger Krekel and others Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in diff --git a/doc/en/links.inc b/doc/en/links.inc index 1b7cbd05b..7c5e8d88c 100644 --- a/doc/en/links.inc +++ b/doc/en/links.inc @@ -14,7 +14,7 @@ .. _`distribute docs`: .. _`distribute`: https://pypi.org/project/distribute/ .. _`pip`: https://pypi.org/project/pip/ -.. _`venv`: https://docs.python.org/3/library/venv.html/ +.. _`venv`: https://docs.python.org/3/library/venv.html .. _`virtualenv`: https://pypi.org/project/virtualenv/ .. _hudson: http://hudson-ci.org/ .. _jenkins: http://jenkins-ci.org/ diff --git a/doc/en/mark.rst b/doc/en/mark.rst index de6ab7822..3899dab88 100644 --- a/doc/en/mark.rst +++ b/doc/en/mark.rst @@ -40,7 +40,7 @@ You can register custom marks in your ``pytest.ini`` file like this: Note that everything after the ``:`` is an optional description. -Alternatively, you can register new markers programatically in a +Alternatively, you can register new markers programmatically in a :ref:`pytest_configure ` hook: .. code-block:: python diff --git a/doc/en/monkeypatch.rst b/doc/en/monkeypatch.rst index 8e4622982..fd277d234 100644 --- a/doc/en/monkeypatch.rst +++ b/doc/en/monkeypatch.rst @@ -46,10 +46,13 @@ environment variable is missing, or to set multiple values to a known variable. :py:meth:`monkeypatch.setenv` and :py:meth:`monkeypatch.delenv` can be used for these patches. -4. Use :py:meth:`monkeypatch.syspath_prepend` to modify the system ``$PATH`` safely, and +4. Use ``monkeypatch.setenv("PATH", value, prepend=os.pathsep)`` to modify ``$PATH``, and :py:meth:`monkeypatch.chdir` to change the context of the current working directory during a test. +5. Use py:meth:`monkeypatch.syspath_prepend` to modify ``sys.path`` which will also +call :py:meth:`pkg_resources.fixup_namespace_packages` and :py:meth:`importlib.invalidate_caches`. + See the `monkeypatch blog post`_ for some introduction material and a discussion of its motivation. diff --git a/doc/en/projects.rst b/doc/en/projects.rst index 606e9d47c..226358596 100644 --- a/doc/en/projects.rst +++ b/doc/en/projects.rst @@ -28,7 +28,6 @@ Here are some examples of projects using ``pytest`` (please send notes via :ref: * `sentry `_, realtime app-maintenance and exception tracking * `Astropy `_ and `affiliated packages `_ * `tox `_, virtualenv/Hudson integration tool -* `PIDA `_ framework for integrated development * `PyPM `_ ActiveState's package manager * `Fom `_ a fluid object mapper for FluidDB * `applib `_ cross-platform utilities @@ -37,8 +36,7 @@ Here are some examples of projects using ``pytest`` (please send notes via :ref: * `mwlib `_ mediawiki parser and utility library * `The Translate Toolkit `_ for localization and conversion * `execnet `_ rapid multi-Python deployment -* `pylib `_ cross-platform path, IO, dynamic code library -* `Pacha `_ configuration management in five minutes +* `pylib `_ cross-platform path, IO, dynamic code library * `bbfreeze `_ create standalone executables from Python scripts * `pdb++ `_ a fancier version of PDB * `py-s3fuse `_ Amazon S3 FUSE based filesystem @@ -77,7 +75,7 @@ Some organisations using pytest * `Tandberg `_ * `Shootq `_ * `Stups department of Heinrich Heine University Duesseldorf `_ -* `cellzome `_ +* cellzome * `Open End, Gothenborg `_ * `Laboratory of Bioinformatics, Warsaw `_ * `merlinux, Germany `_ diff --git a/doc/en/reference.rst b/doc/en/reference.rst index 5abb01f50..afbef6b1e 100644 --- a/doc/en/reference.rst +++ b/doc/en/reference.rst @@ -1,5 +1,5 @@ -Reference -========= +API Reference +============= This page contains the full reference to pytest's API. diff --git a/doc/en/talks.rst b/doc/en/talks.rst index 20b6e5b09..f66192817 100644 --- a/doc/en/talks.rst +++ b/doc/en/talks.rst @@ -4,9 +4,8 @@ Talks and Tutorials .. sidebar:: Next Open Trainings - - `Training at Europython 2019 `_, 8th July 2019, Basel, Switzerland. - - `Training at Workshoptage 2019 `_ (German), 10th September 2019, Rapperswil, Switzerland. + - `3 day hands-on workshop covering pytest, tox and devpi: "Professional Testing with Python" `_ (English), October 21 - 23, 2019, Leipzig, Germany. .. _`funcargs`: funcargs.html diff --git a/doc/en/warnings.rst b/doc/en/warnings.rst index 5934e035d..3c1f37f41 100644 --- a/doc/en/warnings.rst +++ b/doc/en/warnings.rst @@ -127,7 +127,7 @@ decorator or to all tests in a module by setting the ``pytestmark`` variable: *Credits go to Florian Schulze for the reference implementation in the* `pytest-warnings`_ *plugin.* -.. _`-W option`: https://docs.python.org/3/using/cmdline.html?highlight=#cmdoption-W +.. _`-W option`: https://docs.python.org/3/using/cmdline.html#cmdoption-w .. _warnings.simplefilter: https://docs.python.org/3/library/warnings.html#warnings.simplefilter .. _`pytest-warnings`: https://github.com/fschulze/pytest-warnings diff --git a/doc/en/writing_plugins.rst b/doc/en/writing_plugins.rst index 4bbc1afce..67a8fcecf 100644 --- a/doc/en/writing_plugins.rst +++ b/doc/en/writing_plugins.rst @@ -164,7 +164,7 @@ If a package is installed this way, ``pytest`` will load .. note:: Make sure to include ``Framework :: Pytest`` in your list of - `PyPI classifiers `_ + `PyPI classifiers `_ to make it easy for users to find your plugin. diff --git a/src/_pytest/cacheprovider.py b/src/_pytest/cacheprovider.py index 17463959f..496931e0f 100755 --- a/src/_pytest/cacheprovider.py +++ b/src/_pytest/cacheprovider.py @@ -14,7 +14,7 @@ import py import pytest from .pathlib import Path from .pathlib import resolve_from_str -from .pathlib import rmtree +from .pathlib import rm_rf README_CONTENT = """\ # pytest cache directory # @@ -44,7 +44,7 @@ class Cache: def for_config(cls, config): cachedir = cls.cache_dir_from_config(config) if config.getoption("cacheclear") and cachedir.exists(): - rmtree(cachedir, force=True) + rm_rf(cachedir) cachedir.mkdir() return cls(cachedir, config) diff --git a/src/_pytest/compat.py b/src/_pytest/compat.py index 0525d6985..df049991b 100644 --- a/src/_pytest/compat.py +++ b/src/_pytest/compat.py @@ -70,13 +70,18 @@ def num_mock_patch_args(function): patchings = getattr(function, "patchings", None) if not patchings: return 0 - mock_modules = [sys.modules.get("mock"), sys.modules.get("unittest.mock")] - if any(mock_modules): - sentinels = [m.DEFAULT for m in mock_modules if m is not None] - return len( - [p for p in patchings if not p.attribute_name and p.new in sentinels] - ) - return len(patchings) + + mock_sentinel = getattr(sys.modules.get("mock"), "DEFAULT", object()) + ut_mock_sentinel = getattr(sys.modules.get("unittest.mock"), "DEFAULT", object()) + + return len( + [ + p + for p in patchings + if not p.attribute_name + and (p.new is mock_sentinel or p.new is ut_mock_sentinel) + ] + ) def getfuncargnames(function, is_method=False, cls=None): diff --git a/src/_pytest/config/argparsing.py b/src/_pytest/config/argparsing.py index 43cf62ab1..8994ff7d9 100644 --- a/src/_pytest/config/argparsing.py +++ b/src/_pytest/config/argparsing.py @@ -399,7 +399,7 @@ class DropShorterLongHelpFormatter(argparse.HelpFormatter): """shorten help for long options that differ only in extra hyphens - collapse **long** options that are the same except for extra hyphens - - special action attribute map_long_option allows surpressing additional + - special action attribute map_long_option allows suppressing additional long options - shortcut if there are only two options and one of them is a short one - cache result on action object as this is called at least 2 times diff --git a/src/_pytest/config/findpaths.py b/src/_pytest/config/findpaths.py index abb248b1d..ec991316a 100644 --- a/src/_pytest/config/findpaths.py +++ b/src/_pytest/config/findpaths.py @@ -30,7 +30,11 @@ def getcfg(args, config=None): for inibasename in inibasenames: p = base.join(inibasename) if exists(p): - iniconfig = py.iniconfig.IniConfig(p) + try: + iniconfig = py.iniconfig.IniConfig(p) + except py.iniconfig.ParseError as exc: + raise UsageError(str(exc)) + if ( inibasename == "setup.cfg" and "tool:pytest" in iniconfig.sections diff --git a/src/_pytest/mark/structures.py b/src/_pytest/mark/structures.py index da4900dc9..332c86bde 100644 --- a/src/_pytest/mark/structures.py +++ b/src/_pytest/mark/structures.py @@ -171,9 +171,7 @@ class Mark: @attr.s class MarkDecorator: """ A decorator for test functions and test classes. When applied - it will create :class:`MarkInfo` objects which may be - :ref:`retrieved by hooks as item keywords `. - MarkDecorator instances are often created like this:: + it will create :class:`Mark` objects which are often created like this:: mark1 = pytest.mark.NAME # simple MarkDecorator mark2 = pytest.mark.NAME(name1=value) # parametrized MarkDecorator @@ -185,17 +183,18 @@ class MarkDecorator: pass When a MarkDecorator instance is called it does the following: - 1. If called with a single class as its only positional argument and no - additional keyword arguments, it attaches itself to the class so it - gets applied automatically to all test cases found in that class. - 2. If called with a single function as its only positional argument and - no additional keyword arguments, it attaches a MarkInfo object to the - function, containing all the arguments already stored internally in - the MarkDecorator. - 3. When called in any other case, it performs a 'fake construction' call, - i.e. it returns a new MarkDecorator instance with the original - MarkDecorator's content updated with the arguments passed to this - call. + + 1. If called with a single class as its only positional argument and no + additional keyword arguments, it attaches itself to the class so it + gets applied automatically to all test cases found in that class. + 2. If called with a single function as its only positional argument and + no additional keyword arguments, it attaches a MarkInfo object to the + function, containing all the arguments already stored internally in + the MarkDecorator. + 3. When called in any other case, it performs a 'fake construction' call, + i.e. it returns a new MarkDecorator instance with the original + MarkDecorator's content updated with the arguments passed to this + call. Note: The rules above prevent MarkDecorator objects from storing only a single function or class reference as their positional argument with no diff --git a/src/_pytest/nodes.py b/src/_pytest/nodes.py index 7e1c40bcb..9b78dca38 100644 --- a/src/_pytest/nodes.py +++ b/src/_pytest/nodes.py @@ -1,5 +1,6 @@ import os import warnings +from functools import lru_cache import py @@ -13,6 +14,7 @@ SEP = "/" tracebackcutdir = py.path.local(_pytest.__file__).dirpath() +@lru_cache(maxsize=None) def _splitnode(nodeid): """Split a nodeid into constituent 'parts'. @@ -30,11 +32,12 @@ def _splitnode(nodeid): """ if nodeid == "": # If there is no root node at all, return an empty list so the caller's logic can remain sane - return [] + return () parts = nodeid.split(SEP) # Replace single last element 'test_foo.py::Bar' with multiple elements 'test_foo.py', 'Bar' parts[-1:] = parts[-1].split("::") - return parts + # Convert parts into a tuple to avoid possible errors with caching of a mutable type + return tuple(parts) def ischildnode(baseid, nodeid): diff --git a/src/_pytest/outcomes.py b/src/_pytest/outcomes.py index a5a4e655b..5e331a323 100644 --- a/src/_pytest/outcomes.py +++ b/src/_pytest/outcomes.py @@ -18,6 +18,12 @@ class OutcomeException(BaseException): """ def __init__(self, msg: Optional[str] = None, pytrace: bool = True) -> None: + if msg is not None and not isinstance(msg, str): + error_msg = ( + "{} expected string as 'msg' parameter, got '{}' instead.\n" + "Perhaps you meant to use a mark?" + ) + raise TypeError(error_msg.format(type(self).__name__, type(msg).__name__)) BaseException.__init__(self, msg) self.msg = msg self.pytrace = pytrace diff --git a/src/_pytest/pathlib.py b/src/_pytest/pathlib.py index ecc38eb0f..1c0c45b14 100644 --- a/src/_pytest/pathlib.py +++ b/src/_pytest/pathlib.py @@ -7,12 +7,15 @@ import os import shutil import sys import uuid +import warnings +from functools import partial from os.path import expanduser from os.path import expandvars from os.path import isabs from os.path import sep from posixpath import sep as posix_sep +from _pytest.warning_types import PytestWarning if sys.version_info[:2] >= (3, 6): from pathlib import Path, PurePath @@ -32,17 +35,53 @@ def ensure_reset_dir(path): ensures the given path is an empty directory """ if path.exists(): - rmtree(path, force=True) + rm_rf(path) path.mkdir() -def rmtree(path, force=False): - if force: - # NOTE: ignore_errors might leave dead folders around. - # Python needs a rm -rf as a followup. - shutil.rmtree(str(path), ignore_errors=True) - else: - shutil.rmtree(str(path)) +def on_rm_rf_error(func, path: str, exc, *, start_path): + """Handles known read-only errors during rmtree.""" + excvalue = exc[1] + + if not isinstance(excvalue, PermissionError): + warnings.warn( + PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue)) + ) + return + + if func not in (os.rmdir, os.remove, os.unlink): + warnings.warn( + PytestWarning("(rm_rf) error removing {}: {}".format(path, excvalue)) + ) + return + + # Chmod + retry. + import stat + + def chmod_rw(p: str): + mode = os.stat(p).st_mode + os.chmod(p, mode | stat.S_IRUSR | stat.S_IWUSR) + + # For files, we need to recursively go upwards in the directories to + # ensure they all are also writable. + p = Path(path) + if p.is_file(): + for parent in p.parents: + chmod_rw(str(parent)) + # stop when we reach the original path passed to rm_rf + if parent == start_path: + break + chmod_rw(str(path)) + + func(path) + + +def rm_rf(path: Path): + """Remove the path contents recursively, even if some elements + are read-only. + """ + onerror = partial(on_rm_rf_error, start_path=path) + shutil.rmtree(str(path), onerror=onerror) def find_prefixed(root, prefix): @@ -82,9 +121,9 @@ def _force_symlink(root, target, link_to): """helper to create the current symlink it's full of race conditions that are reasonably ok to ignore - for the context of best effort linking to the latest testrun + for the context of best effort linking to the latest test run - the presumption being thatin case of much parallelism + the presumption being that in case of much parallelism the inaccuracy is going to be acceptable """ current_symlink = root.joinpath(target) @@ -168,7 +207,7 @@ def maybe_delete_a_numbered_dir(path): garbage = parent.joinpath("garbage-{}".format(uuid.uuid4())) path.rename(garbage) - rmtree(garbage, force=True) + rm_rf(garbage) except (OSError, EnvironmentError): # known races: # * other process did a cleanup at the same time diff --git a/src/_pytest/python_api.py b/src/_pytest/python_api.py index 08426d69c..fbc3d914e 100644 --- a/src/_pytest/python_api.py +++ b/src/_pytest/python_api.py @@ -579,7 +579,7 @@ def raises( string that may contain `special characters`__, the pattern can first be escaped with ``re.escape``. - __ https://docs.python.org/3/library/re.html#regular-expression-syntax + __ https://docs.python.org/3/library/re.html#regular-expression-syntax .. currentmodule:: _pytest._code diff --git a/src/_pytest/stepwise.py b/src/_pytest/stepwise.py index a18a58573..6fa21cd1c 100644 --- a/src/_pytest/stepwise.py +++ b/src/_pytest/stepwise.py @@ -71,7 +71,6 @@ class StepwisePlugin: config.hook.pytest_deselected(items=already_passed) def pytest_runtest_logreport(self, report): - # Skip this hook if plugin is not active or the test is xfailed. if not self.active: return diff --git a/src/_pytest/terminal.py b/src/_pytest/terminal.py index bcd6e1f7c..05d5427c3 100644 --- a/src/_pytest/terminal.py +++ b/src/_pytest/terminal.py @@ -692,7 +692,7 @@ class TerminalReporter: else: excrepr.reprcrash.toterminal(self._tw) self._tw.line( - "(to show a full traceback on KeyboardInterrupt use --fulltrace)", + "(to show a full traceback on KeyboardInterrupt use --full-trace)", yellow=True, ) diff --git a/src/_pytest/unittest.py b/src/_pytest/unittest.py index c9bdf79c1..11dc77cc4 100644 --- a/src/_pytest/unittest.py +++ b/src/_pytest/unittest.py @@ -6,6 +6,7 @@ import _pytest._code import pytest from _pytest.compat import getimfunc from _pytest.config import hookimpl +from _pytest.outcomes import exit from _pytest.outcomes import fail from _pytest.outcomes import skip from _pytest.outcomes import xfail @@ -154,6 +155,11 @@ class TestCaseFunction(Function): self.__dict__.setdefault("_excinfo", []).append(excinfo) def addError(self, testcase, rawexcinfo): + try: + if isinstance(rawexcinfo[1], exit.Exception): + exit(rawexcinfo[1].msg) + except TypeError: + pass self._addexcinfo(rawexcinfo) def addFailure(self, testcase, rawexcinfo): diff --git a/testing/code/test_excinfo.py b/testing/code/test_excinfo.py index 76f974957..7742b4da9 100644 --- a/testing/code/test_excinfo.py +++ b/testing/code/test_excinfo.py @@ -378,7 +378,7 @@ def test_excinfo_no_python_sourcecode(tmpdir): excinfo = pytest.raises(ValueError, template.render, h=h) for item in excinfo.traceback: print(item) # XXX: for some reason jinja.Template.render is printed in full - item.source # shouldnt fail + item.source # shouldn't fail if item.path.basename == "test.txt": assert str(item.source) == "{{ h()}}:" diff --git a/testing/deprecated_test.py b/testing/deprecated_test.py index 97b88e939..b8a22428f 100644 --- a/testing/deprecated_test.py +++ b/testing/deprecated_test.py @@ -36,7 +36,7 @@ def test_terminal_reporter_writer_attr(pytestconfig): assert terminal_reporter.writer is terminal_reporter._tw -@pytest.mark.parametrize("plugin", deprecated.DEPRECATED_EXTERNAL_PLUGINS) +@pytest.mark.parametrize("plugin", sorted(deprecated.DEPRECATED_EXTERNAL_PLUGINS)) @pytest.mark.filterwarnings("default") def test_external_plugins_integrated(testdir, plugin): testdir.syspathinsert() diff --git a/testing/python/integration.py b/testing/python/integration.py index 0b87fea33..73419eef4 100644 --- a/testing/python/integration.py +++ b/testing/python/integration.py @@ -104,21 +104,15 @@ class TestMockDecoration: values = getfuncargnames(f) assert values == ("x",) - @pytest.mark.xfail( - strict=False, reason="getfuncargnames breaks if mock is imported" - ) - def test_wrapped_getfuncargnames_patching(self): + def test_getfuncargnames_patching(self): from _pytest.compat import getfuncargnames + from unittest.mock import patch - def wrap(f): - def func(): + class T: + def original(self, x, y, z): pass - func.__wrapped__ = f - func.patchings = ["qwe"] - return func - - @wrap + @patch.object(T, "original") def f(x, y, z): pass @@ -126,7 +120,6 @@ class TestMockDecoration: assert values == ("y", "z") def test_unittest_mock(self, testdir): - pytest.importorskip("unittest.mock") testdir.makepyfile( """ import unittest.mock @@ -142,7 +135,6 @@ class TestMockDecoration: reprec.assertoutcome(passed=1) def test_unittest_mock_and_fixture(self, testdir): - pytest.importorskip("unittest.mock") testdir.makepyfile( """ import os.path @@ -164,7 +156,6 @@ class TestMockDecoration: reprec.assertoutcome(passed=1) def test_unittest_mock_and_pypi_mock(self, testdir): - pytest.importorskip("unittest.mock") pytest.importorskip("mock", "1.0.1") testdir.makepyfile( """ @@ -187,6 +178,34 @@ class TestMockDecoration: reprec = testdir.inline_run() reprec.assertoutcome(passed=2) + def test_mock_sentinel_check_against_numpy_like(self, testdir): + """Ensure our function that detects mock arguments compares against sentinels using + identity to circumvent objects which can't be compared with equality against others + in a truth context, like with numpy arrays (#5606). + """ + testdir.makepyfile( + dummy=""" + class NumpyLike: + def __init__(self, value): + self.value = value + def __eq__(self, other): + raise ValueError("like numpy, cannot compare against others for truth") + FOO = NumpyLike(10) + """ + ) + testdir.makepyfile( + """ + from unittest.mock import patch + import dummy + class Test(object): + @patch("dummy.FOO", new=dummy.NumpyLike(50)) + def test_hello(self): + assert dummy.FOO.value == 50 + """ + ) + reprec = testdir.inline_run() + reprec.assertoutcome(passed=1) + def test_mock(self, testdir): pytest.importorskip("mock", "1.0.1") testdir.makepyfile( diff --git a/testing/test_config.py b/testing/test_config.py index 143cb90d1..fc3659d2a 100644 --- a/testing/test_config.py +++ b/testing/test_config.py @@ -123,6 +123,12 @@ class TestParseIni: config = testdir.parseconfigure(sub) assert config.getini("minversion") == "2.0" + def test_ini_parse_error(self, testdir): + testdir.tmpdir.join("pytest.ini").write("addopts = -x") + result = testdir.runpytest() + assert result.ret != 0 + result.stderr.fnmatch_lines(["ERROR: *pytest.ini:1: no section header defined"]) + @pytest.mark.xfail(reason="probably not needed") def test_confcutdir(self, testdir): sub = testdir.mkdir("sub") diff --git a/testing/test_junitxml.py b/testing/test_junitxml.py index 15643b081..3196f0ebd 100644 --- a/testing/test_junitxml.py +++ b/testing/test_junitxml.py @@ -592,7 +592,7 @@ class TestPython: assert "hx" in fnode.toxml() def test_assertion_binchars(self, testdir): - """this test did fail when the escaping wasnt strict""" + """this test did fail when the escaping wasn't strict""" testdir.makepyfile( """ @@ -715,7 +715,7 @@ def test_dont_configure_on_slaves(tmpdir): return "pytest" junitprefix = None - # XXX: shouldnt need tmpdir ? + # XXX: shouldn't need tmpdir ? xmlpath = str(tmpdir.join("junix.xml")) register = gotten.append diff --git a/testing/test_mark.py b/testing/test_mark.py index 8747d1c6b..c8d5851ac 100644 --- a/testing/test_mark.py +++ b/testing/test_mark.py @@ -430,7 +430,7 @@ class TestFunctional: def test_b(self): assert True class TestC(object): - # this one didnt get marked + # this one didn't get marked def test_d(self): assert True """ diff --git a/testing/test_nose.py b/testing/test_nose.py index f60c3af53..16d8d1fc0 100644 --- a/testing/test_nose.py +++ b/testing/test_nose.py @@ -253,7 +253,7 @@ def test_apiwrapper_problem_issue260(testdir): def test_setup_teardown_linking_issue265(testdir): - # we accidentally didnt integrate nose setupstate with normal setupstate + # we accidentally didn't integrate nose setupstate with normal setupstate # this test ensures that won't happen again testdir.makepyfile( ''' diff --git a/testing/test_pytester.py b/testing/test_pytester.py index f115ad3d0..cf92741af 100644 --- a/testing/test_pytester.py +++ b/testing/test_pytester.py @@ -230,8 +230,8 @@ class TestInlineRunModulesCleanup: ): spy_factory = self.spy_factory() monkeypatch.setattr(pytester, "SysModulesSnapshot", spy_factory) - original = dict(sys.modules) testdir.syspathinsert() + original = dict(sys.modules) testdir.makepyfile(import1="# you son of a silly person") testdir.makepyfile(import2="# my hovercraft is full of eels") test_mod = testdir.makepyfile( diff --git a/testing/test_runner.py b/testing/test_runner.py index 15180c071..82e413518 100644 --- a/testing/test_runner.py +++ b/testing/test_runner.py @@ -11,6 +11,7 @@ from _pytest import main from _pytest import outcomes from _pytest import reports from _pytest import runner +from _pytest.outcomes import OutcomeException class TestSetupState: @@ -990,3 +991,18 @@ class TestReportContents: rep = reports[1] assert rep.capstdout == "" assert rep.capstderr == "" + + +def test_outcome_exception_bad_msg(): + """Check that OutcomeExceptions validate their input to prevent confusing errors (#5578)""" + + def func(): + pass + + expected = ( + "OutcomeException expected string as 'msg' parameter, got 'function' instead.\n" + "Perhaps you meant to use a mark?" + ) + with pytest.raises(TypeError) as excinfo: + OutcomeException(func) + assert str(excinfo.value) == expected diff --git a/testing/test_skipping.py b/testing/test_skipping.py index 6bb5f7aff..8bba479f1 100644 --- a/testing/test_skipping.py +++ b/testing/test_skipping.py @@ -1066,7 +1066,8 @@ def test_module_level_skip_error(testdir): testdir.makepyfile( """ import pytest - @pytest.skip + pytest.skip("skip_module_level") + def test_func(): assert True """ diff --git a/testing/test_stepwise.py b/testing/test_stepwise.py index 591d67b6c..f61425b6b 100644 --- a/testing/test_stepwise.py +++ b/testing/test_stepwise.py @@ -207,7 +207,8 @@ def test_xfail_handling(testdir): # because we are writing to the same file, mtime might not be affected enough to # invalidate the cache, making this next run flaky - testdir.tmpdir.join("__pycache__").remove() + if testdir.tmpdir.join("__pycache__").exists(): + testdir.tmpdir.join("__pycache__").remove() testdir.makepyfile(contents.format(assert_value="0", strict="True")) result = testdir.runpytest("--sw", "-v") result.stdout.fnmatch_lines( diff --git a/testing/test_terminal.py b/testing/test_terminal.py index bf029fbc5..381a5b2e1 100644 --- a/testing/test_terminal.py +++ b/testing/test_terminal.py @@ -233,7 +233,7 @@ class TestTerminal: ) else: result.stdout.fnmatch_lines( - ["(to show a full traceback on KeyboardInterrupt use --fulltrace)"] + ["(to show a full traceback on KeyboardInterrupt use --full-trace)"] ) result.stdout.fnmatch_lines(["*KeyboardInterrupt*"]) diff --git a/testing/test_tmpdir.py b/testing/test_tmpdir.py index 11556594b..ebde9044c 100644 --- a/testing/test_tmpdir.py +++ b/testing/test_tmpdir.py @@ -1,3 +1,5 @@ +import os +import stat import sys import attr @@ -303,22 +305,6 @@ class TestNumberedDir: p, consider_lock_dead_if_created_before=p.stat().st_mtime + 1 ) - def test_rmtree(self, tmp_path): - from _pytest.pathlib import rmtree - - adir = tmp_path / "adir" - adir.mkdir() - rmtree(adir) - - assert not adir.exists() - - adir.mkdir() - afile = adir / "afile" - afile.write_bytes(b"aa") - - rmtree(adir, force=True) - assert not adir.exists() - def test_cleanup_ignores_symlink(self, tmp_path): the_symlink = tmp_path / (self.PREFIX + "current") attempt_symlink_to(the_symlink, tmp_path / (self.PREFIX + "5")) @@ -331,6 +317,83 @@ class TestNumberedDir: assert folder.is_dir() +class TestRmRf: + def test_rm_rf(self, tmp_path): + from _pytest.pathlib import rm_rf + + adir = tmp_path / "adir" + adir.mkdir() + rm_rf(adir) + + assert not adir.exists() + + adir.mkdir() + afile = adir / "afile" + afile.write_bytes(b"aa") + + rm_rf(adir) + assert not adir.exists() + + def test_rm_rf_with_read_only_file(self, tmp_path): + """Ensure rm_rf can remove directories with read-only files in them (#5524)""" + from _pytest.pathlib import rm_rf + + fn = tmp_path / "dir/foo.txt" + fn.parent.mkdir() + + fn.touch() + + self.chmod_r(fn) + + rm_rf(fn.parent) + + assert not fn.parent.is_dir() + + def chmod_r(self, path): + mode = os.stat(str(path)).st_mode + os.chmod(str(path), mode & ~stat.S_IWRITE) + + def test_rm_rf_with_read_only_directory(self, tmp_path): + """Ensure rm_rf can remove read-only directories (#5524)""" + from _pytest.pathlib import rm_rf + + adir = tmp_path / "dir" + adir.mkdir() + + (adir / "foo.txt").touch() + self.chmod_r(adir) + + rm_rf(adir) + + assert not adir.is_dir() + + def test_on_rm_rf_error(self, tmp_path): + from _pytest.pathlib import on_rm_rf_error + + adir = tmp_path / "dir" + adir.mkdir() + + fn = adir / "foo.txt" + fn.touch() + self.chmod_r(fn) + + # unknown exception + with pytest.warns(pytest.PytestWarning): + exc_info = (None, RuntimeError(), None) + on_rm_rf_error(os.unlink, str(fn), exc_info, start_path=tmp_path) + assert fn.is_file() + + # unknown function + with pytest.warns(pytest.PytestWarning): + exc_info = (None, PermissionError(), None) + on_rm_rf_error(None, str(fn), exc_info, start_path=tmp_path) + assert fn.is_file() + + exc_info = (None, PermissionError(), None) + on_rm_rf_error(os.unlink, str(fn), exc_info, start_path=tmp_path) + assert not fn.is_file() + + def attempt_symlink_to(path, to_path): """Try to make a symlink from "path" to "to_path", skipping in case this platform does not support it or we don't have sufficient privileges (common on Windows).""" @@ -342,3 +405,24 @@ def attempt_symlink_to(path, to_path): def test_tmpdir_equals_tmp_path(tmpdir, tmp_path): assert Path(tmpdir) == tmp_path + + +def test_basetemp_with_read_only_files(testdir): + """Integration test for #5524""" + testdir.makepyfile( + """ + import os + import stat + + def test(tmp_path): + fn = tmp_path / 'foo.txt' + fn.write_text('hello') + mode = os.stat(str(fn)).st_mode + os.chmod(str(fn), mode & ~stat.S_IREAD) + """ + ) + result = testdir.runpytest("--basetemp=tmp") + assert result.ret == 0 + # running a second time and ensure we don't crash + result = testdir.runpytest("--basetemp=tmp") + assert result.ret == 0 diff --git a/testing/test_unittest.py b/testing/test_unittest.py index 039068269..9b1b688ff 100644 --- a/testing/test_unittest.py +++ b/testing/test_unittest.py @@ -1048,3 +1048,39 @@ def test_setup_inheritance_skipping(testdir, test_name, expected_outcome): testdir.copy_example("unittest/{}".format(test_name)) result = testdir.runpytest() result.stdout.fnmatch_lines(["* {} in *".format(expected_outcome)]) + + +def test_BdbQuit(testdir): + testdir.makepyfile( + test_foo=""" + import unittest + + class MyTestCase(unittest.TestCase): + def test_bdbquit(self): + import bdb + raise bdb.BdbQuit() + + def test_should_not_run(self): + pass + """ + ) + reprec = testdir.inline_run() + reprec.assertoutcome(failed=1, passed=1) + + +def test_exit_outcome(testdir): + testdir.makepyfile( + test_foo=""" + import pytest + import unittest + + class MyTestCase(unittest.TestCase): + def test_exit_outcome(self): + pytest.exit("pytest_exit called") + + def test_should_not_run(self): + pass + """ + ) + result = testdir.runpytest() + result.stdout.fnmatch_lines(["*Exit: pytest_exit called*", "*= no tests ran in *"]) diff --git a/tox.ini b/tox.ini index 3e3afb3a3..e158ece67 100644 --- a/tox.ini +++ b/tox.ini @@ -127,11 +127,10 @@ norecursedirs = testing/example_scripts xfail_strict=true filterwarnings = error + default:Using or importing the ABCs:DeprecationWarning:unittest2.* ignore:Module already imported so cannot be rewritten:pytest.PytestWarning - # produced by path.local - ignore:bad escape.*:DeprecationWarning:re - # produced by path.readlines - ignore:.*U.*mode is deprecated:DeprecationWarning + # produced by python3.6/site.py itself (3.6.7 on Travis, could not trigger it with 3.6.8). + ignore:.*U.*mode is deprecated:DeprecationWarning:(?!(pytest|_pytest)) # produced by pytest-xdist ignore:.*type argument to addoption.*:DeprecationWarning # produced by python >=3.5 on execnet (pytest-xdist) @@ -139,6 +138,8 @@ filterwarnings = # pytest's own futurewarnings ignore::pytest.PytestExperimentalApiWarning # Do not cause SyntaxError for invalid escape sequences in py37. + # Those are caught/handled by pyupgrade, and not easy to filter with the + # module being the filename (with .py removed). default:invalid escape sequence:DeprecationWarning # ignore use of unregistered marks, because we use many to test the implementation ignore::_pytest.warning_types.PytestUnknownMarkWarning