Merge pull request #4301 from nicoddemus/merge-master-into-features

Merge master into features
This commit is contained in:
Ronny Pfannschmidt 2018-11-04 20:23:33 +01:00 committed by GitHub
commit 5eb4016110
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
42 changed files with 470 additions and 83 deletions

View File

@ -24,12 +24,12 @@ repos:
- id: flake8
language_version: python3
- repo: https://github.com/asottile/reorder_python_imports
rev: v1.3.2
rev: v1.3.3
hooks:
- id: reorder-python-imports
args: ['--application-directories=.:src']
- repo: https://github.com/asottile/pyupgrade
rev: v1.8.0
rev: v1.10.1
hooks:
- id: pyupgrade
args: [--keep-percent-format]

View File

@ -18,6 +18,72 @@ with advance notice in the **Deprecations** section of releases.
.. towncrier release notes start
pytest 3.10.0 (2018-11-03)
==========================
Features
--------
- `#2619 <https://github.com/pytest-dev/pytest/issues/2619>`_: Resume capturing output after ``continue`` with ``__import__("pdb").set_trace()``.
This also adds a new ``pytest_leave_pdb`` hook, and passes in ``pdb`` to the
existing ``pytest_enter_pdb`` hook.
- `#4147 <https://github.com/pytest-dev/pytest/issues/4147>`_: Add ``-sw``, ``--stepwise`` as an alternative to ``--lf -x`` for stopping at the first failure, but starting the next test invocation from that test. See `the documentation <https://docs.pytest.org/en/latest/cache.html#stepwise>`__ for more info.
- `#4188 <https://github.com/pytest-dev/pytest/issues/4188>`_: Make ``--color`` emit colorful dots when not running in verbose mode. Earlier, it would only colorize the test-by-test output if ``--verbose`` was also passed.
- `#4225 <https://github.com/pytest-dev/pytest/issues/4225>`_: Improve performance with collection reporting in non-quiet mode with terminals.
The "collecting …" message is only printed/updated every 0.5s.
Bug Fixes
---------
- `#2701 <https://github.com/pytest-dev/pytest/issues/2701>`_: Fix false ``RemovedInPytest4Warning: usage of Session... is deprecated, please use pytest`` warnings.
- `#4046 <https://github.com/pytest-dev/pytest/issues/4046>`_: Fix problems with running tests in package ``__init__.py`` files.
- `#4260 <https://github.com/pytest-dev/pytest/issues/4260>`_: Swallow warnings during anonymous compilation of source.
- `#4262 <https://github.com/pytest-dev/pytest/issues/4262>`_: Fix access denied error when deleting stale directories created by ``tmpdir`` / ``tmp_path``.
- `#611 <https://github.com/pytest-dev/pytest/issues/611>`_: Naming a fixture ``request`` will now raise a warning: the ``request`` fixture is internal and
should not be overwritten as it will lead to internal errors.
Improved Documentation
----------------------
- `#4255 <https://github.com/pytest-dev/pytest/issues/4255>`_: Added missing documentation about the fact that module names passed to filter warnings are not regex-escaped.
Trivial/Internal Changes
------------------------
- `#4272 <https://github.com/pytest-dev/pytest/issues/4272>`_: Display cachedir also in non-verbose mode if non-default.
- `#4277 <https://github.com/pytest-dev/pytest/issues/4277>`_: pdb: improve message about output capturing with ``set_trace``.
Do not display "IO-capturing turned off/on" when ``-s`` is used to avoid
confusion.
- `#4279 <https://github.com/pytest-dev/pytest/issues/4279>`_: Improve message and stack level of warnings issued by ``monkeypatch.setenv`` when the value of the environment variable is not a ``str``.
pytest 3.9.3 (2018-10-27)
=========================
@ -366,7 +432,7 @@ Features
the standard warnings filters to manage those warnings. This introduces ``PytestWarning``,
``PytestDeprecationWarning`` and ``RemovedInPytest4Warning`` warning types as part of the public API.
Consult `the documentation <https://docs.pytest.org/en/latest/warnings.html#internal-pytest-warnings>`_ for more info.
Consult `the documentation <https://docs.pytest.org/en/latest/warnings.html#internal-pytest-warnings>`__ for more info.
- `#2908 <https://github.com/pytest-dev/pytest/issues/2908>`_: ``DeprecationWarning`` and ``PendingDeprecationWarning`` are now shown by default if no other warning filter is
@ -541,7 +607,7 @@ Bug Fixes
- `#3473 <https://github.com/pytest-dev/pytest/issues/3473>`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn't understand (e.g. strings, nested dicts, etc.).
- `#3712 <https://github.com/pytest-dev/pytest/issues/3712>`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``.
- `#3712 <https://github.com/pytest-dev/pytest/issues/3712>`_: Correctly represent the dimensions of a numpy array when calling ``repr()`` on ``approx()``.
- `#3742 <https://github.com/pytest-dev/pytest/issues/3742>`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute '_collectfile'``.

View File

@ -46,5 +46,3 @@ taking a lot of time to make a new one.
* testing-in-python@lists.idyll.org (only major/minor releases)
And announce it on `Twitter <https://twitter.com/>`_ with the ``#pytest`` hashtag.
#. After a minor/major release, merge ``release-X.Y.Z`` into ``master`` and push (or open a PR).

View File

@ -1,4 +0,0 @@
Resume capturing output after ``continue`` with ``__import__("pdb").set_trace()``.
This also adds a new ``pytest_leave_pdb`` hook, and passes in ``pdb`` to the
existing ``pytest_enter_pdb`` hook.

View File

@ -1 +0,0 @@
Add ``-sw``, ``--stepwise`` as an alternative to ``--lf -x`` for stopping at the first failure, but starting the next test invocation from that test. See `the documentation <https://docs.pytest.org/en/latest/cache.html#stepwise>`_ for more info.

View File

@ -1 +0,0 @@
Make ``--color`` emit colorful dots when not running in verbose mode. Earlier, it would only colorize the test-by-test output if ``--verbose`` was also passed.

View File

@ -1,3 +0,0 @@
Improve performance with collection reporting in non-quiet mode with terminals.
The "collecting …" message is only printed/updated every 0.5s.

View File

@ -1 +0,0 @@
Display cachedir also in non-verbose mode if non-default.

View File

@ -1,4 +0,0 @@
pdb: improve message about output capturing with ``set_trace``.
Do not display "IO-capturing turned off/on" when ``-s`` is used to avoid
confusion.

View File

@ -0,0 +1 @@
Fix nested usage of debugging plugin (pdb), e.g. with pytester's ``testdir.runpytest``.

View File

@ -1,2 +0,0 @@
Naming a fixture ``request`` will now raise a warning: the ``request`` fixture is internal and
should not be overwritten as it will lead to internal errors.

1
doc/4266.bugfix.rst Normal file
View File

@ -0,0 +1 @@
Handle (ignore) exceptions raised during collection, e.g. with Django's LazySettings proxy class.

View File

@ -6,6 +6,7 @@ Release announcements
:maxdepth: 2
release-3.10.0
release-3.9.3
release-3.9.2
release-3.9.1

View File

@ -0,0 +1,43 @@
pytest-3.10.0
=======================================
The pytest team is proud to announce the 3.10.0 release!
pytest is a mature Python testing tool with more than a 2000 tests
against itself, passing on many different interpreters and platforms.
This release contains a number of bugs fixes and improvements, so users are encouraged
to take a look at the CHANGELOG:
https://docs.pytest.org/en/latest/changelog.html
For complete documentation, please visit:
https://docs.pytest.org/en/latest/
As usual, you can upgrade from pypi via:
pip install -U pytest
Thanks to all who contributed to this release, among them:
* Anders Hovmöller
* Andreu Vallbona Plazas
* Ankit Goel
* Anthony Sottile
* Bernardo Gomes
* Brianna Laugher
* Bruno Oliveira
* Daniel Hahler
* David Szotten
* Mick Koch
* Niclas Olofsson
* Palash Chatterjee
* Ronny Pfannschmidt
* Sven-Hendrik Haase
* Ville Skyttä
* William Jamir Silva
Happy testing,
The Pytest Development Team

View File

@ -244,6 +244,8 @@ You can always peek at the content of the cache using the
{'test_caching.py::test_function': True}
cache/nodeids contains:
['test_caching.py::test_function']
cache/stepwise contains:
[]
example/value contains:
42

View File

@ -33,6 +33,7 @@ Full pytest documentation
reference
goodpractices
flaky
pythonpath
customize
example/index

View File

@ -33,7 +33,7 @@ class Python(object):
dumpfile = self.picklefile.dirpath("dump.py")
dumpfile.write(
textwrap.dedent(
"""\
r"""
import pickle
f = open({!r}, 'wb')
s = pickle.dump({!r}, f, protocol=2)
@ -49,7 +49,7 @@ class Python(object):
loadfile = self.picklefile.dirpath("load.py")
loadfile.write(
textwrap.dedent(
"""\
r"""
import pickle
f = open({!r}, 'rb')
obj = pickle.load(f)

View File

@ -85,8 +85,9 @@ interesting to just look at the collection tree::
rootdir: $REGENDOC_TMPDIR/nonpython, inifile:
collected 2 items
<Package '$REGENDOC_TMPDIR/nonpython'>
<YamlFile 'test_simple.yml'>
<YamlItem 'hello'>
<YamlItem 'ok'>
<Package '$REGENDOC_TMPDIR/nonpython'>
<YamlFile 'test_simple.yml'>
<YamlItem 'hello'>
<YamlItem 'ok'>
======================= no tests ran in 0.12 seconds =======================

View File

@ -153,7 +153,7 @@ This makes use of the automatic caching mechanisms of pytest.
Another good approach is by adding the data files in the ``tests`` folder.
There are also community plugins available to help managing this aspect of
testing, e.g. `pytest-datadir <https://github.com/gabrielcnr/pytest-datadir>`__
testing, e.g. `pytest-datadir <https://pypi.org/project/pytest-datadir/>`__
and `pytest-datafiles <https://pypi.org/project/pytest-datafiles/>`__.
.. _smtpshared:

125
doc/en/flaky.rst Normal file
View File

@ -0,0 +1,125 @@
Flaky tests
-----------
A "flaky" test is one that exhibits intermittent or sporadic failure, that seems to have non-deterministic behaviour. Sometimes it passes, sometimes it fails, and it's not clear why. This page discusses pytest features that can help and other general strategies for identifying, fixing or mitigating them.
Why flaky tests are a problem
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Flaky tests are particularly troublesome when a continuous integration (CI) server is being used, so that all tests must pass before a new code change can be merged. If the test result is not a reliable signal -- that a test failure means the code change broke the test -- developers can become mistrustful of the test results, which can lead to overlooking genuine failures. It is also a source of wasted time as developers must re-run test suites and investigate spurious failures.
Potential root causes
^^^^^^^^^^^^^^^^^^^^^
System state
~~~~~~~~~~~~
Broadly speaking, a flaky test indicates that the test relies on some system state that is not being appropriately controlled - the test environment is not sufficiently isolated. Higher level tests are more likely to be flaky as they rely on more state.
Flaky tests sometimes appear when a test suite is run in parallel (such as use of pytest-xdist). This can indicate a test is reliant on test ordering.
- Perhaps a different test is failing to clean up after itself and leaving behind data which causes the flaky test to fail.
- The flaky test is reliant on data from a previous test that doesn't clean up after itself, and in parallel runs that previous test is not always present
- Tests that modify global state typically cannot be run in parallel.
Overly strict assertion
~~~~~~~~~~~~~~~~~~~~~~~
Overly strict assertions can cause problems with floating point comparison as well as timing issues. `pytest.approx <https://docs.pytest.org/en/latest/reference.html#pytest-approx>`_ is useful here.
Pytest features
^^^^^^^^^^^^^^^
Xfail strict
~~~~~~~~~~~~
:ref:`pytest.mark.xfail ref` with ``strict=False`` can be used to mark a test so that its failure does not cause the whole build to break. This could be considered like a manual quarantine, and is rather dangerous to use permanently.
PYTEST_CURRENT_TEST
~~~~~~~~~~~~~~~~~~~
:ref:`pytest current test env` may be useful for figuring out "which test got stuck".
Plugins
~~~~~~~
Rerunning any failed tests can mitigate the negative effects of flaky tests by giving them additional chances to pass, so that the overall build does not fail. Several pytest plugins support this:
* `flaky <https://github.com/box/flaky>`_
* `pytest-flakefinder <https://github.com/dropbox/pytest-flakefinder>`_ - `blog post <https://blogs.dropbox.com/tech/2016/03/open-sourcing-pytest-tools/>`_
* `pytest-rerunfailures <https://github.com/pytest-dev/pytest-rerunfailures>`_
* `pytest-replay <https://github.com/ESSS/pytest-replay>`_: This plugin helps to reproduce locally crashes or flaky tests observed during CI runs.
Plugins to deliberately randomize tests can help expose tests with state problems:
* `pytest-random-order <https://github.com/jbasko/pytest-random-order>`_
* `pytest-randomly <https://github.com/pytest-dev/pytest-randomly>`_
Other general strategies
^^^^^^^^^^^^^^^^^^^^^^^^
Split up test suites
~~~~~~~~~~~~~~~~~~~~
It can be common to split a single test suite into two, such as unit vs integration, and only use the unit test suite as a CI gate. This also helps keep build times manageable as high level tests tend to be slower. However, it means it does become possible for code that breaks the build to be merged, so extra vigilance is needed for monitoring the integration test results.
Video/screenshot on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~
For UI tests these are important for understanding what the state of the UI was when the test failed. pytest-splinter can be used with plugins like pytest-bdd and can `save a screenshot on test failure <https://pytest-splinter.readthedocs.io/en/latest/#automatic-screenshots-on-test-failure>`_, which can help to isolate the cause.
Delete or rewrite the test
~~~~~~~~~~~~~~~~~~~~~~~~~~
If the functionality is covered by other tests, perhaps the test can be removed. If not, perhaps it can be rewritten at a lower level which will remove the flakiness or make its source more apparent.
Quarantine
~~~~~~~~~~
Mark Lapierre discusses the `Pros and Cons of Quarantined Tests <https://dev.to/mlapierre/pros-and-cons-of-quarantined-tests-2emj>`_ in a post from 2018.
CI tools that rerun on failure
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Azure Pipelines (the Azure cloud CI/CD tool, formerly Visual Studio Team Services or VSTS) has a feature to `identify flaky tests <https://docs.microsoft.com/en-us/azure/devops/release-notes/2017/dec-11-vsts#identify-flaky-tests>`_ and rerun failed tests.
Research
^^^^^^^^
This is a limited list, please submit an issue or pull request to expand it!
* Gao, Zebao, Yalan Liang, Myra B. Cohen, Atif M. Memon, and Zhen Wang. "Making system user interactive tests repeatable: When and what should we control?." In *Software Engineering (ICSE), 2015 IEEE/ACM 37th IEEE International Conference on*, vol. 1, pp. 55-65. IEEE, 2015. `PDF <http://www.cs.umd.edu/~atif/pubs/gao-icse15.pdf>`__
* Palomba, Fabio, and Andy Zaidman. "Does refactoring of test smells induce fixing flaky tests?." In *Software Maintenance and Evolution (ICSME), 2017 IEEE International Conference on*, pp. 1-12. IEEE, 2017. `PDF in Google Drive <https://drive.google.com/file/d/10HdcCQiuQVgW3yYUJD-TSTq1NbYEprl0/view>`__
* Bell, Jonathan, Owolabi Legunsen, Michael Hilton, Lamyaa Eloussi, Tifany Yung, and Darko Marinov. "DeFlaker: Automatically detecting flaky tests." In *Proceedings of the 2018 International Conference on Software Engineering*. 2018. `PDF <https://www.jonbell.net/icse18-deflaker.pdf>`__
Resources
^^^^^^^^^
* `Eradicating Non-Determinism in Tests <https://martinfowler.com/articles/nonDeterminism.html>`_ by Martin Fowler, 2011
* `No more flaky tests on the Go team <https://www.thoughtworks.com/insights/blog/no-more-flaky-tests-go-team>`_ by Pavan Sudarshan, 2012
* `The Build That Cried Broken: Building Trust in your Continuous Integration Tests <https://www.youtube.com/embed/VotJqV4n8ig>`_ talk (video) by `Angie Jones <http://angiejones.tech/>`_ at SeleniumConf Austin 2017
* `Test and Code Podcast: Flaky Tests and How to Deal with Them <https://testandcode.com/50>`_ by Brian Okken and Anthony Shaw, 2018
* Microsoft:
* `How we approach testing VSTS to enable continuous delivery <https://blogs.msdn.microsoft.com/bharry/2017/06/28/testing-in-a-cloud-delivery-cadence/>`_ by Brian Harry MS, 2017
* `Eliminating Flaky Tests <https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/eliminating-flaky-tests>`_ blog and talk (video) by Munil Shah, 2017
* Google:
* `Flaky Tests at Google and How We Mitigate Them <https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html>`_ by John Micco, 2016
* `Where do Google's flaky tests come from? <https://docs.google.com/document/d/1mZ0-Kc97DI_F3tf_GBW_NB_aqka-P1jVOsFfufxqUUM/edit#heading=h.ec0r4fypsleh>`_ by Jeff Listfield, 2017

View File

@ -75,7 +75,7 @@ Issues
------
* By using ``request.getfuncargvalue()`` we rely on actual fixture function
execution to know what fixtures are involved, due to it's dynamic nature
execution to know what fixtures are involved, due to its dynamic nature
* More importantly, ``request.getfuncargvalue()`` cannot be combined with
parametrized fixtures, such as ``extra_context``
* This is very inconvenient if you wish to extend an existing test suite by

View File

@ -117,6 +117,7 @@ Add warning filters to marked test items.
A *warning specification string*, which is composed of contents of the tuple ``(action, message, category, module, lineno)``
as specified in `The Warnings filter <https://docs.python.org/3/library/warnings.html#warning-filter>`_ section of
the Python documentation, separated by ``":"``. Optional fields can be omitted.
Module names passed for filtering are not regex-escaped.
For example:

View File

@ -23,6 +23,8 @@ Books
Talks and blog postings
---------------------------------------------
- pytest: recommendations, basic packages for testing in Python and Django, Andreu Vallbona, PyconES 2017 (`slides in english <http://talks.apsl.io/testing-pycones-2017/>`_, `video in spanish <https://www.youtube.com/watch?v=K20GeR-lXDk>`_)
- `Pythonic testing, Igor Starikov (Russian, PyNsk, November 2016)
<https://www.youtube.com/watch?v=_92nfdd5nK8>`_.

View File

@ -386,7 +386,7 @@ return a result object, with which we can assert the tests' outcomes.
result.assert_outcomes(passed=4)
additionally it is possible to copy examples for a example folder before running pytest on it
additionally it is possible to copy examples for an example folder before running pytest on it
.. code:: ini
@ -421,21 +421,9 @@ additionally it is possible to copy examples for a example folder before running
test_example.py::test_plugin
$REGENDOC_TMPDIR/test_example.py:4: PytestExperimentalApiWarning: testdir.copy_example is an experimental api that may change over time
testdir.copy_example("test_example.py")
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.Class is deprecated, please use pytest.Class instead
return getattr(object, name, default)
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.File is deprecated, please use pytest.File instead
return getattr(object, name, default)
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.Function is deprecated, please use pytest.Function instead
return getattr(object, name, default)
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.Instance is deprecated, please use pytest.Instance instead
return getattr(object, name, default)
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.Item is deprecated, please use pytest.Item instead
return getattr(object, name, default)
$PYTHON_PREFIX/lib/python3.6/site-packages/_pytest/compat.py:332: RemovedInPytest4Warning: usage of Session.Module is deprecated, please use pytest.Module instead
return getattr(object, name, default)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
=================== 2 passed, 7 warnings in 0.12 seconds ===================
=================== 2 passed, 1 warnings in 0.12 seconds ===================
For more information about the result object that ``runpytest()`` returns, and
the methods that it provides please check out the :py:class:`RunResult

View File

@ -8,14 +8,13 @@ import linecache
import sys
import textwrap
import tokenize
import warnings
from ast import PyCF_ONLY_AST as _AST_FLAG
from bisect import bisect_right
import py
import six
cpy_compile = compile
class Source(object):
""" an immutable object holding a source code fragment,
@ -161,7 +160,7 @@ class Source(object):
filename = base + "%r %s:%d>" % (filename, fn, lineno)
source = "\n".join(self.lines) + "\n"
try:
co = cpy_compile(source, filename, mode, flag)
co = compile(source, filename, mode, flag)
except SyntaxError:
ex = sys.exc_info()[1]
# re-represent syntax errors from parsing python strings
@ -195,7 +194,7 @@ def compile_(source, filename=None, mode="exec", flags=0, dont_inherit=0):
"""
if isinstance(source, ast.AST):
# XXX should Source support having AST?
return cpy_compile(source, filename, mode, flags, dont_inherit)
return compile(source, filename, mode, flags, dont_inherit)
_genframe = sys._getframe(1) # the caller
s = Source(source)
co = s.compile(filename, mode, flags, _genframe=_genframe)
@ -290,7 +289,11 @@ def get_statement_startend2(lineno, node):
def getstatementrange_ast(lineno, source, assertion=False, astnode=None):
if astnode is None:
content = str(source)
astnode = compile(content, "source", "exec", 1024) # 1024 for AST
# See #4260:
# don't produce duplicate warnings when compiling source to find ast
with warnings.catch_warnings():
warnings.simplefilter("ignore")
astnode = compile(content, "source", "exec", _AST_FLAG)
start, end = get_statement_startend2(lineno, astnode)
# we need to correct the end:

View File

@ -127,7 +127,7 @@ class CaptureManager(object):
def read_global_capture(self):
return self._global_capturing.readouterr()
# Fixture Control (its just forwarding, think about removing this later)
# Fixture Control (it's just forwarding, think about removing this later)
def activate_fixture(self, item):
"""If the current item is using ``capsys`` or ``capfd``, activate them so they take precedence over

View File

@ -334,6 +334,14 @@ def safe_getattr(object, name, default):
return default
def safe_isclass(obj):
"""Ignore any exception via isinstance on Python 3."""
try:
return isclass(obj)
except Exception:
return False
def _is_unittest_unexpected_success_a_failure():
"""Return if the test suite should fail if an @expectedFailure unittest test PASSES.

View File

@ -152,7 +152,7 @@ class ArgumentError(Exception):
class Argument(object):
"""class that mimics the necessary behaviour of optparse.Option
its currently a least effort implementation
it's currently a least effort implementation
and ignoring choices and integer prefixes
https://docs.python.org/3/library/optparse.html#optparse-standard-option-types
"""

View File

@ -47,17 +47,24 @@ def pytest_configure(config):
if config.getvalue("usepdb"):
config.pluginmanager.register(PdbInvoke(), "pdbinvoke")
old = (pdb.set_trace, pytestPDB._pluginmanager)
def fin():
pdb.set_trace, pytestPDB._pluginmanager = old
pytestPDB._config = None
pytestPDB._pdb_cls = pdb.Pdb
pytestPDB._saved.append(
(pdb.set_trace, pytestPDB._pluginmanager, pytestPDB._config, pytestPDB._pdb_cls)
)
pdb.set_trace = pytestPDB.set_trace
pytestPDB._pluginmanager = config.pluginmanager
pytestPDB._config = config
pytestPDB._pdb_cls = pdb_cls
# NOTE: not using pytest_unconfigure, since it might get called although
# pytest_configure was not (if another plugin raises UsageError).
def fin():
(
pdb.set_trace,
pytestPDB._pluginmanager,
pytestPDB._config,
pytestPDB._pdb_cls,
) = pytestPDB._saved.pop()
config._cleanup.append(fin)
@ -67,6 +74,7 @@ class pytestPDB(object):
_pluginmanager = None
_config = None
_pdb_cls = pdb.Pdb
_saved = []
@classmethod
def set_trace(cls, set_break=True):

View File

@ -619,7 +619,7 @@ class FixtureRequest(FuncargnamesCompatAttr):
subrequest._check_scope(argname, self.scope, scope)
# clear sys.exc_info before invoking the fixture (python bug?)
# if its not explicitly cleared it will leak into the call
# if it's not explicitly cleared it will leak into the call
exc_clear()
try:
# call the fixture function
@ -1197,6 +1197,7 @@ class FixtureManager(object):
nodeid = p.dirpath().relto(self.config.rootdir)
if p.sep != nodes.SEP:
nodeid = nodeid.replace(p.sep, nodes.SEP)
self.parsefactories(plugin, nodeid)
def _getautousenames(self, nodeid):
@ -1301,11 +1302,18 @@ class FixtureManager(object):
nodeid = node_or_obj.nodeid
if holderobj in self._holderobjseen:
return
from _pytest.nodes import _CompatProperty
self._holderobjseen.add(holderobj)
autousenames = []
for name in dir(holderobj):
# The attribute can be an arbitrary descriptor, so the attribute
# access below can raise. safe_getatt() ignores such exceptions.
maybe_property = safe_getattr(type(holderobj), name, None)
if isinstance(maybe_property, _CompatProperty):
# deprecated
continue
obj = safe_getattr(holderobj, name, None)
marker = getfixturemarker(obj)
# fixture functions have a pytest_funcarg__ prefix (pre-2.3 style)

View File

@ -41,10 +41,10 @@ def pytest_namespace():
Plugins whose users depend on the current namespace functionality should prepare to migrate to a
namespace they actually own.
To support the migration its suggested to trigger ``DeprecationWarnings`` for objects they put into the
To support the migration it's suggested to trigger ``DeprecationWarnings`` for objects they put into the
pytest namespace.
An stopgap measure to avoid the warning is to monkeypatch the ``pytest`` module, but just as the
A stopgap measure to avoid the warning is to monkeypatch the ``pytest`` module, but just as the
``pytest_namespace`` hook this should be seen as a temporary measure to be removed in future versions after
an appropriate transition period.
"""

View File

@ -559,7 +559,15 @@ class Session(nodes.FSCollector):
col = root._collectfile(argpath)
if col:
self._node_cache[argpath] = col
for y in self.matchnodes(col, names):
m = self.matchnodes(col, names)
# If __init__.py was the only file requested, then the matched node will be
# the corresponding Package, and the first yielded item will be the __init__
# Module itself, so just use that. If this special case isn't taken, then all
# the files in the package will be yielded.
if argpath.basename == "__init__.py":
yield next(m[0].collect())
return
for y in m:
yield y
def _collectfile(self, path):

View File

@ -443,7 +443,7 @@ class NodeKeywords(MappingMixin):
@attr.s(cmp=False, hash=False)
class NodeMarkers(object):
"""
internal strucutre for storing marks belongong to a node
internal structure for storing marks belonging to a node
..warning::

View File

@ -230,10 +230,12 @@ class MonkeyPatch(object):
if not isinstance(value, str):
warnings.warn(
pytest.PytestWarning(
"Environment variable value {!r} should be str, converted to str implicitly".format(
value
"Value of environment variable {name} type should be str, but got "
"{value!r} (type: {type}); converted to str implicitly".format(
name=name, value=value, type=type(value).__name__
)
)
),
stacklevel=2,
)
value = str(value)
if prepend and name in os.environ:

View File

@ -35,7 +35,7 @@ get_lock_path = operator.methodcaller("joinpath", ".lock")
def ensure_reset_dir(path):
"""
ensures the given path is a empty directory
ensures the given path is an empty directory
"""
if path.exists():
rmtree(path, force=True)
@ -98,8 +98,8 @@ else:
def _force_symlink(root, target, link_to):
"""helper to create the current symlink
its full of race conditions that are reasonably ok to ignore
for the contex of best effort linking to the latest testrun
it's full of race conditions that are reasonably ok to ignore
for the context of best effort linking to the latest testrun
the presumption being thatin case of much parallelism
the inaccuracy is going to be acceptable
@ -116,7 +116,7 @@ def _force_symlink(root, target, link_to):
def make_numbered_dir(root, prefix):
"""create a directory with a increased number as suffix for the given prefix"""
"""create a directory with an increased number as suffix for the given prefix"""
for i in range(10):
# try up to 10 times to create the folder
max_existing = _max(map(parse_num, find_suffixes(root, prefix)), default=-1)
@ -156,7 +156,7 @@ def create_cleanup_lock(p):
os.write(fd, spid)
os.close(fd)
if not lock_path.is_file():
raise EnvironmentError("lock path got renamed after sucessfull creation")
raise EnvironmentError("lock path got renamed after successful creation")
return lock_path
@ -178,19 +178,29 @@ def register_cleanup_lock_removal(lock_path, register=atexit.register):
def maybe_delete_a_numbered_dir(path):
"""removes a numbered directory if its lock can be obtained"""
"""removes a numbered directory if its lock can be obtained and it does not seem to be in use"""
lock_path = None
try:
create_cleanup_lock(path)
lock_path = create_cleanup_lock(path)
parent = path.parent
garbage = parent.joinpath("garbage-{}".format(uuid.uuid4()))
path.rename(garbage)
rmtree(garbage, force=True)
except (OSError, EnvironmentError):
# known races:
# * other process did a cleanup at the same time
# * deletable folder was found
# * process cwd (Windows)
return
parent = path.parent
garbage = parent.joinpath("garbage-{}".format(uuid.uuid4()))
path.rename(garbage)
rmtree(garbage, force=True)
finally:
# if we created the lock, ensure we remove it even if we failed
# to properly remove the numbered dir
if lock_path is not None:
try:
lock_path.unlink()
except (OSError, IOError):
pass
def ensure_deletable(path, consider_lock_dead_if_created_before):
@ -213,7 +223,7 @@ def ensure_deletable(path, consider_lock_dead_if_created_before):
def try_cleanup(path, consider_lock_dead_if_created_before):
"""tries to cleanup a folder if we can ensure its deletable"""
"""tries to cleanup a folder if we can ensure it's deletable"""
if ensure_deletable(path, consider_lock_dead_if_created_before):
maybe_delete_a_numbered_dir(path)

View File

@ -33,6 +33,7 @@ from _pytest.compat import NoneType
from _pytest.compat import NOTSET
from _pytest.compat import REGEX_TYPE
from _pytest.compat import safe_getattr
from _pytest.compat import safe_isclass
from _pytest.compat import safe_str
from _pytest.compat import STRING_TYPES
from _pytest.config import hookimpl
@ -196,7 +197,7 @@ def pytest_pycollect_makeitem(collector, name, obj):
if res is not None:
return
# nothing was collected elsewhere, let's do it here
if isclass(obj):
if safe_isclass(obj):
if collector.istestclass(obj, name):
Class = collector._getcustomclass("Class")
outcome.force_result(Class(name, parent=collector))
@ -652,7 +653,7 @@ class Instance(PyCollector):
_ALLOW_MARKERS = False # hack, destroy later
# instances share the object with their parents in a way
# that duplicates markers instances if not taken out
# can be removed at node strucutre reorganization time
# can be removed at node structure reorganization time
def _getobj(self):
return self.parent.obj()
@ -1334,7 +1335,7 @@ class Function(FunctionMixin, nodes.Item, fixtures.FuncargnamesCompatAttr):
"""
_genid = None
# disable since functions handle it themselfes
# disable since functions handle it themselves
_ALLOW_MARKERS = False
def __init__(

View File

@ -1,5 +1,7 @@
import sys
import six
import pytest
from _pytest.outcomes import Failed
@ -170,3 +172,25 @@ class TestRaises(object):
Failed, match="DID NOT RAISE <class 'raises.ClassLooksIterableException'>"
):
pytest.raises(ClassLooksIterableException, lambda: None)
def test_raises_with_raising_dunder_class(self):
"""Test current behavior with regard to exceptions via __class__ (#4284)."""
class CrappyClass(Exception):
@property
def __class__(self):
assert False, "via __class__"
if six.PY2:
with pytest.raises(pytest.fail.Exception) as excinfo:
with pytest.raises(CrappyClass()):
pass
assert "DID NOT RAISE" in excinfo.value.args[0]
with pytest.raises(CrappyClass) as excinfo:
raise CrappyClass()
else:
with pytest.raises(AssertionError) as excinfo:
with pytest.raises(CrappyClass()):
pass
assert "via __class__" in excinfo.value.args[0]

View File

@ -957,6 +957,21 @@ def test_collect_init_tests(testdir):
"*<Function 'test_foo'>",
]
)
result = testdir.runpytest("./tests", "--collect-only")
result.stdout.fnmatch_lines(
[
"*<Module '__init__.py'>",
"*<Function 'test_init'>",
"*<Module 'test_foo.py'>",
"*<Function 'test_foo'>",
]
)
result = testdir.runpytest("./tests/test_foo.py", "--collect-only")
result.stdout.fnmatch_lines(["*<Module 'test_foo.py'>", "*<Function 'test_foo'>"])
assert "test_init" not in result.stdout.str()
result = testdir.runpytest("./tests/__init__.py", "--collect-only")
result.stdout.fnmatch_lines(["*<Module '__init__.py'>", "*<Function 'test_init'>"])
assert "test_foo" not in result.stdout.str()
def test_collect_invalid_signature_message(testdir):
@ -977,3 +992,30 @@ def test_collect_invalid_signature_message(testdir):
result.stdout.fnmatch_lines(
["Could not determine arguments of *.fix *: invalid method signature"]
)
def test_collect_handles_raising_on_dunder_class(testdir):
"""Handle proxy classes like Django's LazySettings that might raise on
``isinstance`` (#4266).
"""
testdir.makepyfile(
"""
class ImproperlyConfigured(Exception):
pass
class RaisesOnGetAttr(object):
def raises(self):
raise ImproperlyConfigured
__class__ = property(raises)
raises = RaisesOnGetAttr()
def test_1():
pass
"""
)
result = testdir.runpytest()
assert result.ret == 0
result.stdout.fnmatch_lines(["*1 passed in*"])

View File

@ -12,6 +12,7 @@ from _pytest.compat import _PytestWrapper
from _pytest.compat import get_real_func
from _pytest.compat import is_generator
from _pytest.compat import safe_getattr
from _pytest.compat import safe_isclass
from _pytest.outcomes import OutcomeException
@ -140,3 +141,14 @@ def test_safe_getattr():
helper = ErrorsHelper()
assert safe_getattr(helper, "raise_exception", "default") == "default"
assert safe_getattr(helper, "raise_fail", "default") == "default"
def test_safe_isclass():
assert safe_isclass(type) is True
class CrappyClass(Exception):
@property
def __class__(self):
assert False, "Should be ignored"
assert safe_isclass(CrappyClass()) is False

View File

@ -3,6 +3,7 @@ from __future__ import division
from __future__ import print_function
import os
import re
import sys
import textwrap
@ -226,9 +227,10 @@ class TestEnvironWarnings(object):
def test_setenv_non_str_warning(self, monkeypatch):
value = 2
msg = (
"Environment variable value {!r} should be str, converted to str implicitly"
"Value of environment variable PYTEST_INTERNAL_MY_VAR type should be str, "
"but got 2 (type: int); converted to str implicitly"
)
with pytest.warns(pytest.PytestWarning, match=msg.format(value)):
with pytest.warns(pytest.PytestWarning, match=re.escape(msg)):
monkeypatch.setenv(str(self.VAR_NAME), value)

View File

@ -4,6 +4,9 @@ import py
import pytest
from _pytest.pathlib import fnmatch_ex
from _pytest.pathlib import get_lock_path
from _pytest.pathlib import maybe_delete_a_numbered_dir
from _pytest.pathlib import Path
class TestPort:
@ -66,3 +69,18 @@ class TestPort:
)
def test_not_matching(self, match, pattern, path):
assert not match(pattern, path)
def test_access_denied_during_cleanup(tmp_path, monkeypatch):
"""Ensure that deleting a numbered dir does not fail because of OSErrors (#4262)."""
path = tmp_path / "temp-1"
path.mkdir()
def renamed_failed(*args):
raise OSError("access denied")
monkeypatch.setattr(Path, "rename", renamed_failed)
lock_path = get_lock_path(path)
maybe_delete_a_numbered_dir(path)
assert not lock_path.is_file()

View File

@ -826,3 +826,30 @@ class TestTraceOption:
assert "1 passed" in rest
assert "reading from stdin while output" not in rest
TestPDB.flush(child)
def test_trace_after_runpytest(testdir):
"""Test that debugging's pytest_configure is re-entrant."""
p1 = testdir.makepyfile(
"""
from _pytest.debugging import pytestPDB
def test_outer(testdir):
from _pytest.debugging import pytestPDB
assert len(pytestPDB._saved) == 1
testdir.runpytest("-k test_inner")
__import__('pdb').set_trace()
def test_inner(testdir):
assert len(pytestPDB._saved) == 2
"""
)
child = testdir.spawn_pytest("-p pytester %s -k test_outer" % p1)
child.expect(r"\(Pdb")
child.sendline("c")
rest = child.read().decode("utf8")
TestPDB.flush(child)
assert child.exitstatus == 0, rest