TestDurations tests the `--durations=N` functionality which reports N
slowest tests, with durations <= 0.005s not shown by default.
The test relies on real time.sleep() (in addition to the code which uses
time.perf_counter()) which makes it flaky and inconsistent between
platforms.
Instead of trying to tweak it more, make it use fake time instead. The
way it is done is a little hacky but seems to work.
This hook has some functionality to provide explicit markup for the test
status. It seemed unused and wasn't tested, so I was tempted to remove
it, but I found that the pytest-rerunfailures plugin uses it, so
document it and add a test instead.
Use `testdir.syspathinsert()` with multiprocessing tests:
- test_chained_exceptions_no_reprcrash
- test_exception_handling_no_traceback
This only works currently because `_importtestmodule` changes `sys.path`
as a side-effect.
It appears to be only required on Windows though - likely due to the
multiprocessing method used there.
The dynamic scope feature added in 10bf6aac76
necessitated some wrangling of arguments in pytest.fixture(). In
particular, it deprecated positional arguments in favor of keyword-only
arguments, while keeping backward compatibility.
The way it did this avoided some code duplication but ended up being
quite hard to follow and to annotate with types.
Replace it with some straightforward code, which is not very DRY but is
simple and easy to remove when the time comes.
Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Previously, writing to sys.stdout/stderr in text-mode (e.g.
`print('foo')`) while a `capsysbinary` fixture is active, would crash
with:
/usr/lib/python3.7/contextlib.py:119: in __exit__
next(self.gen)
E TypeError: write() argument must be str, not bytes
This is due to some confusion in the types. The relevant functions are
`snap()` and `writeorg()`. The function `snap()` returns what was
captured, and the return type should be `bytes` for the binary captures
and `str` for the regular ones. The `snap()` return value is eventually
passed to `writeorg()` to be written to the original file, so it's input
type should correspond to `snap()`. But this was incorrect for
`SysCaptureBinary`, which handled it like `str`.
To fix this, be explicit in the `snap()` and `writeorg()`
implementations, also of the other Capture types.
We can't add type annotations yet, because the current inheritance
scheme breaks Liskov Substitution and mypy would complain. To be
refactored later.
Fixes: https://github.com/pytest-dev/pytest/issues/6871
Co-authored-by: Ran Benita (some modifications & commit message)
I tried to understand what the `safe_text_dupfile()` function and
`EncodedFile` class do. Outside tests, `EncodedFile` is only used by
`safe_text_dupfile`, and `safe_text_dupfile` is only used by
`FDCaptureBinary.__init__()`. I then started to eliminate always-true
conditions based on the single call site, and in the end nothing was
left except of a couple workarounds that are still needed.
The commit which added the checks for os.dup a15afb5e48
suggests it was done for Jython. But pytest doesn't support Jython
anymore (Jython is Python 2 only).
Furthermore, it looks like the faulthandler plugin (bundled in pytest
and enabled by default) uses os.dup() unprotected and there have not
been any complaints.
So seems better to just remove these checks, and only add if someone
with a legitimate use case complains.
This started from fixing the test, where `"xdist_strict True"` was used
as a single argument, although you typically would see `["xdist_strict",
"True"]`.
Improves the error message to mention the option that caused the error.
pytest has several instances where plugins set their own attributes on
objects they receive in hooks, like nodes and config. Since plugins are
detached from these object's definition by design, this causes a problem
for type checking because these attributes are not defined and mypy
complains.
Fix this by giving these objects a "store" which can be used by plugins
in a type-safe manner.
Currently this mechanism is private. We can consider exposing it at a
later point.
Passing in a tuple crashes in `_prepareconfig`:
def test_invoke_with_tuple(self):
> pytest.main(("-h",))
src/_pytest/config/__init__.py:82: in main
config = _prepareconfig(args, plugins)
src/_pytest/config/__init__.py:229: in _prepareconfig
return pluginmanager.hook.pytest_cmdline_parse(
…
src/_pytest/helpconfig.py:98: in pytest_cmdline_parse
config = outcome.get_result() # type: Config
src/_pytest/config/__init__.py:808: in pytest_cmdline_parse
self.parse(args)
src/_pytest/config/__init__.py:1017: in parse
self._preparse(args, addopts=addopts)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
def _preparse(self, args: List[str], addopts: bool = True) -> None:
…
if addopts:
ini_addopts = self.getini("addopts")
if ini_addopts:
> args[:] = self._validate_args(ini_addopts, "via addopts config") + args
E TypeError: can only concatenate list (not "tuple") to list
addopts = True
args = ('-h',)
env_addopts = ''
ini_addopts = ['-rfEX', …]
src/_pytest/config/__init__.py:956: TypeError: can only concatenate list (not "tuple") to list
Might be worth handling (converting it to a list for example), but it
was documented to be a list to begin with when removing support for
strings (a7e401656).
Mypy currently is unable to handle assigning attributes on function:
https://github.com/python/mypy/issues/2087.
pytest uses this for the outcome exceptions -- `pytest.fail.Exception`,
`pytest.exit.Exception` etc, and this is the canonical name by which they
are referred.
Initially we started working around this with type: ignores, and later
by switching e.g. `pytest.fail.Exception` with the direct exception
`Failed`. But this causes a lot of churn and is not as nice. And I also
found that some code relies on it, in skipping.py:
def pytest_configure(config):
if config.option.runxfail:
# yay a hack
import pytest
old = pytest.xfail
config._cleanup.append(lambda: setattr(pytest, "xfail", old))
def nop(*args, **kwargs):
pass
nop.Exception = xfail.Exception
setattr(pytest, "xfail", nop)
...
So it seems better to support it. Use a hack to make it work. The rest
of the commit rolls back all of the workarounds we added up to now.
`pytest.raises.Exception` also exists, but it's not used much so I kept
it as-is for now.
Hopefully in the future mypy supports this and this ugliness can be
removed.
Allows for filtering of PytestCacheWarning.
Using `_issue_warning_captured` is not necessary here, and was probably
only used because the cacheprovider misses warnings during
`pytest_sessionfinish`, which is also fixed here.
I think the usage of `_issue_warning_captured` can be removed/reduced
further, but also that this is good enough for now.
Ref: https://github.com/pytest-dev/pytest/issues/6681.
* Use code highlighting if pygments is installed
* Use colorama constants instead of bare ascii codes
Could not find the exact equivalent of 'hl-reset' code using colorama
constants though.
* Refactor ASCII color handling into a fixture
* Revert back to using explicit color codes
* In Python 3.5 skip rest of tests that require ordered markup in colored output
Use suggestion in review and use a subplugin so hooks will only be active
if we enable faulthandler ourselves.
Fix#6575
Co-authored-by: Daniel Hahler <git@thequod.de>
ExitCode is used in several internal modules and hooks and so with type
annotations added, needs to be imported a lot.
_pytest.main, being the entry point, generally sits at the top of the
import tree.
So, it's not great to have ExitCode defined in _pytest.main, because it
will cause a lot of import cycles once type annotations are added (in
fact there is already one, which this change removes).
Move it to _pytest.config instead.
_pytest.main still imports ExitCode, so importing from there still
works, although external users should really be importing from `pytest`.
The convention is "assert result is expected". Pytest's error diffs now
reflect this. "-" means that sth. expected is missing in the result and
"+" means that there are unexpected extras in the result.
Fixes: #3333
Fixes collection error with Python 3.5.3 (Travis):
testing/test_parseopt.py:2: in <module>
import distutils.spawn
.tox/py35-coverage/lib/python3.5/distutils/__init__.py:4: in <module>
import imp
.tox/py35-coverage/lib/python3.5/imp.py:33: in <module>
PendingDeprecationWarning, stacklevel=2)
E PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
Build log: https://travis-ci.org/blueyed/pytest/builds/648305304
This started by looking at how to get the current test item in general,
and then I noticed that it is not necessary for the capture plugin to
track it manually in the first place.
Previously this would be turned via `py.path.local("")` into the current
working directory.
This appears to be what `fspath = fn and py.path.local(fn) or None`
tries to avoid in `getfslineno`'s `TypeError` handling already, if
`Code` would raise it.
Plugins specified with ``-p`` are now loaded after internal plugins, which
results in their hooks being called *before* the internal ones.
This makes the ``-p`` behavior consistent with ``PYTEST_PLUGINS``.
* fix/adjust test_disable_plugin_autoload
* adjust test_plugin_loading_order
Reverts https://github.com/pytest-dev/pytest/pull/6610.
The tested `getfslineno` is `src/_pytest/_code/source.py` actually,
exported via `src/_pytest/_code/__init__.py`.
I've confused it with the one in `src/_pytest/compat.py` apparently.
It should be in `test_code` when testing `_pytest._code.getfslineno`,
not to be confused with `_pytest._code.source.getfslineno`.
Adds an extra assert (via https://github.com/pytest-dev/pytest/pull/6590).
- refactor _get_main_color/build_summary_stats_line
- factor out property _is_last_item; test_summary_stats: tr._is_last_item
- _write_progress_information_filling_space: remove color arg
- use setter for stats, handling main color
- _get_main_color: skip cache for last item
- Handle random order in test for py35.
Previously, _initialparts was a list whose first item was a
`py.path.local` and the rest were `str`s. This is not something that
mypy is capable of modeling. The type `List[Union[str, py.path.local]]`
is too broad and would require asserts for every access.
Instead, make each item a `Tuple[py.path.local, List[str]]`. This way
the structure is clear and the types are accurate.
To make sure any users who might have been accessing this (private)
field will not break silently, change the name to _initial_parts.
Make EncodedFile, used for captured output streams, method .write return
the number of characters written. Add test for captured stderr write.
Fixes#6557.
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
Without restoring the cwd, successive tests might fail to parse the
config (via `_pytest.config._prepareconfig()`, for when `--lsof` is
used).
And it is good practice to restore the cwd in any case anyway.
These are more "dirty" than the previous batch (that's why they were
left out). The trouble is that `compile` can return either a code object
or an AST depending on a flag, so we need to add an overload to make the
common case Union free. But it's still worthwhile.
Somehow in Python 3.5 on Windows this test fails with:
File "c:\hostedtoolcache\windows\python\3.5.4\x64\Lib\multiprocessing\connection.py", line 302, in _recv_bytes
overlapped=True)
OSError: [WinError 6] The handle is invalid
This only happens in this platform and Python version, decided to use
a dummy traceback as originally done in #6412.
(cherry picked from commit b9c136b809)
Somehow in Python 3.5 on Windows this test fails with:
File "c:\hostedtoolcache\windows\python\3.5.4\x64\Lib\multiprocessing\connection.py", line 302, in _recv_bytes
overlapped=True)
OSError: [WinError 6] The handle is invalid
This only happens in this platform and Python version, decided to use
a dummy traceback as originally done in #6412.
Tracebacks coming from remote processes crated by the multiprocess module
will contain "RemoteTracebacks" which don't have a 'reprcrash' attribute
Fix#5971
This fixes some type: ignores due to typeshed update.
Newer mypy seem to ignore unannotated functions better, so add a few
minor annotations so that existing correct type:ignores make sense.
As far as the output is concerned, they are both identical so it doesn't
make sense to have both.
setup, teardown, and collect failures are already reported as "errors", "E".
Sets `PY_COLORS=0` in the environment by default, which is used by pylib.
Via https://github.com/blueyed/pytest/pull/58
(initially cherry picked from commit f153ad33d10)
This feature was added in commit
007a77c2ba, but was never used in pytest
itself. A GitHub code search doesn't find any users either (only pytest
repo copies). It seems safe to clean up.
The previous test was better in that it used fakes to test all of the
real code paths. The problem with that is that it makes it impossible to
simplify the code with `isinstance` checks. So let's just simulate the
issue directly with a monkeypatch.
It currently fails with a TypeError, and was not updated since 2013 -
therefore it can be assumed that it is not important to support it.
```
____________________ ERROR collecting test_nested_marks.py _____________________
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:235: in pytest_pycollect_makeitem
res = list(collector._genfunctions(name, obj))
…/Vcs/pytest/src/_pytest/python.py:404: in _genfunctions
self.ihook.pytest_generate_tests.call_extra(methods, dict(metafunc=metafunc))
…/Vcs/pluggy/src/pluggy/hooks.py:324: in call_extra
return self(**kwargs)
…/Vcs/pluggy/src/pluggy/hooks.py:286: in __call__
return self._hookexec(self, self.get_hookimpls(), kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:93: in _hookexec
return self._inner_hookexec(hook, methods, kwargs)
…/Vcs/pluggy/src/pluggy/manager.py:337: in traced_hookexec
return outcome.get_result()
…/Vcs/pluggy/src/pluggy/manager.py:335: in <lambda>
outcome = _Result.from_call(lambda: oldcall(hook, hook_impls, kwargs))
…/Vcs/pluggy/src/pluggy/manager.py:87: in <lambda>
firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
…/Vcs/pytest/src/_pytest/python.py:130: in pytest_generate_tests
metafunc.parametrize(*marker.args, **marker.kwargs)
…/Vcs/pytest/src/_pytest/python.py:965: in parametrize
function_definition=self.definition,
…/Vcs/pytest/src/_pytest/mark/structures.py:111: in _for_parametrize
if len(param.values) != len(argnames):
E TypeError: object of type 'MarkDecorator' has no len()
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
```
This is a useful utility to abstract the caching property idiom.
It is in compat.py since eventually it will be replaced by
functools.cached_property.
Fixes#6131.
It is a little too obscure IMO, but the reason I want to drop it is that
type checking has no hope of understanding such dynamic constructs.
The warning argument wasn't used.
This fixes/removes the previous hack of re-trying with minimum width,
which fails short when it splits strings.
This inherits from `pprint.PrettyPrinter` to override `_format` in a
minimal way to always dispatch, regardless of the given width.
Code ref: 5c0c325453/Lib/pprint.py (L170-L178)
Harden one test where it is tested.
All tests testing this:
testing/acceptance_test.py:184(TestGeneralUsage::test_not_collectable_arguments)
testing/acceptance_test.py:373(TestGeneralUsage::test_direct_addressing_notfound)
testing/acceptance_test.py:403(TestGeneralUsage::test_issue134_report_error_when_collecting_member[test_fun.py::test_a])
testing/acceptance_test.py:420(TestGeneralUsage::test_report_all_failed_collections_initargs)
testing/test_config.py:1309(test_config_blocked_default_plugins[python])
(via https://github.com/blueyed/pytest/pull/88)
Daniel Hahler
Ran Benita
Ronny Pfannschmidt
smarie
Bruno Oliveira
earonesty
Ronny Pfannschmidt
Bruno Oliveira
Stefan Scherfke
Vladyslav Rachek
Anthony Sottile
Holger Kohr
Tomáš Gavenčiak
Jakub Mitoraj
Chris NeJame
Pauli Virtanen
Alexandre Mulatinho
Vinay Calastry
marc
gftea
cmachalo
Christoph Buelter
Daniel Hahler
Claudio Madotto
Nikolay Kondratyev
Philipp Loose
Mark Dickinson
Bruno Oliveira
林玮
JoshKarpel
Gregory Lee
NNRepos