`Parser` is used by many plugins and custom hooks. `OptionGroup` is
exposed by the `parser.addgroup` API.
The constructors of both are marked private, they are not meant to be
constructed directly.
* add feature to view fixture source location in invocations with --fixtures-per-test option
* remove unrelated changes to show_fixtures_per_test::test_doctest_items
* eshew the extraneous else in _show_fixtures_per_test.write_fixture
* enable the accommodation of multi-line docstring with --fixtures-per-test option
* add feature to view fixture source location in invocations with --fixtures
* add colour encoding to fixture location paths
* add changelog for #8606 fixing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix issue where TestCase.setUpClass is not called for test methods with a / in its name by checking if there is :: before the selected / or any :: after. Also added a test case for this.
* removed print statement that was added
* Change iterparentnodeids to consume / parts until the first ::. Then consider ::. Tests were changed to reflect this.
* Update changelog/8509.improvement.rst
Co-authored-by: Ran Benita <ran@unusedvar.com>
pytest uses a root temp directory named `/tmp/pytest-of-<username>`. The
name is predictable, and the directory might already exists from a
previous run, so that's allowed.
This makes it possible for my_user to pre-create
`/tmp/pytest-of-another_user`, thus giving my_user control of
another_user's tempdir.
Prevent this scenario by adding a couple of safety checks. I believe
they are sufficient.
Testing the first check requires changing the owner, which requires
root permissions, so can't be unit-tested easily, but I checked it
manually.
(Written for a Unix system, but might be applicable to Windows as well).
pytest creates a root temporary directory under /tmp, named
`pytest-of-<username>`, and creates tmp_path's and other under it.
/tmp is shared between all users of the system.
This root temporary directory was created with 0o777&~umask permissions,
which usually becomes 0o755, meaning any user in the system could list
and read the files, which is undesirable.
Use 0o700 permissions instead. Also for subdirectories, because the root
dir is adjustable.
Calling pkg_resources.fixup_namespace_packages() is only needed for packages
that use pkg_resources.declare_namespace() and hence they already imported
pkg_resources. When pkg_resources is not imported, we don't need to use it.
This avoids an unneeded runtime dependency on setuptools.
The code is tested by test_syspath_prepend_with_namespace_packages,
behavior should remain unchanged, hence no new test was added.
When people drop pkg_resources from sys.modules, they are on their own.
If someone has a actual use case making this valid to support,
they can come in and provide a test, a reference and a fix.
This type is most prominent in `pytest.raises` and we should allow to
refer to it by a public name.
The type is not in a perfectly "exposable" state. In particular:
- The `traceback` property with type `Traceback` which is derived from
the `py.code` API and exposes a bunch more types transitively. This
stuff is *not* exported and probably won't be.
- The `getrepr` method which probably should be private.
But they're already used in the wild so no point in just hiding them
now.
The __init__ API is hidden -- the public API for this are the `from_*`
classmethods.
Moved various documents into subdirectories, how-to and
reference.
Updated multiple links to use `:ref:` instead of `:doc:`,
meaning that files can henceforth be moved around without
breaking references.
* Fix test_strict_and_skip
The `--strict` argument was removed in #2552, but the removal wasn't
actually correct - see #1472.
* Fix argument handling in pytest.mark.skip
See #8384
* Raise from None
* Fix test name
* retry writing pytest-of dir when invalid chars are in directory name
* add unit tests for getbasetemp() and changelog
* patch _basetemp & _given_basetemp for testing basetemp()
* Tweak changelog for #8317, tidy up comments
Similarly to #7143, at work we have a project with a custom pytest.Class
subclass, adding an additional argument to the constructor.
All from_parent implementations in pytest accept and forward *kw, except
Class (before this change) and DoctestItem - since I'm not familiar with
doctest support, I've left the latter as-is.
It seems the code that would not install pytest's faulthandler support
if it was already enabled is not really needed at all, and even detrimental
when using `python -X dev -m pytest` to run Python in "dev" mode.
Also simplified the plugin by removing the hook class, now the hooks
will always be active so there's no need to delay the hook definitions anymore.
Fix#8258
The Twisted Logger will return an invalid file descriptor since it is
not backed by an FD. So, let's also forward this to the same code path
as with `pytest-xdist`.
as PyObjMixin is always supposed to be mixed in the mro
before nodes.Node the behavior doesn't change,
but all the typing information carry over to help mypy.
extracted from #8037
It is not very clear why this code exists -- we are not running any
unittest or nose code during collection, and really these frameworks
don't have the concept of collection at all, and just raising these
exceptions at e.g. the module level would cause an error. So unless I'm
missing something, I don't think anyone is using this.
Deprecate it so we can eventually clear up this code and keep unittest
more tightly restricted to its plugin.
The type cannot be constructed directly, but is exported for use in type
annotations, since it is reachable through existing public API.
This also documents `from_call` as public, because at least
pytest-forked uses it, so we must treat it as public already anyway.
Add a new hook , `pytest_markeval_namespace` which should return a dictionary.
This dictionary will be used to augment the "global" variables available to evaluate skipif/xfail/xpass markers.
Pseudo example
``conftest.py``:
.. code-block:: python
def pytest_markeval_namespace():
return {"color": "red"}
``test_func.py``:
.. code-block:: python
@pytest.mark.skipif("color == 'blue'", reason="Color is not red")
def test_func():
assert False
On Windows, os.path.samefile returns false for paths mounted in UNC paths which
point to the same location.
I couldn't reproduce the actual case reported, but looking at the code it seems
this commit should fix the issue.
Fix#7678Fix#8076
Python 3.7 changes the pyc format by adding a flags byte. Even though it
is not necessary for us to match it, it is nice to be able to read pyc
files we emit for debugging the rewriter.
Update our custom pyc files to use that format. We write flags==0
meaning we still use the mtime+size format rather the newer hash format.
In order to allow users to type annotate fixtures they request, the
types need to be imported from the `pytest` namespace. They are/were
always available to import from the `_pytest` namespace, but that is
not guaranteed to be stable.
These types are only exported for the purpose of typing. Specifically,
the following are *not* public:
- Construction (`__init__`)
- Subclassing
- staticmethods and classmethods
We try to combat them being used anyway by:
- Marking the classes as `@final` when possible (already done).
- Not documenting private stuff in the API Reference.
- Using `_`-prefixed names or marking as `:meta private:` for private
stuff.
- Adding a keyword-only `_ispytest=False` to private constructors,
warning if False, and changing pytest itself to pass True. In the
future it will (hopefully) become a hard error.
Hopefully that will be enough.
When --doctest-modules is used, an `__init__.py` file is not a `Package`
but a `DoctestModule`, but some collection code assumed that
`__init__.py` implies a `Package`. That code caused only a single test
to be collected in the scenario in the subject.
Tighten up this check to explicitly check for `Package`. There are
better solutions, but for another time.
Report & test by Nick Gates <nickgatzgates@gmail.com>.
We want to export `pytest.MonkeyPatch` for the purpose of
type-annotating the `monkeypatch` fixture. For other fixtures we export
in this way, we also make direct construction of them (e.g.
`MonkeyPatch()`) private. But unlike the others, `MonkeyPatch` is also
widely used directly already, mostly because the `monkeypatch` fixture
only works in `function` scope (issue #363), but also in other cases. So
making it private will be annoying and we don't offer a decent
replacement yet.
So, let's just make direct construction public & documented.
When pytest was run on a directory containing a recursive symlink it failed
with ELOOP as the library was not able to determine the type of the
direntry:
src/_pytest/main.py:685: in collect
if not direntry.is_file():
E OSError: [Errno 40] Too many levels of symbolic links: '/home/florian/proj/pytest/tests/recursive'
This is fixed by handling ELOOP and other errors in the visit function in
pathlib.py, so the entries whose is_file() call raises an OSError with the
pre-defined list of error numbers will be exluded from the result.
The _ignore_errors function was copied from Lib/pathlib.py of cpython 3.9.
Fixes#7951
* adding --sw-skip shorthand for stepwise skip
* be explicit rather than implicit with default args for stepwise
* add constant for sw cache dir; only register plugin if necessary rather check check activity always;
* use str format; remove unused args in hooks
* assert cache upfront, allow stepwise to have a reference to the cache
* type hinting lf, skip, move literal strings into module constants
* convert parametrized option into a list
* add a sessionfinish hook for stepwise to keep backwards behaviour the same
* add changelog for #7938
* Improve performance of stepwise modifyitems & address PR feedback
* add test for stepwise deselected based on performance enhancements
* Apply suggestions from code review
* delete from items, account for edge case where failed_index = 0
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
In pytester tests, pytest stashes & restores the sys.modules for each
test. So if the test imports a new module, it is initialized anew each
time.
Turns out the readline module isn't multi-init safe, which causes
pytester.spawn to crash or hang. So preserve it as a workaround.
It turns out all autouse fixtures are kept in a global list, and thinned
out for a particular node using a linear scan of the entire list each
time.
Change the list to a dict, and only take the nodes we need.
--lf has an optimization where it skips collecting Modules (python
files) which don't contain failing tests. The optimization works by
getting the paths of all cached failed tests and skipping the collection
of Modules whose path is not included in that list.
In pytest, Package nodes are Module nodes with the fspath being the file
`<package dir>/__init__.py`. Since it's a Module the logic above
triggered for it, and because it's an `__init__.py` file which is
unlikely to have any failing tests in it, it is skipped, which causes
its entire directory to be skipped, including any Modules inside it with
failing tests.
Fix by special-casing Packages to never filter. This means entire
Packages are never filtered, the Modules themselves are always checked.
It is reasonable to consider an optimization which does filter entire
packages bases on parent paths etc. but this wouldn't actually save any
real work so is really not worth it.
Regressed in 6.1.0 in 62e249a1f9.
The `x` is an `str` but is expected to be a `pathlib.Path`. Not caught
by mypy because `config.getini()` returns `Any`.
Fix by just removing the `bestrelpath` call:
- testpaths are always relative to the rootdir, it thus would be very
unusual to specify an absolute path there.
- The code was wrong even before the regression: `py.path.local`'s
`bestrelpath` function expects a `py.path.local`, not an `str`. But it
had some weird `try ... except AttributeError` fallback which just
returns the argument, i.e. it was a no-op. So there is no behavior
change.
- It seems reasonable to me to just print the full path if that's what
the ini specifies.
When switching from py.path.local to pathlib (70f3ad1c1f),
`local.parts(reverse=True)` was translated incorrectly, leading to the
wrong rootdir being determined in some non-trivial cases where parent
directories have config files as well.
This indicates at least for people using type checkers that these
classes are not designed for inheritance and we make no stability
guarantees regarding inheritance of them.
Currently this doesn't show up in the docs. Sphinx does actually support
`@final`, however it only works when imported directly from `typing`,
while we import from `_pytest.compat`.
In the future there might also be a `@sealed` decorator which would
cover some more cases.
For decorated functions, the lineno of the FunctionDef AST node points
to the `def` line, not to the first decorator line. On the other hand,
in code objects, the `co_firstlineno` points to the first decorator
line.
Assertion rewriting inserts some imports to code it rewrites. The
imports are inserted at the lineno of the first statement in the AST. In
turn, the code object compiled from the rewritten AST uses the lineno of
the first statement (which is the first inserted import).
This means that given a module like this,
```py
@foo
@bar
def baz(): pass
```
the lineno of the code object without assertion rewriting
(`--assertion=plain`) is 1, but with assertion rewriting it is 3.
And *this* causes some issues for the exception repr when e.g. the
decorator line is invalid and raises during collection. The code becomes
confused and crashes with
INTERNALERROR> File "_pytest/_code/code.py", line 638, in get_source
INTERNALERROR> lines.append(space_prefix + source.lines[line_index].strip())
INTERNALERROR> IndexError: list index out of range
Fix it by special casing decorators. Maybe there are other cases like
this but off hand I can't think of another Python construct where the
lineno of the item would be after its first line, and this is the only
such issue we have had reported.
Warnings are a central part of Python, so much that Python itself has
command-line and environtment variables to handle warnings.
By moving the concept of warning handling into Config, it becomes natural to
filter warnings issued as early as possible, even before the "_pytest.warnings"
plugin is given a chance to spring into action. This also avoids the weird
coupling between config and the warnings plugin that was required before.
Fix#6681Fix#2891Fix#7620Fix#7626Close#7649
Co-authored-by: Ran Benita <ran@unusedvar.com>
When a Python object (module/class/instance) is collected, for each name
in `obj.__dict__` (and up its MRO) the pytest_pycollect_makeitem hook is
called for potentially creating a node for it.
These Python objects have a bunch of builtin attributes that are
extremely unlikely to be collected. But due to their pervasiveness,
dispatching the hook for them ends up being mildly expensive and also
pollutes PYTEST_DEBUG=1 output and such.
Let's just ignore these attributes.
On the pandas test suite commit 04e9e0afd476b1b8bed930e47bf60e,
collect only, irrelevant lines snipped, about 5% improvement:
Before:
```
51195095 function calls (48844352 primitive calls) in 39.089 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
226602/54 0.145 0.000 38.940 0.721 manager.py:90(_hookexec)
72227 0.285 0.000 20.146 0.000 python.py:424(_makeitem)
72227 0.171 0.000 16.678 0.000 python.py:218(pytest_pycollect_makeitem)
```
After:
```
48410921 function calls (46240870 primitive calls) in 36.950 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
181429/54 0.113 0.000 36.777 0.681 manager.py:90(_hookexec)
27054 0.130 0.000 17.755 0.001 python.py:465(_makeitem)
27054 0.121 0.000 16.219 0.001 python.py:218(pytest_pycollect_makeitem)
```
In ff8b7884e8 NOTSET was changed to a
singleton enum, which ended up unexpectedly triggering a code path in ID
generation which checks for `isinstance(Enum)`.
Add an explicit case for it, which is not too bad anyway.
The `CaptureManager.global_and_fixture_disabled()` context manager (and
`CaptureFixture.disabled()` which calls it) did `suspend(); ...;
resume()` but if the capturing was already suspended, the `resume()`
would resume it when it shouldn't.
This caused caused some messages to be swallowed when `--log-cli` is
used because it uses `global_and_fixture_disabled` when capturing is not
necessarily resumed.
This fixes an issue where pylint complains about missing implementations
of abstract methods in subclasses of `File` which only override
`collect()` (as they should).
It is also cleaner and makes sense, these methods really don't need to
be overridden.
The previous methods defined directly on `FSCollector` and `Package` are
deprecated, to be removed in pytest 7.
See commits e2934c3f8c and
f10ab021e2 for reference.
We barely use it; the couple places that do are not really worth the
extra dependency, I think the code is clearer without it.
Also simplifies one (regular) itertools usage.
Also improves a check and an error message in `pytest.raises`.
Part of the effort to reduce dependency on the py library.
Besides that, py.xml implements its own XML serialization which is
pretty scary.
I tried to keep the code with minimal changes (though it could use some
cleanups). The differences in behavior I have noticed are:
- Attributes in the output are not sorted.
- Some unneeded escaping is no longer performed, for example escaping
`"` to `"` in a text node.
Setting log_level via the CLI or .ini will control the log level of the
report that is dumped upon failure of a test.
If caplog modified the log level during the execution of that test, it
should not impact the level that is displayed upon failure in the
"captured log report" section.
[
ran:
- rebased
- reused handler
- changed store keys also to "caplog_handler_*"
- added changelog
all bugs are mine :)
]
`TerminalWriter`, imported recently from `py`, contains its own
incomplete wcwidth (`char_with`/`get_line_width`) implementation. The
`TerminalReporter` also needs this, but uses the external `wcwidth`
package.
This commit brings the `TerminalWriter` implementation up-to-par with
`wcwidth`, moves to implementation to a new file `_pytest._io.wcwidth`
which is used everywhere, and removes the dependency.
The differences compared to the `wcwidth` package are:
- Normalizes the string before counting.
- Uses Python's `unicodedata` instead of vendored Unicode tables. This
means the data corresponds to the Python's version Unicode version
instead of the `wcwidth`'s package version.
- Apply some optimizations.
pytest.fixture() can be used either as
@pytest.fixture
def func(): ...
or as
@pytest.fixture()
def func(): ...
or (while maybe not intended)
func = pytest.fixture(func)
so it needs to inspect internally whether it got a function in the first
positional argument or not.
Previously, there were was oddity. In the following,
func = pytest.fixture(func, autouse=True)
# OR
func = pytest.fixture(func, parms=['a', 'b'])
The result is as if `func` wasn't passed.
There isn't any reason for this special that I can understand, so remove
it.
The `FDCapture`/`FDCaptureBinary` classes, used by `capfd`/`capfdbinary`
fixtures and the `--capture=fd` option (set by default), redirect FDs
1/2 (stdout/stderr) to a temporary file. To do this, they need to save
the old file by duplicating the FD before redirecting it, to be restored
once finished.
Previously, if this duplicating (`os.dup()`) failed, most likely due to
that FD being invalid, the FD redirection would silently not be done. The
FD capturing also performs python-level redirection (monkeypatching
`sys.stdout`/`sys.stderr`) which would still be done, but direct writes
to the FDs would fail.
This is not great. If pytest is run with `--capture=fd`, or a test is
using `capfd`, it expects writes to the FD to work and be captured,
regardless of external circumstances.
So, instead of disabling FD capturing, keep the redirection to a
temporary file, just don't restore it after closing, because there is
nothing to restore to.
Currently, a bad logging call, e.g.
logger.info('oops', 'first', 2)
triggers the default logging handling, which is printing an error to
stderr but otherwise continuing.
For regular programs this behavior makes sense, a bad log message
shouldn't take down the program. But during tests, it is better not to
skip over such mistakes, but propagate them to the user.
Only filter with known failures, and explicitly keep paths of passed
arguments.
This also displays the "run-last-failure" status before collected files,
and does not update the cache with "--collect-only".
Fixes https://github.com/pytest-dev/pytest/issues/6968.
The `-k '-expr'` syntax is an old alias to `-k 'not expr'`. It's also
not a very convenient to have syntax that start with `-` on the CLI.
Deprecate it and suggest replacing with `not`.
---
The `-k 'expr:'` syntax discards all items until the first match and
keeps all subsequent, e.g. `-k foo` with
test_bar
test_foo
test_baz
results in `test_foo`, `test_baz`. That's a bit weird, so deprecate it
without a replacement. If someone complains we can reconsider or devise
a better alternative.
Running `pytest | head -1` and similar causes an annoying error to be
printed to stderr:
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
(or possibly even a propagating exception in older/other Python versions).
The standard UNIX behavior is to handle the EPIPE silently. To
recommended method to do this in Python is described here:
https://docs.python.org/3/library/signal.html#note-on-sigpipe
It is not appropriate to apply this recommendation to `pytest.main()`,
which is used programmatically for in-process runs. Hence, change
pytest's entrypoint to a new `pytest.console_main()` function, to be
used exclusively by pytest's CLI, and add the SIGPIPE code there.
Fixes#4375.
Also delay calling tearDown() when --pdb is given, so users still have
access to the instance variables (which are usually cleaned up during tearDown())
when debugging.
Fix#6947
Instead of trying to handle unittest-async functions in pytest_pyfunc_call,
let the unittest framework handle them instead.
This lets us remove the hack in pytest_pyfunc_call, with the upside that
we should support any unittest-async based framework.
Also included 'asynctest' as test dependency for py37-twisted, and renamed
'twisted' to 'unittestextras' to better reflect that we install 'twisted' and
'asynctest' now.
This also fixes the problem of cleanUp functions not being properly called
for async functions.
Fix#7110Fix#6924
Previously, the expressions given to the `-m` and `-k` options were
evaluated with `eval`. This causes a few issues:
- Python keywords cannot be used.
- Constants like numbers, None, True, False are not handled correctly.
- Various syntax like numeric operators and `X if Y else Z` is supported
unintentionally.
- `eval()` is somewhat dangerous for arbitrary input.
- Can fail in many ways so requires `except Exception`.
The format we want to support is quite simple, so change to a custom
parser. This fixes the issues above, and gives us full control of the
format, so can be documented comprehensively and even be extended in the
future if we wish.
This function is exposed and kept alive for the oejskit plugin which is
abandoned and no longer works with recent plugins, so let's prepare to
completely remove it.
Co-authored-by: Sylvain MARIE <sylvain.marie@se.com>
Co-authored-by: Ran Benita <ran@unusedvar.com>
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Previously, writing to sys.stdout/stderr in text-mode (e.g.
`print('foo')`) while a `capsysbinary` fixture is active, would crash
with:
/usr/lib/python3.7/contextlib.py:119: in __exit__
next(self.gen)
E TypeError: write() argument must be str, not bytes
This is due to some confusion in the types. The relevant functions are
`snap()` and `writeorg()`. The function `snap()` returns what was
captured, and the return type should be `bytes` for the binary captures
and `str` for the regular ones. The `snap()` return value is eventually
passed to `writeorg()` to be written to the original file, so it's input
type should correspond to `snap()`. But this was incorrect for
`SysCaptureBinary`, which handled it like `str`.
To fix this, be explicit in the `snap()` and `writeorg()`
implementations, also of the other Capture types.
We can't add type annotations yet, because the current inheritance
scheme breaks Liskov Substitution and mypy would complain. To be
refactored later.
Fixes: https://github.com/pytest-dev/pytest/issues/6871
Co-authored-by: Ran Benita (some modifications & commit message)
The commit which added the checks for os.dup a15afb5e48
suggests it was done for Jython. But pytest doesn't support Jython
anymore (Jython is Python 2 only).
Furthermore, it looks like the faulthandler plugin (bundled in pytest
and enabled by default) uses os.dup() unprotected and there have not
been any complaints.
So seems better to just remove these checks, and only add if someone
with a legitimate use case complains.
This started from fixing the test, where `"xdist_strict True"` was used
as a single argument, although you typically would see `["xdist_strict",
"True"]`.
Improves the error message to mention the option that caused the error.
* Use code highlighting if pygments is installed
* Use colorama constants instead of bare ascii codes
Could not find the exact equivalent of 'hl-reset' code using colorama
constants though.
* Refactor ASCII color handling into a fixture
* Revert back to using explicit color codes
* In Python 3.5 skip rest of tests that require ordered markup in colored output
Use suggestion in review and use a subplugin so hooks will only be active
if we enable faulthandler ourselves.
Fix#6575
Co-authored-by: Daniel Hahler <git@thequod.de>
The convention is "assert result is expected". Pytest's error diffs now
reflect this. "-" means that sth. expected is missing in the result and
"+" means that there are unexpected extras in the result.
Fixes: #3333
Plugins specified with ``-p`` are now loaded after internal plugins, which
results in their hooks being called *before* the internal ones.
This makes the ``-p`` behavior consistent with ``PYTEST_PLUGINS``.
* fix/adjust test_disable_plugin_autoload
* adjust test_plugin_loading_order
- refactor _get_main_color/build_summary_stats_line
- factor out property _is_last_item; test_summary_stats: tr._is_last_item
- _write_progress_information_filling_space: remove color arg
- use setter for stats, handling main color
- _get_main_color: skip cache for last item
- Handle random order in test for py35.
Make EncodedFile, used for captured output streams, method .write return
the number of characters written. Add test for captured stderr write.
Fixes#6557.
Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
Tracebacks coming from remote processes crated by the multiprocess module
will contain "RemoteTracebacks" which don't have a 'reprcrash' attribute
Fix#5971
While using pytest-repeat, I noticed the previous implementation is slow
for a large number of duplicate test ids. To optimize, this commit
reduces the amount of data copied and avoids using `in` with List
(unhashable type, and therefore is very slow for many elements).
Now `tox -e docs` will also include the draft changelog for the
next version (locally only).
`CHANGELOG.rst` now only points to the changelog on READTHEDOCS so
sphinx diretives can be used.
Followup to https://github.com/pytest-dev/pytest/pull/6272