It seems the code that would not install pytest's faulthandler support
if it was already enabled is not really needed at all, and even detrimental
when using `python -X dev -m pytest` to run Python in "dev" mode.
Also simplified the plugin by removing the hook class, now the hooks
will always be active so there's no need to delay the hook definitions anymore.
Fix#8258
The Twisted Logger will return an invalid file descriptor since it is
not backed by an FD. So, let's also forward this to the same code path
as with `pytest-xdist`.
as PyObjMixin is always supposed to be mixed in the mro
before nodes.Node the behavior doesn't change,
but all the typing information carry over to help mypy.
extracted from #8037
It is not very clear why this code exists -- we are not running any
unittest or nose code during collection, and really these frameworks
don't have the concept of collection at all, and just raising these
exceptions at e.g. the module level would cause an error. So unless I'm
missing something, I don't think anyone is using this.
Deprecate it so we can eventually clear up this code and keep unittest
more tightly restricted to its plugin.
When `pytest.skip()` is called inside a test function, the skip location
should be reported as the line that made the call, however when
`pytest.skip()` is called by the `pytest.mark.skip` and similar
mechanisms, the location should be reported at the item's location,
because the exact location is some irrelevant internal code.
Currently the item-location case is implemented by the caller setting a
boolean key on the item's store and the `skipping` plugin checking it
and fixing up the location if needed. This is really roundabout IMO and
breaks encapsulation.
Instead, allow the caller to specify directly on the skip exception
whether to use the item's location or not. For now, this is entirely
private.
When line_index was a large negative number, get_source failed
on `source.lines[line_index]`.
Use the same dummy Source as with a large positive line_index.
* [pre-commit.ci] pre-commit autoupdate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* manual fixes after configuration update
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Anthony Sottile <asottile@umich.edu>
The type cannot be constructed directly, but is exported for use in type
annotations, since it is reachable through existing public API.
This also documents `from_call` as public, because at least
pytest-forked uses it, so we must treat it as public already anyway.
Add a new hook , `pytest_markeval_namespace` which should return a dictionary.
This dictionary will be used to augment the "global" variables available to evaluate skipif/xfail/xpass markers.
Pseudo example
``conftest.py``:
.. code-block:: python
def pytest_markeval_namespace():
return {"color": "red"}
``test_func.py``:
.. code-block:: python
@pytest.mark.skipif("color == 'blue'", reason="Color is not red")
def test_func():
assert False
On Windows, os.path.samefile returns false for paths mounted in UNC paths which
point to the same location.
I couldn't reproduce the actual case reported, but looking at the code it seems
this commit should fix the issue.
Fix#7678Fix#8076
Explanation: The default handling of these lifetimes is done in
`tmpdir.TempPathFactory.getbasetemp`, which passes `keep=3` to
`pathlib.make_numbered_dir_with_cleanup`.
GH Issue: #8036
* Prefix contextmanagers with module name in doc examples
* Import pytest explicitly for doctests
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
Python 3.7 changes the pyc format by adding a flags byte. Even though it
is not necessary for us to match it, it is nice to be able to read pyc
files we emit for debugging the rewriter.
Update our custom pyc files to use that format. We write flags==0
meaning we still use the mtime+size format rather the newer hash format.
In order to allow users to type annotate fixtures they request, the
types need to be imported from the `pytest` namespace. They are/were
always available to import from the `_pytest` namespace, but that is
not guaranteed to be stable.
These types are only exported for the purpose of typing. Specifically,
the following are *not* public:
- Construction (`__init__`)
- Subclassing
- staticmethods and classmethods
We try to combat them being used anyway by:
- Marking the classes as `@final` when possible (already done).
- Not documenting private stuff in the API Reference.
- Using `_`-prefixed names or marking as `:meta private:` for private
stuff.
- Adding a keyword-only `_ispytest=False` to private constructors,
warning if False, and changing pytest itself to pass True. In the
future it will (hopefully) become a hard error.
Hopefully that will be enough.
An upcoming commit wants to import from `_pytest.pytester` in the public
`pytest` module. This means that `_pytest.pytester` would start to get
imported during import time, which it hasn't up to now -- it was
imported by the plugin loader (if requested). When a plugin is loaded,
it is subjected to assertion rewriting, but only if the module isn't
imported yet, it issues a warning "Module already imported so cannot be
rewritten" and skips the rewriting. So we'd end up with the pytester
plugin not being rewritten, but it wants to be.
Absent better ideas, the solution here is to split the pytester
assertions to their own plugin (which will always only be imported by
the plugin loader) and exclude pytester itself from plugin rewriting.
When --doctest-modules is used, an `__init__.py` file is not a `Package`
but a `DoctestModule`, but some collection code assumed that
`__init__.py` implies a `Package`. That code caused only a single test
to be collected in the scenario in the subject.
Tighten up this check to explicitly check for `Package`. There are
better solutions, but for another time.
Report & test by Nick Gates <nickgatzgates@gmail.com>.
We want to export `pytest.MonkeyPatch` for the purpose of
type-annotating the `monkeypatch` fixture. For other fixtures we export
in this way, we also make direct construction of them (e.g.
`MonkeyPatch()`) private. But unlike the others, `MonkeyPatch` is also
widely used directly already, mostly because the `monkeypatch` fixture
only works in `function` scope (issue #363), but also in other cases. So
making it private will be annoying and we don't offer a decent
replacement yet.
So, let's just make direct construction public & documented.
Don't import `pytest` from within some `_pytest` modules since an
upcoming commit will import from them into `pytest`.
It would have been nice not to have to do it, so that internal plugins
look more like external plugins, but with the existing layout this seems
unavoidable.
When pytest was run on a directory containing a recursive symlink it failed
with ELOOP as the library was not able to determine the type of the
direntry:
src/_pytest/main.py:685: in collect
if not direntry.is_file():
E OSError: [Errno 40] Too many levels of symbolic links: '/home/florian/proj/pytest/tests/recursive'
This is fixed by handling ELOOP and other errors in the visit function in
pathlib.py, so the entries whose is_file() call raises an OSError with the
pre-defined list of error numbers will be exluded from the result.
The _ignore_errors function was copied from Lib/pathlib.py of cpython 3.9.
Fixes#7951
* adding --sw-skip shorthand for stepwise skip
* be explicit rather than implicit with default args for stepwise
* add constant for sw cache dir; only register plugin if necessary rather check check activity always;
* use str format; remove unused args in hooks
* assert cache upfront, allow stepwise to have a reference to the cache
* type hinting lf, skip, move literal strings into module constants
* convert parametrized option into a list
* add a sessionfinish hook for stepwise to keep backwards behaviour the same
* add changelog for #7938
* Improve performance of stepwise modifyitems & address PR feedback
* add test for stepwise deselected based on performance enhancements
* Apply suggestions from code review
* delete from items, account for edge case where failed_index = 0
Co-authored-by: Bruno Oliveira <nicoddemus@gmail.com>
In pytester tests, pytest stashes & restores the sys.modules for each
test. So if the test imports a new module, it is initialized anew each
time.
Turns out the readline module isn't multi-init safe, which causes
pytester.spawn to crash or hang. So preserve it as a workaround.
It turns out all autouse fixtures are kept in a global list, and thinned
out for a particular node using a linear scan of the entire list each
time.
Change the list to a dict, and only take the nodes we need.
ischildnode can be quite hot in some cases involving many fixtures.
However it is always used in a way that the nodeid is constant and the
baseid is iterated. So we can save work by pre-computing the parents of
the nodeid and use a simple containment test.
The `_getautousenames` function has the same stuff open-coded, so change
it to use the new function as well.
Since commit 0f918b1a9d pytest uses auto-generated autouse
pytest fixtures for the xunit fixtures
{setup,teardown}_{module,class,method,function}. All of these fixtures
were given the same name.
Unfortunately, pytest fixture lookup for a name works by grabbing all of
the fixtures globally declared with a name and filtering to only those
which match the specific node. So each xunit-using item iterates over a
list (of fixturedefs) of a size of all previous same-xunit-using items,
i.e. quadratic.
Fixing this properly to use a better data structure is likely to take
some effort, but we can avoid the immediate problem by just using
a different name for each item's autouse fixture, so it only matches
itself.
A benchmark is added to demonstrate the issue. It is still way too slow
after the fix and possibly still quadratic, but for a different reason
which is another matter.
Running --collect-only, before (snipped):
202533232 function calls (201902604 primitive calls) in 86.379 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 85.688 85.688 main.py:320(pytest_collection)
1 0.000 0.000 85.688 85.688 main.py:567(perform_collect)
80557/556 0.021 0.000 85.050 0.153 {method 'extend' of 'list' objects}
85001/15001 0.166 0.000 85.045 0.006 main.py:785(genitems)
10002 0.050 0.000 84.219 0.008 runner.py:455(collect_one_node)
10002 0.049 0.000 83.763 0.008 runner.py:340(pytest_make_collect_report)
10002 0.079 0.000 83.668 0.008 runner.py:298(from_call)
10002 0.019 0.000 83.541 0.008 runner.py:341(<lambda>)
5001 0.184 0.000 81.922 0.016 python.py:412(collect)
5000 0.020 0.000 81.072 0.016 python.py:842(collect)
30003 0.118 0.000 78.478 0.003 python.py:218(pytest_pycollect_makeitem)
30000 0.190 0.000 77.957 0.003 python.py:450(_genfunctions)
40001 0.081 0.000 76.664 0.002 nodes.py:183(from_parent)
30000 0.087 0.000 76.629 0.003 python.py:1595(from_parent)
40002 0.092 0.000 76.583 0.002 nodes.py:102(_create)
30000 0.305 0.000 76.404 0.003 python.py:1533(__init__)
15000 0.132 0.000 74.765 0.005 fixtures.py:1439(getfixtureinfo)
15000 0.165 0.000 73.664 0.005 fixtures.py:1492(getfixtureclosure)
15000 0.044 0.000 57.584 0.004 fixtures.py:1653(getfixturedefs)
30000 18.840 0.001 57.540 0.002 fixtures.py:1668(_matchfactories)
37507500 31.352 0.000 38.700 0.000 nodes.py:76(ischildnode)
15000 10.464 0.001 15.806 0.001 fixtures.py:1479(_getautousenames)
112930587/112910019 7.333 0.000 7.339 0.000 {built-in method builtins.len}
After:
51890333 function calls (51259706 primitive calls) in 27.306 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 26.783 26.783 main.py:320(pytest_collection)
1 0.000 0.000 26.783 26.783 main.py:567(perform_collect)
80557/556 0.020 0.000 26.108 0.047 {method 'extend' of 'list' objects}
85001/15001 0.151 0.000 26.103 0.002 main.py:785(genitems)
10002 0.047 0.000 25.324 0.003 runner.py:455(collect_one_node)
10002 0.045 0.000 24.888 0.002 runner.py:340(pytest_make_collect_report)
10002 0.069 0.000 24.805 0.002 runner.py:298(from_call)
10002 0.017 0.000 24.690 0.002 runner.py:341(<lambda>)
5001 0.168 0.000 23.150 0.005 python.py:412(collect)
5000 0.019 0.000 22.223 0.004 python.py:858(collect)
30003 0.101 0.000 19.818 0.001 python.py:218(pytest_pycollect_makeitem)
30000 0.161 0.000 19.368 0.001 python.py:450(_genfunctions)
30000 0.302 0.000 18.236 0.001 python.py:1611(from_parent)
40001 0.084 0.000 18.051 0.000 nodes.py:183(from_parent)
40002 0.116 0.000 17.967 0.000 nodes.py:102(_create)
30000 0.308 0.000 17.770 0.001 python.py:1549(__init__)
15000 0.117 0.000 16.111 0.001 fixtures.py:1439(getfixtureinfo)
15000 0.134 0.000 15.135 0.001 fixtures.py:1492(getfixtureclosure)
15000 9.320 0.001 14.738 0.001 fixtures.py:1479(_getautousenames)
--lf has an optimization where it skips collecting Modules (python
files) which don't contain failing tests. The optimization works by
getting the paths of all cached failed tests and skipping the collection
of Modules whose path is not included in that list.
In pytest, Package nodes are Module nodes with the fspath being the file
`<package dir>/__init__.py`. Since it's a Module the logic above
triggered for it, and because it's an `__init__.py` file which is
unlikely to have any failing tests in it, it is skipped, which causes
its entire directory to be skipped, including any Modules inside it with
failing tests.
Fix by special-casing Packages to never filter. This means entire
Packages are never filtered, the Modules themselves are always checked.
It is reasonable to consider an optimization which does filter entire
packages bases on parent paths etc. but this wouldn't actually save any
real work so is really not worth it.
Regressed in 6.1.0 in 62e249a1f9.
The `x` is an `str` but is expected to be a `pathlib.Path`. Not caught
by mypy because `config.getini()` returns `Any`.
Fix by just removing the `bestrelpath` call:
- testpaths are always relative to the rootdir, it thus would be very
unusual to specify an absolute path there.
- The code was wrong even before the regression: `py.path.local`'s
`bestrelpath` function expects a `py.path.local`, not an `str`. But it
had some weird `try ... except AttributeError` fallback which just
returns the argument, i.e. it was a no-op. So there is no behavior
change.
- It seems reasonable to me to just print the full path if that's what
the ini specifies.
When switching from py.path.local to pathlib (70f3ad1c1f),
`local.parts(reverse=True)` was translated incorrectly, leading to the
wrong rootdir being determined in some non-trivial cases where parent
directories have config files as well.
This indicates at least for people using type checkers that these
classes are not designed for inheritance and we make no stability
guarantees regarding inheritance of them.
Currently this doesn't show up in the docs. Sphinx does actually support
`@final`, however it only works when imported directly from `typing`,
while we import from `_pytest.compat`.
In the future there might also be a `@sealed` decorator which would
cover some more cases.
For decorated functions, the lineno of the FunctionDef AST node points
to the `def` line, not to the first decorator line. On the other hand,
in code objects, the `co_firstlineno` points to the first decorator
line.
Assertion rewriting inserts some imports to code it rewrites. The
imports are inserted at the lineno of the first statement in the AST. In
turn, the code object compiled from the rewritten AST uses the lineno of
the first statement (which is the first inserted import).
This means that given a module like this,
```py
@foo
@bar
def baz(): pass
```
the lineno of the code object without assertion rewriting
(`--assertion=plain`) is 1, but with assertion rewriting it is 3.
And *this* causes some issues for the exception repr when e.g. the
decorator line is invalid and raises during collection. The code becomes
confused and crashes with
INTERNALERROR> File "_pytest/_code/code.py", line 638, in get_source
INTERNALERROR> lines.append(space_prefix + source.lines[line_index].strip())
INTERNALERROR> IndexError: list index out of range
Fix it by special casing decorators. Maybe there are other cases like
this but off hand I can't think of another Python construct where the
lineno of the item would be after its first line, and this is the only
such issue we have had reported.
When a name is exported from `pytest`, prefer to refer to it by that
rather than its `_pytest` import path. It is shorter and more
appropriate in user-facing documentation (although that's not really
visible).
Our plan is to expose more names for typing purposes, in which can this
could be more comprehensive.
Warnings are a central part of Python, so much that Python itself has
command-line and environtment variables to handle warnings.
By moving the concept of warning handling into Config, it becomes natural to
filter warnings issued as early as possible, even before the "_pytest.warnings"
plugin is given a chance to spring into action. This also avoids the weird
coupling between config and the warnings plugin that was required before.
Fix#6681Fix#2891Fix#7620Fix#7626Close#7649
Co-authored-by: Ran Benita <ran@unusedvar.com>
Inline `_makeitem()` so that `self.ihook` (which is moderately
expensive) can be called only once.
Note: the removed test "test_makeitem_non_underscore" comes from an old
behavior of skipping names that start with `_` which has since been
generalized, making the test no longer relevant.
When a Python object (module/class/instance) is collected, for each name
in `obj.__dict__` (and up its MRO) the pytest_pycollect_makeitem hook is
called for potentially creating a node for it.
These Python objects have a bunch of builtin attributes that are
extremely unlikely to be collected. But due to their pervasiveness,
dispatching the hook for them ends up being mildly expensive and also
pollutes PYTEST_DEBUG=1 output and such.
Let's just ignore these attributes.
On the pandas test suite commit 04e9e0afd476b1b8bed930e47bf60e,
collect only, irrelevant lines snipped, about 5% improvement:
Before:
```
51195095 function calls (48844352 primitive calls) in 39.089 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
226602/54 0.145 0.000 38.940 0.721 manager.py:90(_hookexec)
72227 0.285 0.000 20.146 0.000 python.py:424(_makeitem)
72227 0.171 0.000 16.678 0.000 python.py:218(pytest_pycollect_makeitem)
```
After:
```
48410921 function calls (46240870 primitive calls) in 36.950 seconds
ncalls tottime percall cumtime percall filename:lineno(function)
181429/54 0.113 0.000 36.777 0.681 manager.py:90(_hookexec)
27054 0.130 0.000 17.755 0.001 python.py:465(_makeitem)
27054 0.121 0.000 16.219 0.001 python.py:218(pytest_pycollect_makeitem)
```
In ff8b7884e8 NOTSET was changed to a
singleton enum, which ended up unexpectedly triggering a code path in ID
generation which checks for `isinstance(Enum)`.
Add an explicit case for it, which is not too bad anyway.
This is a more sensible interface for matchnodes.
This also fixes a sort-of bug where a recursive call to matchnodes
raises NoMatch which would terminate the entire tree, even if other
branches may find a match. Though I don't think it's actually possible.
Ran Benita
Hong Xu
Bruno Oliveira
Andreas Motl
Ronny Pfannschmidt
Anton
Petr Viktorin
bengartner
pre-commit-ci[bot]
Christophe Bedard
antonblr
Jakob van Santen
Pedro Algarvio
Katarzyna
Matthew Hughes
Jürgen Gmach
Simon K
Maximilian Cosmo Sitter
Tim Hoffmann
Petter Strandmark
Adam Johnson
Hugo Martins
Sanket Duthade
Cserna Zsolt
Karthikeyan Singaravelan
Vasilis Gerakaris
Hugo van Kemenade
Tanvi Mehta
Tanvi Mehta
Kyle Altendorf
Anthony Sottile
Manuel Mariñez
Albert Villanova del Moral
Sorin Sbarnea
Florian Bruhin