- Separate the requesting from the requested.
- Avoid the term "factory", I think most people don't distinguish
between "fixture" and "fixture function" (i.e. "factory") and would
find the term "factory" unfamiliar.
There are two non-optimal things in the current way scope checking is
done:
- It runs on `SubRequest`, but doesn't use the `SubRequest's scope,
which is confusing. Instead it takes `invoking_scope` and
`requested_scope`.
- Because `_check_scope` is only defined on `SubRequest` and not
`TopRequest`, `_compute_fixture_value` first creates the `SubRequest`
only then checks the scope (hence the need for the previous point).
Instead, also define `_check_scope` on `TopRequest` (always valid), and
remove `invoking_scope`, using `self._scope` instead.
Previously, the `obj` of a `TestCaseFunction` (the unittest plugin item
type) was the unbound method. This is unlike regular `Class` where the
`obj` is a bound method to a fresh instance.
This difference necessitated several special cases in in places outside
of the unittest plugin, such as `FixtureDef` and `FixtureRequest`, and
made things a bit harder to understand.
Instead, match how the python plugin does it, including collecting
fixtures from a fresh instance.
The downside is that now this instance for fixture-collection is kept
around in memory, but it's the same as `Class` so nothing new. Users
should only initialize stuff in `setUp`/`setUpClass` and similar
methods, and not in `__init__` which is generally off-limits in
`TestCase` subclasses.
I am not sure why there was a difference in the first place, though I
will say the previous unittest approach is probably the preferable one,
but first let's get consistency.
The check for short paths under Windows via os.path.samefile, introduced in #11936, also found similar tests in symlinked tests in the GH Actions CI.
Fixes#12039.
Co-authored-by: Bruno Oliveira <bruno@soliv.dev>
Previously, if more than one fixture finalizer raised, only the first
was reported, and the other errors were lost.
Use an exception group to report them all. This is similar to the change
we made in node teardowns (in `SetupState`).
As detailed in
https://github.com/pytest-dev/pytest/issues/11475#issuecomment-1937043670,
currently with `--import-mode=importlib` pytest will try to import every
file by using a unique module name, regardless if that module could be
imported using the normal import mechanism without touching `sys.path`.
This has the consequence that non-test modules available in `sys.path`
(via other mechanism, such as being installed into a virtualenv,
PYTHONPATH, etc) would end up being imported as standalone modules,
instead of imported with their expected module names.
To illustrate:
```
.env/
lib/
site-packages/
anndata/
core.py
```
Given `anndata` is installed into the virtual environment, `python -c
"import anndata.core"` works, but pytest with `importlib` mode would
import that module as a standalone module named
`".env.lib.site-packages.anndata.core"`, because importlib module was
designed to import test files which are not reachable from `sys.path`,
but now it is clear that normal modules should be imported using the
standard mechanisms if possible.
Now `imporlib` mode will first try to import the module normally,
without changing `sys.path`, and if that fails it falls back to
importing the module as a standalone module.
This also makes `importlib` respect namespace packages.
This supersedes #11931.
Fix#11475Close#11931
(diff better viewed ignoring whitespace)
In pytest<8, the collection tree for `pyargs` arguments in an invocation
like this:
pytest --collect-only --pyargs pyflakes.test.test_undefined_names
looked like this:
```
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
The pytest 8 collection improvements changed it to this:
```
<Dir pytest>
<Dir .tox>
<Dir venv>
<Dir lib>
<Dir python3.11>
<Dir site-packages>
<Package pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Besides being egregious (and potentially even worse than the above,
going all the way to the root, for system-installed packages, as is
apparently common in CI), this also caused permission errors when trying
to probe some of those intermediate directories.
This change makes `--pyargs` arguments no longer try to add parent
directories to the collection tree according to the `--confcutdir` like
they're regular arguments. Instead, only add the parents that are in the
import path. This now looks like this:
```
<Package .tox/venv/lib/python3.11/site-packages/pyflakes>
<Package test>
<Module test_undefined_names.py>
<UnitTestCase Test>
<TestCaseFunction test_annotationUndefined>
... snipped ...
```
Fix#11904.
pytest allows a fixture to request its own name (directly or
indirectly), in which case the fixture with the same name but one level
up is used.
To know which fixture should be used next, pytest keeps a mutable
item-global dict `_arg2index` which maintains this state. This is not
great:
- Mutable state like this is hard to understand and reason about.
- It is conceptually buggy; the indexing is global (e.g. if requesting
`fix1` and `fix2`, the indexing is shared between them), but actually
different branches of the subrequest tree should not affect each
other.
This is not an issue in practice because pytest keeps a cache of the
fixturedefs it resolved anyway (`_fixture_defs`), but if the cache is
removed it becomes evident.
Instead of the `_arg2index` state, count how many `argname`s deep we are
in the subrequest tree ("the fixture stack") and use that for the index.
This way, no global mutable state and the logic is very localized and
easier to understand.
This is slower, however fixture stacks should not be so deep that this
matters much, I hope.
According to my understanding, this code, which handles obtaining the
relevant fixturedefs when a dynamic `getfixturevalue` is used, has an
optimization where it only grabs fixturedefs that are visible to the
*parent* of the item, instead of the item itself, under the assumption
that a fixturedef can't be visible to a single item, only to a
collector.
Remove this optimization for the following reasons:
- It doesn't save much (one loop iteration in `matchfactories`)
- It slightly complicates the complex fixtures code
- If some plugin wants to make a fixture visible only to a single item,
why not let it?
- In the static case (`getfixtureclosure`), this optimization is not
done (despite the confusing name `parentnode`, it is *not* the parent
node). This is inconsistent.
Allow for the output of test case execution to be controlled independently from the application verbosity level.
`verbosity_test_case` is the new ini setting to adjust this functionality.
Fix#11639
Passing a short path in the command line was causing the matchparts check to fail, because ``Path(short_path) != Path(long_path)``.
Using ``os.path.samefile`` as fallback ensures the comparsion works on Windows when comparing short/long paths.
Fix#11895
In `CallSpec2.setmulti` the `params` and `_arg2scope` dicts are always
set together.
Further, the `get_parametrized_fixture_keys` accesses `_arg2scope` for
all argnames without a check, which I think rules out that the code
protects against plugin shenanigans.
After removing the suppress, adjust the comment and code to make more
sense.
Fix#12021.
Reopens#11706.
This reverts commit 12b9bd5801.
This change caused a bad regression in pytest-xdist:
https://github.com/pytest-dev/pytest-xdist/issues/1024
pytest-xdist necessarily has special handling of `--maxfail` and session
fixture teardown get executed multiple times with the change.
Since I'm not sure how to adapt pytest-xdist myself, revert for now.
I kept the sticky `shouldstop`/`shouldfail` changes as they are good
ideas regardless I think.
Dicts these days preserve order, so the sort is no longer needed to
achieve determinism.
As shown by the `test_dynamic_parametrized_ordering` test, this can
change the ordering of items, but only in equivalent ways (same number
of setups/teardowns per scope), it will just respect the user's given
ordering better (hence `vxlan` items now ordered before `vlan` items
compared to the previous ordering).
Follow up to #12006, let's put some comments clarifying `is_in_confcutdir` semantics, as this is not the first time someone misunderstands it.
Also removed an obsolete comment in `_loadconftestmodules`: we already set the `confcutdir` based on `rootdir`/`initfile` if not explicitly given.
Co-authored-by: Ran Benita <ran@unusedvar.com>
As the comment above it states, we need to keep the comparison simple so mypy can understand it, otherwise it will fail to properly handle this on Windows and will flag ``os.getuid()`` missing.
Test:
`warnings.warn()` expects that its first argument is a `str` or a
`Warning`, but since 9454fc38d3
`pytest.warns()` no longer allows `Warning` instances unless the first
argument the `Warning` was initialized with is a `str`. Furthermore, if
the `Warning` was created without arguments then `pytest.warns()` raises
an unexpected `IndexError`. The new tests reveal the problem.
Fix:
`pytest.warns()` now allows using `warnings.warn()` with a `Warning`
instance, as is required by Python, with one exception. If the warning
used is a `UserWarning` that was created by passing it arguments and the
first argument was not a `str` then `pytest.raises()` still considers
that an error. This is because if an invalid type was used in
`warnings.warn()` then Python creates a `UserWarning` anyways and it
becomes impossible for `pytest` to figure out if that was done
automatically or not.
[ran: rebased on previous commit]
Today `pyproject.toml` is the standard for declaring a Python project root, so seems reasonable to consider it for the ini configuration (and specially `rootdir`) in case we do not find other suitable candidates.
Related to #11311
Previously this would trigger an `AssertionError`.
While it could be considered a bug-fix, but given it now can be relied upon, it is probably better to consider it an improvement.
Fix#11311
Should be around 40% faster according to this simplified small benchmark:
python -m timeit "a=[0, 1, 2, 3, 4];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
200000 loops, best of 5: 1.12 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
500000 loops, best of 5: 651 nsec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((e if i in {0, len(a) -1} else str(e)) for i, e in enumerate(a))"
100000 loops, best of 5: 3.31 usec per loop
python -m timeit "a=[0, 1, 2, 3, 4,5,6,7,8,9,10,11,12,13,14,15,16];b=list((a[0], *(str(e) for e in a[1:-1]), a[-1]))"
200000 loops, best of 5: 1.72 usec per loop
* Improve error message when using @pytest.fixture twice
While obvious in hindsight, this error message confused me. I thought my fixture
function was used in a test function twice, since the wording is ambiguous.
Also, the error does not tell me *which* function is the culprit.
Finally, this adds a test, which wasn't done in
cfd16d0dac where this was originally implemented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fix#11929.
Figured out what's going on. We have the following collection tree:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
```
And the `test_main.py` contains an autouse fixture (`fake_game_ui`) that
`doctest_main` needs in order to run properly. The fixture doesn't run!
It doesn't run because nothing collects the fixtures from (calls
`parsefactories()` on) the `test_main.py` `DoctestModule`.
How come it only started happening with commit
ab63ebb3dc07b89670b96ae97044f48406c44fa0? Turns out it mostly only
worked accidentally. Each `DoctestModule` is also collected as a normal
`Module`, with the `Module` collected after the `DoctestModule`. For
example, if we add a non-doctest test to `test_main.py`, the collection
tree looks like this:
```
<Dir pyspacewar>
<Dir src>
<Package pyspacewar>
<Package tests>
<DoctestModule test_main.py>
<DoctestItem pyspacewar.tests.test_main.doctest_main>
<Module test_main.py>
<Function test_it>
```
Now, `Module` *does* collect fixtures. When autouse fixtures are
collected, they are added to the `_nodeid_autousenames` dict.
Before ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *setup* time. At this point, the `Module` has
collected and so it ended up picking the autouse fixture (this relies on
another "accident", that the `DoctestModule` and `Module` have the same
node ID).
After ab63ebb3dc, `DoctestItem` consults
`_nodeid_autousenames` at *collection* time (= when it's created). At
this point, the `Module` hasn't collected yet, so the autouse fixture is
not picked out.
The fix is simple -- have `DoctestModule.collect()` call
`parsefactories`. From some testing I've done it shouldn't have negative
consequences (I hope).
Currently, `DoctestModule` does `import_path` on its own. This changes
it to use `importtestmodule` as `Module` does. The behavioral changes
are:
- Much better error messages on import errors.
- Handles a few more error cases (see `importtestmodule`). This
technically expands the cover of `--doctest-ignore-import-errors` but
I think it makes sense.
- Considers `pytest_plugins` in the module.
- Populates `self.obj` as properly (without double-imports) as is
expected from a `PyCollector`.
This is also needed for the next commit.
ruff is faster and handle everything we had prior.
isort configuration done based on the indication from
https://github.com/astral-sh/ruff/issues/4670, previousely based on
reorder-python-import (#11896)
flake8-docstrings was a wrapper around pydocstyle (now archived) that
explicitly asks to use ruff in https://github.com/PyCQA/pydocstyle/pull/658.
flake8-typing-import is useful mainly for project that support python 3.7
and the one useful check will be implemented in https://github.com/astral-sh/ruff/issues/2302
We need to keep blacken-doc because ruff does not handle detection
of python code inside .md and .rst. The direct link to the repo is
now used to avoid a redirection.
Manual fixes:
- Lines that became too long
- % formatting that was not done automatically
- type: ignore that were moved around
- noqa of hard to fix issues (UP031 generally)
- fmt: off and fmt: on that is not really identical
between black and ruff
- autofix re-order in pre-commit from faster to slower
Co-authored-by: Ran Benita <ran@unusedvar.com>
- Previously, `getpass.getuser()` would leak an ImportError if the
USERNAME environment variable was not set on Windows because the `pwd`
module cannot be imported.
- Starting in Python 3.13.0a3, it only raises `OSError`.
Fixes#11874
Change our mypy configuration to disallow untyped defs by default, which ensures *new* files added to the code base are fully typed.
To avoid having to type-annotate everything now, add `# mypy: allow-untyped-defs` to files which are not fully type annotated yet.
As we fully type annotate those modules, we can then just remove that directive from the top.
The function currently uses `find_suffixes` which iterates the entire
directory searching for files with the given suffix. In some cases
though, like in pytest's selftest, the directory can get big:
$ ls /tmp/pytest-of-ran/pytest-0/
7686
and iterating it many times can get slow.
This doesn't fix the underlying issue (iterating the directory) but at
least speeds it up a bit by using `os.scandir` instead of
`path.iterdir`. So `make_numbered_dir` is still slow for pytest's
selftests, but reduces ~10s for me.
We have a use case for this in the next commit.
The name can be obtained by using `manager.get_name(plugin)`, however
this is currently O(num plugins) in pluggy, which would be good to
avoid. Besides, it seems generally useful.
This removes one thing that directory collectors need to worry about.
This adds one hook dispatch per `__pycache__` file, but I think it's
worth it for consistency.
Ran Benita
Bruno Oliveira
Bruno Oliveira
mrbean-bremen
Patrick Lannigan
robotherapist
Eric Larson
Ben Leith
Reagan Lee
Eero Vaher
Pierre Sassoulas
Florian Bruhin
whysage
Clément Robert
Russell Martin
Dương Quốc Khánh
clee2000
faph
woutdenolf
Loïc Estève