Commit Graph

3719 Commits

Author SHA1 Message Date
Daniel Hahler 81a9df6ed1
tests: fix flaky test_timeout (#6773)
Use a longer timeout on CI - seen a failure with it with GHA (Windows).

Ref: a4554e666a
2020-02-21 12:52:31 +01:00
Bruno Oliveira 435ad221f9 Deprecate TerminalReporter.writer
Fix #6779
2020-02-20 21:04:49 -03:00
Bruno Oliveira c91abe48ba Assorted improvements following up #6658 2020-02-20 18:51:41 -03:00
Daniel Hahler d1b50526fa
tests: harden some UsageError tests (matching the error msg) (#6775) 2020-02-20 13:29:59 +01:00
Daniel Hahler fb16d3e27a
capture: revisit/fix __repr__, define _in_suspended (#6749) 2020-02-20 00:51:57 +01:00
Daniel Hahler 2b13a9b95d
Use TYPE_CHECKING (#6771) 2020-02-19 22:07:54 +01:00
Daniel Hahler 1b30514783
LFPlugin: use sub-plugins to deselect during collection (#6448)
Fixes https://github.com/pytest-dev/pytest/issues/5301.

Refactor/steps:

- use var
- harden test_lastfailed_usecase
- harden test_failedfirst_order
- revisit last_failed_paths
- harden test_lastfailed_with_known_failures_not_being_selected
2020-02-19 21:33:03 +01:00
Ran Benita af2b0e1174
Merge pull request #6758 from bluetech/outcome-exception-callable-2
Use a hack to make typing of pytest.fail.Exception & co work
2020-02-19 22:08:21 +02:00
Daniel Hahler f95c7f5803
doctest: handle any OutcomeException (#6669)
Fixes using `pytest.xfail()` and `pytest.importorskip()` in doctests.

Ref: https://github.com/pytest-dev/pytest/issues/310
2020-02-19 13:16:37 +01:00
Ran Benita 24dcc76495 Use a hack to make typing of pytest.fail.Exception & co work
Mypy currently is unable to handle assigning attributes on function:
https://github.com/python/mypy/issues/2087.
pytest uses this for the outcome exceptions -- `pytest.fail.Exception`,
`pytest.exit.Exception` etc, and this is the canonical name by which they
are referred.

Initially we started working around this with type: ignores, and later
by switching e.g. `pytest.fail.Exception` with the direct exception
`Failed`. But this causes a lot of churn and is not as nice. And I also
found that some code relies on it, in skipping.py:

    def pytest_configure(config):
        if config.option.runxfail:
            # yay a hack
            import pytest

            old = pytest.xfail
            config._cleanup.append(lambda: setattr(pytest, "xfail", old))

            def nop(*args, **kwargs):
                pass

            nop.Exception = xfail.Exception
            setattr(pytest, "xfail", nop)
        ...

So it seems better to support it. Use a hack to make it work. The rest
of the commit rolls back all of the workarounds we added up to now.

`pytest.raises.Exception` also exists, but it's not used much so I kept
it as-is for now.

Hopefully in the future mypy supports this and this ugliness can be
removed.
2020-02-18 23:17:27 +02:00
Daniel Hahler 442f7a7706
Do not use fixed line number with test_cache_failure_warns (#6748)
It was not previously checking for the line number also (02aa8ad), and this is
obviously wrong (affected by changes to the file).
2020-02-18 09:27:33 +01:00
Ran Benita 7b8968ff80
Merge pull request #6735 from bluetech/metafunc-annotate
Type annotate Metafunc
2020-02-15 23:32:14 +02:00
Daniel Hahler e872532d0c Merge branch 'master' into term-color
Conflicts:
	src/_pytest/terminal.py
	testing/test_debugging.py
	testing/test_terminal.py
2020-02-15 18:46:29 +01:00
Ran Benita 5945c3fe88 Type annotate Metafunc 2020-02-15 17:13:18 +02:00
Daniel Hahler 2b5adc88a7 Move test_issue4445_cacheprovider_set into test_cache_failure_warns
Would need to be adjusted anyway non-trivially, and we can just harden
`test_cache_failure_warns` instead.
2020-02-15 02:01:22 +01:00
Daniel Hahler 02aa8adae1 cacheprovider: use warnings directly
Allows for filtering of PytestCacheWarning.

Using `_issue_warning_captured` is not necessary here, and was probably
only used because the cacheprovider misses warnings during
`pytest_sessionfinish`, which is also fixed here.

I think the usage of `_issue_warning_captured` can be removed/reduced
further, but also that this is good enough for now.

Ref: https://github.com/pytest-dev/pytest/issues/6681.
2020-02-15 01:36:57 +01:00
Daniel Hahler 67e69a7e49
tests: harden test_xdist_verbose (#6700) 2020-02-15 01:22:01 +01:00
Daniel Hahler 4b70ba2c21
tests: harden test_better_reporting_on_conftest_load_failure (#6713) 2020-02-14 17:00:01 +01:00
Daniel Hahler de3353aac1
test_load_initial_conftest_last_ordering: handle testing package (#6706) 2020-02-14 16:58:17 +01:00
Daniel Hahler d89b5057ca
assertrepr_compare: provide more info (location) with exceptions (#6728) 2020-02-14 02:17:05 +01:00
Daniel Hahler 83137c89e9
tests: test_unicode_plus_minus: use unicode sign directly (#6727)
Was globbed for Python 2 before (57c448991).
2020-02-14 02:16:25 +01:00
Bruno Oliveira e6ea9edffe
Merge pull request #6673 from sscherfke/features
Reverse / fix meaning of "+/-" in error diffs
2020-02-12 15:05:38 -03:00
Daniel Hahler b7ad4c2bed
_pformat_dispatch: pass through args (#6715) 2020-02-12 16:07:57 +01:00
Ran Benita c9f9664336 Merge branch 'features' into master
The features branch is no more. Development of features is now also done
on master.

See https://github.com/pytest-dev/pytest/pull/6571.
2020-02-12 13:50:02 +02:00
Bruno Oliveira 4209ad6fca
Use code highlighting if pygments is installed (#6658)
* Use code highlighting if pygments is installed

* Use colorama constants instead of bare ascii codes

Could not find the exact equivalent of 'hl-reset' code using colorama
constants though.

* Refactor ASCII color handling into a fixture

* Revert back to using explicit color codes

* In Python 3.5 skip rest of tests that require ordered markup in colored output
2020-02-12 08:32:37 -03:00
Bruno Oliveira 3ea74310d7
Fix crash when faulthandler starts initialized (#6598)
Use suggestion in review and use a subplugin so hooks will only be active
if we enable faulthandler ourselves.

Fix #6575

Co-authored-by: Daniel Hahler <git@thequod.de>
2020-02-12 11:06:20 +01:00
Bruno Oliveira 78baa7b575 Merge remote-tracking branch 'upstream/master' into mm
Conflicts:
	src/_pytest/main.py
	src/_pytest/mark/structures.py
	src/_pytest/python.py
	testing/test_main.py
	testing/test_parseopt.py
2020-02-11 19:22:28 -03:00
Ran Benita d33da078a8 Move ExitCode's definition from _pytest.main to _pytest.config
ExitCode is used in several internal modules and hooks and so with type
annotations added, needs to be imported a lot.

_pytest.main, being the entry point, generally sits at the top of the
import tree.

So, it's not great to have ExitCode defined in _pytest.main, because it
will cause a lot of import cycles once type annotations are added (in
fact there is already one, which this change removes).

Move it to _pytest.config instead.

_pytest.main still imports ExitCode, so importing from there still
works, although external users should really be importing from `pytest`.
2020-02-10 23:55:06 +02:00
Stefan Scherfke d59adc61f9 Reverse / fix meaning of "+/-" in error diffs
The convention is "assert result is expected".  Pytest's error diffs now
reflect this. "-" means that sth. expected is missing in the result and
"+" means that there are unexpected extras in the result.

Fixes: #3333
2020-02-10 21:12:37 +01:00
Daniel Hahler 449290406c test_argcomplete: remove usage of `distutils.spawn` (#6703)
Fixes collection error with Python 3.5.3 (Travis):

    testing/test_parseopt.py:2: in <module>
        import distutils.spawn
    .tox/py35-coverage/lib/python3.5/distutils/__init__.py:4: in <module>
        import imp
    .tox/py35-coverage/lib/python3.5/imp.py:33: in <module>
        PendingDeprecationWarning, stacklevel=2)
    E   PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

Build log: https://travis-ci.org/blueyed/pytest/builds/648305304
2020-02-10 11:52:19 +01:00
Daniel Hahler b4ace46c42
capture: cleanup item fixture handling (#6663)
This started by looking at how to get the current test item in general,
and then I noticed that it is not necessary for the capture plugin to
track it manually in the first place.
2020-02-07 19:23:37 +01:00
Daniel Hahler a8fc056aad
Handle `Exit` exception in `pytest_sessionfinish` (#6660) 2020-02-07 00:40:10 +01:00
Vladyslav Rachek 9e262038c8
[parametrize] enforce explicit argnames declaration (#6330)
Every argname used in `parametrize` either must
be declared explicitly in the python test function, or via
`indirect` list

Fix #5712
2020-02-06 20:20:25 -03:00
Daniel Hahler 39d9f7cff5
pytester: LineMatcher: typing, docs, consecutive line matching (#6653) 2020-02-04 22:47:18 +01:00
Daniel Hahler 632800add5
internal: clean up getfslineno (#6656) 2020-02-04 22:46:00 +01:00
Daniel Hahler bc494661ad
Remove testing/test_modimport.py (#6666) 2020-02-04 08:26:40 +01:00
Daniel Hahler 4316fe8a92
testing/conftest.py: testdir: set PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 (#6655)
Fixes https://github.com/pytest-dev/pytest/pull/4518.
2020-02-04 02:59:20 +01:00
Daniel Hahler aa0328782f
assertion: save/restore hooks on item (#6646) 2020-02-04 02:56:23 +01:00
Daniel Hahler 61f2a26675 Code/getfslineno: keep empty co_filename
Previously this would be turned via `py.path.local("")` into the current
working directory.

This appears to be what `fspath = fn and py.path.local(fn) or None`
tries to avoid in `getfslineno`'s `TypeError` handling already, if
`Code` would raise it.
2020-02-03 19:09:08 +01:00
Daniel Hahler 75714ee707
pluginmanager.consider_preparse: add exclude_only kwarg (#6443)
Plugins specified with ``-p`` are now loaded after internal plugins, which
results in their hooks being called *before* the internal ones.

This makes the ``-p`` behavior consistent with ``PYTEST_PLUGINS``.

* fix/adjust test_disable_plugin_autoload
* adjust test_plugin_loading_order
2020-02-03 14:10:54 +01:00
Daniel Hahler abffd16ce6 Keep (revisited) comment from https://github.com/pytest-dev/pytest/commit/4d31ea831 2020-02-03 14:04:16 +01:00
Daniel Hahler fb289667e3 Remove testing/test_modimport.py
testing/test_meta.py ensures this already as a side effect
(+ tests a few more (`__init__.py` files) and should have been
combined with it right away [1].

1: https://github.com/pytest-dev/pytest/pull/4510#discussion_r289123446

Ref: https://github.com/pytest-dev/pytest/commit/eaa05531e
Ref: https://github.com/pytest-dev/pytest/commit/4d31ea831
2020-02-03 13:58:32 +01:00
Daniel Hahler 99d162e44a Handle `Exit` exception in `pytest_sessionfinish`
Similar to a7268aa (https://github.com/pytest-dev/pytest/pull/6258).
2020-02-02 23:13:23 +01:00
Daniel Hahler 5256542ea4 pytester.LineMatcher: add support for matching lines consecutively 2020-02-01 23:48:58 +01:00
Daniel Hahler 2681b0aed7 typing: pytester: LineMatcher 2020-02-01 23:48:57 +01:00
Daniel Hahler b10ab0211c Use TypeError instead of AssertionError for no sequence
Improve/extends tests.
2020-02-01 23:48:57 +01:00
Daniel Hahler 09a0e45492 testing/test_pytester.py: cosmetics 2020-02-01 23:48:57 +01:00
Daniel Hahler b5b6e051ed Merge master into features 2020-01-30 02:29:02 +01:00
Daniel Hahler 78eddcb5b1 tests: move test_getfslineno back
Reverts https://github.com/pytest-dev/pytest/pull/6610.

The tested `getfslineno` is `src/_pytest/_code/source.py` actually,
exported via `src/_pytest/_code/__init__.py`.
I've confused it with the one in `src/_pytest/compat.py` apparently.
2020-01-29 23:47:39 +01:00
Bruno Oliveira 64ab68ff0a
Fix 6341 disallow session config in fromparent (#6387)
Fix 6341 disallow session config in fromparent
2020-01-29 19:21:02 -03:00
Daniel Hahler 3dbc61dd80 tests: test_code: improve/clarify imports 2020-01-29 19:44:37 +01:00
Daniel Hahler b42938421e
Merge pull request #6610 from blueyed/tests-move-test_getfslineno
tests: move test_getfslineno
2020-01-29 19:43:28 +01:00
Daniel Hahler 87fecce77b
Merge pull request #6600 from blueyed/harden-test_teardown_many_verbose
tests: harden test_teardown_many_verbose
2020-01-29 18:56:10 +01:00
Anthony Sottile 595d62bc3e
Merge pull request #6607 from asottile/empty_string_parametrize_nodeid
Fix node ids which contain a parametrized empty-string variable
2020-01-29 08:42:04 -08:00
Daniel Hahler 55e5817570 Merge master into features 2020-01-29 03:07:35 +01:00
Daniel Hahler 3f4b8d3aec test_code: improve coverage 2020-01-29 02:54:12 +01:00
Daniel Hahler a3f482ceba tests: move test_getfslineno
It should be in `test_code` when testing `_pytest._code.getfslineno`,
not to be confused with `_pytest._code.source.getfslineno`.

Adds an extra assert (via https://github.com/pytest-dev/pytest/pull/6590).
2020-01-29 01:26:10 +01:00
Daniel Hahler 3ccf2a5e61
Merge pull request #6524 from blueyed/reportchars-default
terminal: default to `fE` with `-r` (reportchars)
2020-01-29 01:00:41 +01:00
Daniel Hahler 8e1d59a8dd
Merge pull request #6579 from blueyed/pytester-makefile-joins-abspath
pytester: test for _makefile joining an absolute path
2020-01-29 00:58:43 +01:00
Bruno Oliveira d282424589
Fix unguarded `==` comparison in fixtures. (#6541)
Fix unguarded `==` comparison in fixtures.
2020-01-28 20:54:14 -03:00
Daniel Hahler ddaa5d88ac terminal: default to `fE` with `-r` (reportchars)
Adds handling of `N` to reset `reportchars`, which can be used to get
the old behavior (`-rN`), and also allows for an alternative to
`--disable-warnings` (https://github.com/pytest-dev/pytest/issues/5066),
since `w` was included by default (without `--disable-warnings`).

Fixes https://github.com/pytest-dev/pytest/issues/6454
2020-01-29 00:33:15 +01:00
Anthony Sottile abd5fc80e8 Fix node ids which contain a parametrized empty-string variable 2020-01-28 13:27:54 -08:00
Holger Kohr 80d4dd6f0b Replace `==` with `is` for comparison of cache keys
Closes #6497
2020-01-28 18:05:53 -03:00
Daniel Hahler 1cf9e68dbc tests: cover absolute path handling in _compute_fixture_value 2020-01-28 18:53:28 +01:00
Daniel Hahler b01e379428 tests: harden test_teardown_many_verbose 2020-01-28 16:37:18 +01:00
Daniel Hahler 30922ee694 Merge master into features 2020-01-28 01:40:14 +01:00
Daniel Hahler ad0f4f0ac0 tests: cover collect_by_name with non-existing 2020-01-28 00:41:46 +01:00
Daniel Hahler 20b66e60c0
Merge pull request #6566 from blueyed/rm-encodedfile-writelines
Fix `EncodedFile.writelines`
2020-01-27 22:58:31 +01:00
Daniel Hahler c2980eb80f pytester: test for _makefile joining an absolute path
Ref: https://github.com/pytest-dev/pytest/pull/6578#discussion_r371035867
2020-01-27 01:00:55 +01:00
Daniel Hahler 40758e86ca tests: add test_via_exec
Via https://github.com/pytest-dev/pytest/issues/6574.
2020-01-26 23:04:18 +01:00
Daniel Hahler d678d380cb typing: tests: tmpfile 2020-01-25 19:21:19 +01:00
Daniel Hahler 3f8f395210 typing: EncodedFile 2020-01-25 19:20:48 +01:00
Daniel Hahler 778d4364fa tests: test_collection_collect_only_live_logging: allow for 1s
Might be slow on CI.

Ref: https://github.com/pytest-dev/pytest/pull/6570/checks?check_run_id=408752475#step:6:109
2020-01-25 18:14:49 +01:00
Daniel Hahler 039d582b52 Fix `EncodedFile.writelines`
This is implemented by the underlying stream already, which additionally
checks if the stream is not closed, and calls `write` per line.

Ref/via: https://github.com/pytest-dev/pytest/pull/6558#issuecomment-578210807
2020-01-25 18:06:50 +01:00
Daniel Hahler 510be29db8
Merge pull request #6534 from blueyed/test_plugin_loading_order
tests: add test_plugin_loading_order
2020-01-25 16:20:06 +01:00
Daniel Hahler 1e3bc1814d typing for test_summary_stats 2020-01-25 15:09:02 +01:00
Daniel Hahler 57512aa997 _get_main_color: no yellow ("boring") for non-last item
- refactor _get_main_color/build_summary_stats_line
- factor out property _is_last_item; test_summary_stats: tr._is_last_item
- _write_progress_information_filling_space: remove color arg
- use setter for stats, handling main color
- _get_main_color: skip cache for last item
- Handle random order in test for py35.
2020-01-25 15:09:02 +01:00
Daniel Hahler c51173d426 Merge master into features 2020-01-25 14:18:02 +01:00
Ran Benita a76bc64c54
Merge pull request #6547 from bluetech/session-initialparts
Refactor Session._initialparts to have a more explicit type
2020-01-25 14:30:26 +02:00
Ran Benita dd5c2b22bd Refactor Session._initialparts to have a more explicit type
Previously, _initialparts was a list whose first item was a
`py.path.local` and the rest were `str`s. This is not something that
mypy is capable of modeling. The type `List[Union[str, py.path.local]]`
is too broad and would require asserts for every access.

Instead, make each item a `Tuple[py.path.local, List[str]]`. This way
the structure is clear and the types are accurate.

To make sure any users who might have been accessing this (private)
field will not break silently, change the name to _initial_parts.
2020-01-25 13:57:49 +02:00
Daniel Hahler 6f2943c7b3
Merge pull request #6558 from gavento/patch-1
Make EncodedFile.write() return the return value from inner write()
2020-01-25 11:04:12 +01:00
Tomáš Gavenčiak 5e15c86cc6 Fix EncodedFile.write return value
Make EncodedFile, used for captured output streams, method .write return
the number of characters written. Add test for captured stderr write.
Fixes #6557.

Co-Authored-By: Bruno Oliveira <nicoddemus@gmail.com>
2020-01-25 10:36:23 +01:00
Daniel Hahler 09bdbffbde Merge master into features
Conflicts:
	src/_pytest/_code/code.py
	src/_pytest/main.py
2020-01-24 23:44:50 +01:00
Daniel Hahler 09ab5fd7e9
Merge pull request #6529 from blueyed/fix-test_repr_traceback_with_invalid_cwd
tests: fix test_repr_traceback_with_invalid_cwd
2020-01-24 23:41:31 +01:00
Daniel Hahler 192d3adda3 tests: add test_fixture_arg_ordering
This is a regression test for part of
https://github.com/pytest-dev/pytest/issues/6492, testing one of the
fixes in https://github.com/pytest-dev/pytest/pull/6551.
2020-01-24 17:48:08 -03:00
Daniel Hahler 79ae86cc3f tests: fix test_repr_traceback_with_invalid_cwd
This never worked as expected (since a912d3745), and only py38-windows
triggered the mocked `os.getcwd` unintentionally, via `inspect`.
2020-01-24 12:52:12 +01:00
Daniel Hahler 03bc8aba4e config: typing for create_terminal_writer, re-export TerminalWriter
This also imports `TerminalWriter` explicitly via `_pytest._io`,
allowing for easier extending / replacing it.
2020-01-23 14:09:37 +01:00
Daniel Hahler 6b13379f37
Merge pull request #6521 from blueyed/harden-nose-raises
tests: improve test for `nose.raises`
2020-01-23 13:42:14 +01:00
Daniel Hahler 00097df5cd tests: add test_plugin_loading_order
Ref: https://github.com/pytest-dev/pytest/pull/6443
2020-01-23 11:57:12 +01:00
Daniel Hahler e7444bbd5e tests: remove unnecessary `-rw` option
Warnings are enabled by default, which is tested by `test_getreportopt`.
2020-01-23 11:37:19 +01:00
Daniel Hahler 252eae5bc8 tests: fix/harden test_record_property
`-rv` is not a recognized reportchar.  Probably `-v` was meant, but is
not necessary to check that there are no warnings.

Followup to 2018cf12b (https://github.com/pytest-dev/pytest/pull/3360).
2020-01-23 10:49:59 +01:00
Daniel Hahler 1350c601dc Node.location: handle str with _node_location_to_relpath 2020-01-23 10:45:31 +01:00
Daniel Hahler 8fa57c8384 tests: improve test for `nose.raises`
This should probably get transferred into a `pytest.fail` really, but
tests/documents the current behavior.
2020-01-23 10:45:26 +01:00
Daniel Hahler aca1723d45 Merge master into features 2020-01-22 19:18:13 +01:00
Daniel Hahler 85df6bbe26
Merge pull request #6536 from blueyed/fix-test_cwd_snapshot
tests: fix test_cwd_snapshot
2020-01-22 16:46:04 +01:00
Daniel Hahler d878d9d4d5 tests: use NotImplementedError with uncovered code 2020-01-22 15:38:12 +01:00
Bruno Oliveira 93b74d28d2 Merge remote-tracking branch 'upstream/master' into mm
Conflicts:
 * 	src/_pytest/_code/code.py
 * 	src/_pytest/main.py
 * 	testing/python/metafunc.py
 * 	testing/test_parseopt.py
 * 	testing/test_pytester.py
2020-01-22 11:03:45 -03:00
Daniel Hahler b8787b8732 tests: fix test_cwd_snapshot
Without restoring the cwd, successive tests might fail to parse the
config (via `_pytest.config._prepareconfig()`, for when `--lsof` is
used).

And it is good practice to restore the cwd in any case anyway.
2020-01-22 14:34:11 +01:00
Daniel Hahler 8f5fd537d8 --cache-clear: add test for keeping non-supporting files
Ref: https://github.com/pytest-dev/pytest/pull/6296
2020-01-21 22:16:56 +01:00
Ran Benita fb99b5c66e Revert "Fix type errors after adding types to the `py` dependency"
This reverts commit 930a158a6a.

Regression test from Bruno Oliveira.
2020-01-20 23:44:56 +02:00