diff --git a/.gitignore b/.gitignore index 83b6dbe73..faea9eac0 100644 --- a/.gitignore +++ b/.gitignore @@ -29,6 +29,7 @@ doc/*/_changelog_towncrier_draft.rst build/ dist/ *.egg-info +htmlcov/ issue/ env/ .env/ diff --git a/changelog/1556.feature.rst b/changelog/1556.feature.rst new file mode 100644 index 000000000..402e772e6 --- /dev/null +++ b/changelog/1556.feature.rst @@ -0,0 +1,17 @@ +pytest now supports ``pyproject.toml`` files for configuration. + +The configuration options is similar to the one available in other formats, but must be defined +in a ``[tool.pytest.ini_options]`` table to be picked up by pytest: + +.. code-block:: toml + + # pyproject.toml + [tool.pytest.ini_options] + minversion = "6.0" + addopts = "-ra -q" + testpaths = [ + "tests", + "integration", + ] + +More information can be found `in the docs `__. diff --git a/doc/en/customize.rst b/doc/en/customize.rst index 35c851ebb..e1f1b253b 100644 --- a/doc/en/customize.rst +++ b/doc/en/customize.rst @@ -14,15 +14,112 @@ configurations files by using the general help option: This will display command line and configuration file settings which were registered by installed plugins. -.. _rootdir: -.. _inifiles: +.. _`config file formats`: -Initialization: determining rootdir and inifile ------------------------------------------------ +Configuration file formats +-------------------------- + +Many :ref:`pytest settings ` can be set in a *configuration file*, which +by convention resides on the root of your repository or in your +tests folder. + +A quick example of the configuration files supported by pytest: + +pytest.ini +~~~~~~~~~~ + +``pytest.ini`` files take precedence over other files, even when empty. + +.. code-block:: ini + + # pytest.ini + [pytest] + minversion = 6.0 + addopts = -ra -q + testpaths = + tests + integration + + +pyproject.toml +~~~~~~~~~~~~~~ + +.. versionadded:: 6.0 + +``pyproject.toml`` are considered for configuration when they contain a ``tool.pytest.ini_options`` table. + +.. code-block:: toml + + # pyproject.toml + [tool.pytest.ini_options] + minversion = "6.0" + addopts = "-ra -q" + testpaths = [ + "tests", + "integration", + ] + +.. note:: + + One might wonder why ``[tool.pytest.ini_options]`` instead of ``[tool.pytest]`` as is the + case with other tools. + + The reason is that the pytest team intends to fully utilize the rich TOML data format + for configuration in the future, reserving the ``[tool.pytest]`` table for that. + The ``ini_options`` table is being used, for now, as a bridge between the existing + ``.ini`` configuration system and the future configuration format. + +tox.ini +~~~~~~~ + +``tox.ini`` files are the configuration files of the `tox `__ project, +and can also be used to hold pytest configuration if they have a ``[pytest]`` section. + +.. code-block:: ini + + # tox.ini + [pytest] + minversion = 6.0 + addopts = -ra -q + testpaths = + tests + integration + + +setup.cfg +~~~~~~~~~ + +``setup.cfg`` files are general purpose configuration files, used originally by `distutils `__, and can also be used to hold pytest configuration +if they have a ``[tool:pytest]`` section. + +.. code-block:: ini + + # setup.cfg + [tool:pytest] + minversion = 6.0 + addopts = -ra -q + testpaths = + tests + integration + +.. warning:: + + Usage of ``setup.cfg`` is not recommended unless for very simple use cases. ``.cfg`` + files use a different parser than ``pytest.ini`` and ``tox.ini`` which might cause hard to track + down problems. + When possible, it is recommended to use the latter files, or ``pyproject.toml``, to hold your + pytest configuration. + + +.. _rootdir: +.. _configfiles: + +Initialization: determining rootdir and configfile +-------------------------------------------------- pytest determines a ``rootdir`` for each test run which depends on the command line arguments (specified test files, paths) and on -the existence of *ini-files*. The determined ``rootdir`` and *ini-file* are +the existence of configuration files. The determined ``rootdir`` and ``configfile`` are printed as part of the pytest header during startup. Here's a summary what ``pytest`` uses ``rootdir`` for: @@ -48,48 +145,47 @@ Finding the ``rootdir`` Here is the algorithm which finds the rootdir from ``args``: -- determine the common ancestor directory for the specified ``args`` that are +- Determine the common ancestor directory for the specified ``args`` that are recognised as paths that exist in the file system. If no such paths are found, the common ancestor directory is set to the current working directory. -- look for ``pytest.ini``, ``tox.ini`` and ``setup.cfg`` files in the ancestor - directory and upwards. If one is matched, it becomes the ini-file and its - directory becomes the rootdir. +- Look for ``pytest.ini``, ``pyproject.toml``, ``tox.ini``, and ``setup.cfg`` files in the ancestor + directory and upwards. If one is matched, it becomes the ``configfile`` and its + directory becomes the ``rootdir``. -- if no ini-file was found, look for ``setup.py`` upwards from the common +- If no configuration file was found, look for ``setup.py`` upwards from the common ancestor directory to determine the ``rootdir``. -- if no ``setup.py`` was found, look for ``pytest.ini``, ``tox.ini`` and +- If no ``setup.py`` was found, look for ``pytest.ini``, ``pyproject.toml``, ``tox.ini``, and ``setup.cfg`` in each of the specified ``args`` and upwards. If one is - matched, it becomes the ini-file and its directory becomes the rootdir. + matched, it becomes the ``configfile`` and its directory becomes the ``rootdir``. -- if no ini-file was found, use the already determined common ancestor as root +- If no ``configfile`` was found, use the already determined common ancestor as root directory. This allows the use of pytest in structures that are not part of - a package and don't have any particular ini-file configuration. + a package and don't have any particular configuration file. If no ``args`` are given, pytest collects test below the current working -directory and also starts determining the rootdir from there. +directory and also starts determining the ``rootdir`` from there. -:warning: custom pytest plugin commandline arguments may include a path, as in - ``pytest --log-output ../../test.log args``. Then ``args`` is mandatory, - otherwise pytest uses the folder of test.log for rootdir determination - (see also `issue 1435 `_). - A dot ``.`` for referencing to the current working directory is also - possible. +Files will only be matched for configuration if: -Note that an existing ``pytest.ini`` file will always be considered a match, -whereas ``tox.ini`` and ``setup.cfg`` will only match if they contain a -``[pytest]`` or ``[tool:pytest]`` section, respectively. Options from multiple ini-files candidates are never -merged - the first one wins (``pytest.ini`` always wins, even if it does not -contain a ``[pytest]`` section). +* ``pytest.ini``: will always match and take precedence, even if empty. +* ``pyproject.toml``: contains a ``[tool.pytest.ini_options]`` table. +* ``tox.ini``: contains a ``[pytest]`` section. +* ``setup.cfg``: contains a ``[tool:pytest]`` section. -The ``config`` object will subsequently carry these attributes: +The files are considered in the order above. Options from multiple ``configfiles`` candidates +are never merged - the first match wins. + +The internal :class:`Config <_pytest.config.Config>` object (accessible via hooks or through the :fixture:`pytestconfig` fixture) +will subsequently carry these attributes: - ``config.rootdir``: the determined root directory, guaranteed to exist. -- ``config.inifile``: the determined ini-file, may be ``None``. +- ``config.inifile``: the determined ``configfile``, may be ``None`` (it is named ``inifile`` + for historical reasons). -The rootdir is used as a reference directory for constructing test +The ``rootdir`` is used as a reference directory for constructing test addresses ("nodeids") and can be used also by plugins for storing per-testrun information. @@ -100,75 +196,38 @@ Example: pytest path/to/testdir path/other/ will determine the common ancestor as ``path`` and then -check for ini-files as follows: +check for configuration files as follows: .. code-block:: text # first look for pytest.ini files path/pytest.ini - path/tox.ini # must also contain [pytest] section to match - path/setup.cfg # must also contain [tool:pytest] section to match + path/pyproject.toml # must contain a [tool.pytest.ini_options] table to match + path/tox.ini # must contain [pytest] section to match + path/setup.cfg # must contain [tool:pytest] section to match pytest.ini - ... # all the way down to the root + ... # all the way up to the root # now look for setup.py path/setup.py setup.py - ... # all the way down to the root + ... # all the way up to the root + + +.. warning:: + + Custom pytest plugin commandline arguments may include a path, as in + ``pytest --log-output ../../test.log args``. Then ``args`` is mandatory, + otherwise pytest uses the folder of test.log for rootdir determination + (see also `issue 1435 `_). + A dot ``.`` for referencing to the current working directory is also + possible. .. _`how to change command line options defaults`: .. _`adding default options`: - -How to change command line options defaults ------------------------------------------------- - -It can be tedious to type the same series of command line options -every time you use ``pytest``. For example, if you always want to see -detailed info on skipped and xfailed tests, as well as have terser "dot" -progress output, you can write it into a configuration file: - -.. code-block:: ini - - # content of pytest.ini or tox.ini - [pytest] - addopts = -ra -q - - # content of setup.cfg - [tool:pytest] - addopts = -ra -q - -Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command -line options while the environment is in use: - -.. code-block:: bash - - export PYTEST_ADDOPTS="-v" - -Here's how the command-line is built in the presence of ``addopts`` or the environment variable: - -.. code-block:: text - - $PYTEST_ADDOPTS - -So if the user executes in the command-line: - -.. code-block:: bash - - pytest -m slow - -The actual command line executed is: - -.. code-block:: bash - - pytest -ra -q -v -m slow - -Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example -above will show verbose output because ``-v`` overwrites ``-q``. - - Builtin configuration file options ---------------------------------------------- diff --git a/doc/en/example/pythoncollection.rst b/doc/en/example/pythoncollection.rst index d8261a949..30d106ada 100644 --- a/doc/en/example/pythoncollection.rst +++ b/doc/en/example/pythoncollection.rst @@ -115,15 +115,13 @@ Changing naming conventions You can configure different naming conventions by setting the :confval:`python_files`, :confval:`python_classes` and -:confval:`python_functions` configuration options. +:confval:`python_functions` in your :ref:`configuration file `. Here is an example: .. code-block:: ini # content of pytest.ini # Example 1: have pytest look for "check" instead of "test" - # can also be defined in tox.ini or setup.cfg file, although the section - # name in setup.cfg files should be "tool:pytest" [pytest] python_files = check_*.py python_classes = Check @@ -165,8 +163,7 @@ You can check for multiple glob patterns by adding a space between the patterns: .. code-block:: ini # Example 2: have pytest look for files with "test" and "example" - # content of pytest.ini, tox.ini, or setup.cfg file (replace "pytest" - # with "tool:pytest" for setup.cfg) + # content of pytest.ini [pytest] python_files = test_*.py example_*.py diff --git a/doc/en/example/simple.rst b/doc/en/example/simple.rst index 3282bbda5..d1a1ecdfc 100644 --- a/doc/en/example/simple.rst +++ b/doc/en/example/simple.rst @@ -3,6 +3,50 @@ Basic patterns and examples ========================================================== +How to change command line options defaults +------------------------------------------- + +It can be tedious to type the same series of command line options +every time you use ``pytest``. For example, if you always want to see +detailed info on skipped and xfailed tests, as well as have terser "dot" +progress output, you can write it into a configuration file: + +.. code-block:: ini + + # content of pytest.ini + [pytest] + addopts = -ra -q + + +Alternatively, you can set a ``PYTEST_ADDOPTS`` environment variable to add command +line options while the environment is in use: + +.. code-block:: bash + + export PYTEST_ADDOPTS="-v" + +Here's how the command-line is built in the presence of ``addopts`` or the environment variable: + +.. code-block:: text + + $PYTEST_ADDOPTS + +So if the user executes in the command-line: + +.. code-block:: bash + + pytest -m slow + +The actual command line executed is: + +.. code-block:: bash + + pytest -ra -q -v -m slow + +Note that as usual for other command-line applications, in case of conflicting options the last one wins, so the example +above will show verbose output because ``-v`` overwrites ``-q``. + + .. _request example: Pass different values to a test function, depending on command line options diff --git a/doc/en/reference.rst b/doc/en/reference.rst index 7348636a2..326b3e52a 100644 --- a/doc/en/reference.rst +++ b/doc/en/reference.rst @@ -1019,17 +1019,17 @@ UsageError Configuration Options --------------------- -Here is a list of builtin configuration options that may be written in a ``pytest.ini``, ``tox.ini`` or ``setup.cfg`` -file, usually located at the root of your repository. All options must be under a ``[pytest]`` section -(``[tool:pytest]`` for ``setup.cfg`` files). +Here is a list of builtin configuration options that may be written in a ``pytest.ini``, ``pyproject.toml``, ``tox.ini`` or ``setup.cfg`` +file, usually located at the root of your repository. To see each file format in details, see +:ref:`config file formats`. .. warning:: - Usage of ``setup.cfg`` is not recommended unless for very simple use cases. ``.cfg`` + Usage of ``setup.cfg`` is not recommended except for very simple use cases. ``.cfg`` files use a different parser than ``pytest.ini`` and ``tox.ini`` which might cause hard to track down problems. - When possible, it is recommended to use the latter files to hold your pytest configuration. + When possible, it is recommended to use the latter files, or ``pyproject.toml``, to hold your pytest configuration. -Configuration file options may be overwritten in the command-line by using ``-o/--override-ini``, which can also be +Configuration options may be overwritten in the command-line by using ``-o/--override-ini``, which can also be passed multiple times. The expected format is ``name=value``. For example:: pytest -o console_output_style=classic -o cache_dir=/tmp/mycache @@ -1057,8 +1057,6 @@ passed multiple times. The expected format is ``name=value``. For example:: .. confval:: cache_dir - - Sets a directory where stores content of cache plugin. Default directory is ``.pytest_cache`` which is created in :ref:`rootdir `. Directory may be relative or absolute path. If setting relative path, then directory is created diff --git a/pyproject.toml b/pyproject.toml index aa57762e7..493213d84 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -7,6 +7,49 @@ requires = [ ] build-backend = "setuptools.build_meta" +[tool.pytest.ini_options] +minversion = "2.0" +addopts = "-rfEX -p pytester --strict-markers" +python_files = ["test_*.py", "*_test.py", "testing/*/*.py"] +python_classes = ["Test", "Acceptance"] +python_functions = ["test"] +# NOTE: "doc" is not included here, but gets tested explicitly via "doctesting". +testpaths = ["testing"] +norecursedirs = ["testing/example_scripts"] +xfail_strict = true +filterwarnings = [ + "error", + "default:Using or importing the ABCs:DeprecationWarning:unittest2.*", + "default:the imp module is deprecated in favour of importlib:DeprecationWarning:nose.*", + "ignore:Module already imported so cannot be rewritten:pytest.PytestWarning", + # produced by python3.6/site.py itself (3.6.7 on Travis, could not trigger it with 3.6.8)." + "ignore:.*U.*mode is deprecated:DeprecationWarning:(?!(pytest|_pytest))", + # produced by pytest-xdist + "ignore:.*type argument to addoption.*:DeprecationWarning", + # produced by python >=3.5 on execnet (pytest-xdist) + "ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning", + # pytest's own futurewarnings + "ignore::pytest.PytestExperimentalApiWarning", + # Do not cause SyntaxError for invalid escape sequences in py37. + # Those are caught/handled by pyupgrade, and not easy to filter with the + # module being the filename (with .py removed). + "default:invalid escape sequence:DeprecationWarning", + # ignore use of unregistered marks, because we use many to test the implementation + "ignore::_pytest.warning_types.PytestUnknownMarkWarning", +] +pytester_example_dir = "testing/example_scripts" +markers = [ + # dummy markers for testing + "foo", + "bar", + "baz", + # conftest.py reorders tests moving slow ones to the end of the list + "slow", + # experimental mark for all tests using pexpect + "uses_pexpect", +] + + [tool.towncrier] package = "pytest" package_dir = "src" diff --git a/setup.cfg b/setup.cfg index 5dc778d99..8749334f8 100644 --- a/setup.cfg +++ b/setup.cfg @@ -46,6 +46,7 @@ install_requires = packaging pluggy>=0.12,<1.0 py>=1.5.0 + toml atomicwrites>=1.0;sys_platform=="win32" colorama;sys_platform=="win32" importlib-metadata>=0.12;python_version<"3.8" diff --git a/src/_pytest/config/__init__.py b/src/_pytest/config/__init__.py index 27083900d..4b1d0bd2a 100644 --- a/src/_pytest/config/__init__.py +++ b/src/_pytest/config/__init__.py @@ -34,7 +34,6 @@ import _pytest.hookspec # the extension point definitions from .exceptions import PrintHelp from .exceptions import UsageError from .findpaths import determine_setup -from .findpaths import exists from _pytest._code import ExceptionInfo from _pytest._code import filter_traceback from _pytest._io import TerminalWriter @@ -450,7 +449,7 @@ class PytestPluginManager(PluginManager): if i != -1: path = path[:i] anchor = current.join(path, abs=1) - if exists(anchor): # we found some file object + if anchor.exists(): # we found some file object self._try_load_conftest(anchor) foundanchor = True if not foundanchor: @@ -1069,13 +1068,8 @@ class Config: if Version(minver) > Version(pytest.__version__): raise pytest.UsageError( - "%s:%d: requires pytest-%s, actual pytest-%s'" - % ( - self.inicfg.config.path, - self.inicfg.lineof("minversion"), - minver, - pytest.__version__, - ) + "%s: 'minversion' requires pytest-%s, actual pytest-%s'" + % (self.inifile, minver, pytest.__version__,) ) def _validatekeys(self): @@ -1123,7 +1117,7 @@ class Config: x.append(line) # modifies the cached list inline def getini(self, name: str): - """ return configuration value from an :ref:`ini file `. If the + """ return configuration value from an :ref:`ini file `. If the specified name hasn't been registered through a prior :py:func:`parser.addini <_pytest.config.argparsing.Parser.addini>` call (usually from a plugin), a ValueError is raised. """ @@ -1138,8 +1132,8 @@ class Config: description, type, default = self._parser._inidict[name] except KeyError: raise ValueError("unknown configuration value: {!r}".format(name)) - value = self._get_override_ini_value(name) - if value is None: + override_value = self._get_override_ini_value(name) + if override_value is None: try: value = self.inicfg[name] except KeyError: @@ -1148,18 +1142,35 @@ class Config: if type is None: return "" return [] + else: + value = override_value + # coerce the values based on types + # note: some coercions are only required if we are reading from .ini files, because + # the file format doesn't contain type information, but when reading from toml we will + # get either str or list of str values (see _parse_ini_config_from_pyproject_toml). + # for example: + # + # ini: + # a_line_list = "tests acceptance" + # in this case, we need to split the string to obtain a list of strings + # + # toml: + # a_line_list = ["tests", "acceptance"] + # in this case, we already have a list ready to use + # if type == "pathlist": - dp = py.path.local(self.inicfg.config.path).dirpath() - values = [] - for relpath in shlex.split(value): - values.append(dp.join(relpath, abs=True)) - return values + dp = py.path.local(self.inifile).dirpath() + input_values = shlex.split(value) if isinstance(value, str) else value + return [dp.join(x, abs=True) for x in input_values] elif type == "args": - return shlex.split(value) + return shlex.split(value) if isinstance(value, str) else value elif type == "linelist": - return [t for t in map(lambda x: x.strip(), value.split("\n")) if t] + if isinstance(value, str): + return [t for t in map(lambda x: x.strip(), value.split("\n")) if t] + else: + return value elif type == "bool": - return bool(_strtobool(value.strip())) + return bool(_strtobool(str(value).strip())) else: assert type is None return value diff --git a/src/_pytest/config/findpaths.py b/src/_pytest/config/findpaths.py index 2b252c4f4..796fa9b0a 100644 --- a/src/_pytest/config/findpaths.py +++ b/src/_pytest/config/findpaths.py @@ -1,13 +1,13 @@ import os -from typing import Any +from typing import Dict from typing import Iterable from typing import List from typing import Optional from typing import Tuple +from typing import Union +import iniconfig import py -from iniconfig import IniConfig -from iniconfig import ParseError from .exceptions import UsageError from _pytest.compat import TYPE_CHECKING @@ -17,52 +17,95 @@ if TYPE_CHECKING: from . import Config -def exists(path, ignore=OSError): +def _parse_ini_config(path: py.path.local) -> iniconfig.IniConfig: + """Parses the given generic '.ini' file using legacy IniConfig parser, returning + the parsed object. + + Raises UsageError if the file cannot be parsed. + """ try: - return path.check() - except ignore: - return False + return iniconfig.IniConfig(path) + except iniconfig.ParseError as exc: + raise UsageError(str(exc)) -def getcfg(args, config=None): +def load_config_dict_from_file( + filepath: py.path.local, +) -> Optional[Dict[str, Union[str, List[str]]]]: + """Loads pytest configuration from the given file path, if supported. + + Return None if the file does not contain valid pytest configuration. """ - Search the list of arguments for a valid ini-file for pytest, + + # configuration from ini files are obtained from the [pytest] section, if present. + if filepath.ext == ".ini": + iniconfig = _parse_ini_config(filepath) + + if "pytest" in iniconfig: + return dict(iniconfig["pytest"].items()) + else: + # "pytest.ini" files are always the source of configuration, even if empty + if filepath.basename == "pytest.ini": + return {} + + # '.cfg' files are considered if they contain a "[tool:pytest]" section + elif filepath.ext == ".cfg": + iniconfig = _parse_ini_config(filepath) + + if "tool:pytest" in iniconfig.sections: + return dict(iniconfig["tool:pytest"].items()) + elif "pytest" in iniconfig.sections: + # If a setup.cfg contains a "[pytest]" section, we raise a failure to indicate users that + # plain "[pytest]" sections in setup.cfg files is no longer supported (#3086). + fail(CFG_PYTEST_SECTION.format(filename="setup.cfg"), pytrace=False) + + # '.toml' files are considered if they contain a [tool.pytest.ini_options] table + elif filepath.ext == ".toml": + import toml + + config = toml.load(filepath) + + result = config.get("tool", {}).get("pytest", {}).get("ini_options", None) + if result is not None: + # TOML supports richer data types than ini files (strings, arrays, floats, ints, etc), + # however we need to convert all scalar values to str for compatibility with the rest + # of the configuration system, which expects strings only. + def make_scalar(v: object) -> Union[str, List[str]]: + return v if isinstance(v, list) else str(v) + + return {k: make_scalar(v) for k, v in result.items()} + + return None + + +def locate_config( + args: Iterable[Union[str, py.path.local]] +) -> Tuple[ + Optional[py.path.local], Optional[py.path.local], Dict[str, Union[str, List[str]]], +]: + """ + Search in the list of arguments for a valid ini-file for pytest, and return a tuple of (rootdir, inifile, cfg-dict). - - note: config is optional and used only to issue warnings explicitly (#2891). """ - inibasenames = ["pytest.ini", "tox.ini", "setup.cfg"] + config_names = [ + "pytest.ini", + "pyproject.toml", + "tox.ini", + "setup.cfg", + ] args = [x for x in args if not str(x).startswith("-")] if not args: args = [py.path.local()] for arg in args: arg = py.path.local(arg) for base in arg.parts(reverse=True): - for inibasename in inibasenames: - p = base.join(inibasename) - if exists(p): - try: - iniconfig = IniConfig(p) - except ParseError as exc: - raise UsageError(str(exc)) - - if ( - inibasename == "setup.cfg" - and "tool:pytest" in iniconfig.sections - ): - return base, p, iniconfig["tool:pytest"] - elif "pytest" in iniconfig.sections: - if inibasename == "setup.cfg" and config is not None: - - fail( - CFG_PYTEST_SECTION.format(filename=inibasename), - pytrace=False, - ) - return base, p, iniconfig["pytest"] - elif inibasename == "pytest.ini": - # allowed to be empty - return base, p, {} - return None, None, None + for config_name in config_names: + p = base.join(config_name) + if p.isfile(): + ini_config = load_config_dict_from_file(p) + if ini_config is not None: + return base, p, ini_config + return None, None, {} def get_common_ancestor(paths: Iterable[py.path.local]) -> py.path.local: @@ -118,29 +161,16 @@ def determine_setup( args: List[str], rootdir_cmd_arg: Optional[str] = None, config: Optional["Config"] = None, -) -> Tuple[py.path.local, Optional[str], Any]: +) -> Tuple[py.path.local, Optional[str], Dict[str, Union[str, List[str]]]]: + rootdir = None dirs = get_dirs_from_args(args) if inifile: - iniconfig = IniConfig(inifile) - is_cfg_file = str(inifile).endswith(".cfg") - sections = ["tool:pytest", "pytest"] if is_cfg_file else ["pytest"] - for section in sections: - try: - inicfg = iniconfig[ - section - ] # type: Optional[py.iniconfig._SectionWrapper] - if is_cfg_file and section == "pytest" and config is not None: - fail( - CFG_PYTEST_SECTION.format(filename=str(inifile)), pytrace=False - ) - break - except KeyError: - inicfg = None + inicfg = load_config_dict_from_file(py.path.local(inifile)) or {} if rootdir_cmd_arg is None: rootdir = get_common_ancestor(dirs) else: ancestor = get_common_ancestor(dirs) - rootdir, inifile, inicfg = getcfg([ancestor], config=config) + rootdir, inifile, inicfg = locate_config([ancestor]) if rootdir is None and rootdir_cmd_arg is None: for possible_rootdir in ancestor.parts(reverse=True): if possible_rootdir.join("setup.py").exists(): @@ -148,7 +178,7 @@ def determine_setup( break else: if dirs != [ancestor]: - rootdir, inifile, inicfg = getcfg(dirs, config=config) + rootdir, inifile, inicfg = locate_config(dirs) if rootdir is None: if config is not None: cwd = config.invocation_dir diff --git a/src/_pytest/pytester.py b/src/_pytest/pytester.py index 8df5992d6..2913c6065 100644 --- a/src/_pytest/pytester.py +++ b/src/_pytest/pytester.py @@ -688,6 +688,13 @@ class Testdir: p = self.makeini(source) return IniConfig(p)["pytest"] + def makepyprojecttoml(self, source): + """Write a pyproject.toml file with 'source' as contents. + + .. versionadded:: 6.0 + """ + return self.makefile(".toml", pyproject=source) + def makepyfile(self, *args, **kwargs): r"""Shortcut for .makefile() with a .py extension. Defaults to the test name with a '.py' extension, e.g test_foobar.py, overwriting diff --git a/src/_pytest/terminal.py b/src/_pytest/terminal.py index b37828e5a..9c2665fb8 100644 --- a/src/_pytest/terminal.py +++ b/src/_pytest/terminal.py @@ -691,7 +691,7 @@ class TerminalReporter: line = "rootdir: %s" % config.rootdir if config.inifile: - line += ", inifile: " + config.rootdir.bestrelpath(config.inifile) + line += ", configfile: " + config.rootdir.bestrelpath(config.inifile) testpaths = config.getini("testpaths") if testpaths and config.args == testpaths: diff --git a/testing/test_config.py b/testing/test_config.py index 867012e93..31dfd9fa3 100644 --- a/testing/test_config.py +++ b/testing/test_config.py @@ -18,7 +18,7 @@ from _pytest.config import ExitCode from _pytest.config.exceptions import UsageError from _pytest.config.findpaths import determine_setup from _pytest.config.findpaths import get_common_ancestor -from _pytest.config.findpaths import getcfg +from _pytest.config.findpaths import locate_config from _pytest.pathlib import Path @@ -39,14 +39,14 @@ class TestParseIni: ) ) ) - _, _, cfg = getcfg([sub]) + _, _, cfg = locate_config([sub]) assert cfg["name"] == "value" config = testdir.parseconfigure(sub) assert config.inicfg["name"] == "value" def test_getcfg_empty_path(self): """correctly handle zero length arguments (a la pytest '')""" - getcfg([""]) + locate_config([""]) def test_setupcfg_uses_toolpytest_with_pytest(self, testdir): p1 = testdir.makepyfile("def test(): pass") @@ -61,7 +61,7 @@ class TestParseIni: % p1.basename, ) result = testdir.runpytest() - result.stdout.fnmatch_lines(["*, inifile: setup.cfg, *", "* 1 passed in *"]) + result.stdout.fnmatch_lines(["*, configfile: setup.cfg, *", "* 1 passed in *"]) assert result.ret == 0 def test_append_parse_args(self, testdir, tmpdir, monkeypatch): @@ -85,12 +85,14 @@ class TestParseIni: ".ini", tox=""" [pytest] - minversion=9.0 + minversion=999.0 """, ) result = testdir.runpytest() assert result.ret != 0 - result.stderr.fnmatch_lines(["*tox.ini:2*requires*9.0*actual*"]) + result.stderr.fnmatch_lines( + ["*tox.ini: 'minversion' requires pytest-999.0, actual pytest-*"] + ) @pytest.mark.parametrize( "section, name", @@ -110,6 +112,16 @@ class TestParseIni: config = testdir.parseconfig() assert config.getini("minversion") == "1.0" + def test_pyproject_toml(self, testdir): + testdir.makepyprojecttoml( + """ + [tool.pytest.ini_options] + minversion = "1.0" + """ + ) + config = testdir.parseconfig() + assert config.getini("minversion") == "1.0" + def test_toxini_before_lower_pytestini(self, testdir): sub = testdir.tmpdir.mkdir("sub") sub.join("tox.ini").write( @@ -251,6 +263,18 @@ class TestConfigCmdlineParsing: config = testdir.parseconfig("-c", "custom_tool_pytest_section.cfg") assert config.getini("custom") == "1" + testdir.makefile( + ".toml", + custom=""" + [tool.pytest.ini_options] + custom = 1 + value = [ + ] # this is here on purpose, as it makes this an invalid '.ini' file + """, + ) + config = testdir.parseconfig("-c", "custom.toml") + assert config.getini("custom") == "1" + def test_absolute_win32_path(self, testdir): temp_ini_file = testdir.makefile( ".ini", @@ -350,7 +374,7 @@ class TestConfigAPI: assert val == "hello" pytest.raises(ValueError, config.getini, "other") - def test_addini_pathlist(self, testdir): + def make_conftest_for_pathlist(self, testdir): testdir.makeconftest( """ def pytest_addoption(parser): @@ -358,20 +382,36 @@ class TestConfigAPI: parser.addini("abc", "abc value") """ ) + + def test_addini_pathlist_ini_files(self, testdir): + self.make_conftest_for_pathlist(testdir) p = testdir.makeini( """ [pytest] paths=hello world/sub.py """ ) + self.check_config_pathlist(testdir, p) + + def test_addini_pathlist_pyproject_toml(self, testdir): + self.make_conftest_for_pathlist(testdir) + p = testdir.makepyprojecttoml( + """ + [tool.pytest.ini_options] + paths=["hello", "world/sub.py"] + """ + ) + self.check_config_pathlist(testdir, p) + + def check_config_pathlist(self, testdir, config_path): config = testdir.parseconfig() values = config.getini("paths") assert len(values) == 2 - assert values[0] == p.dirpath("hello") - assert values[1] == p.dirpath("world/sub.py") + assert values[0] == config_path.dirpath("hello") + assert values[1] == config_path.dirpath("world/sub.py") pytest.raises(ValueError, config.getini, "other") - def test_addini_args(self, testdir): + def make_conftest_for_args(self, testdir): testdir.makeconftest( """ def pytest_addoption(parser): @@ -379,20 +419,35 @@ class TestConfigAPI: parser.addini("a2", "", "args", default="1 2 3".split()) """ ) + + def test_addini_args_ini_files(self, testdir): + self.make_conftest_for_args(testdir) testdir.makeini( """ [pytest] args=123 "123 hello" "this" - """ + """ ) + self.check_config_args(testdir) + + def test_addini_args_pyproject_toml(self, testdir): + self.make_conftest_for_args(testdir) + testdir.makepyprojecttoml( + """ + [tool.pytest.ini_options] + args = ["123", "123 hello", "this"] + """ + ) + self.check_config_args(testdir) + + def check_config_args(self, testdir): config = testdir.parseconfig() values = config.getini("args") - assert len(values) == 3 assert values == ["123", "123 hello", "this"] values = config.getini("a2") assert values == list("123") - def test_addini_linelist(self, testdir): + def make_conftest_for_linelist(self, testdir): testdir.makeconftest( """ def pytest_addoption(parser): @@ -400,6 +455,9 @@ class TestConfigAPI: parser.addini("a2", "", "linelist") """ ) + + def test_addini_linelist_ini_files(self, testdir): + self.make_conftest_for_linelist(testdir) testdir.makeini( """ [pytest] @@ -407,6 +465,19 @@ class TestConfigAPI: second line """ ) + self.check_config_linelist(testdir) + + def test_addini_linelist_pprojecttoml(self, testdir): + self.make_conftest_for_linelist(testdir) + testdir.makepyprojecttoml( + """ + [tool.pytest.ini_options] + xy = ["123 345", "second line"] + """ + ) + self.check_config_linelist(testdir) + + def check_config_linelist(self, testdir): config = testdir.parseconfig() values = config.getini("xy") assert len(values) == 2 @@ -832,7 +903,6 @@ def test_consider_args_after_options_for_rootdir(testdir, args): result.stdout.fnmatch_lines(["*rootdir: *myroot"]) -@pytest.mark.skipif("sys.platform == 'win32'") def test_toolongargs_issue224(testdir): result = testdir.runpytest("-m", "hello" * 500) assert result.ret == ExitCode.NO_TESTS_COLLECTED @@ -964,10 +1034,20 @@ class TestRootdir: assert get_common_ancestor([no_path]) == tmpdir assert get_common_ancestor([no_path.join("a")]) == tmpdir - @pytest.mark.parametrize("name", "setup.cfg tox.ini pytest.ini".split()) - def test_with_ini(self, tmpdir: py.path.local, name: str) -> None: + @pytest.mark.parametrize( + "name, contents", + [ + pytest.param("pytest.ini", "[pytest]\nx=10", id="pytest.ini"), + pytest.param( + "pyproject.toml", "[tool.pytest.ini_options]\nx=10", id="pyproject.toml" + ), + pytest.param("tox.ini", "[pytest]\nx=10", id="tox.ini"), + pytest.param("setup.cfg", "[tool:pytest]\nx=10", id="setup.cfg"), + ], + ) + def test_with_ini(self, tmpdir: py.path.local, name: str, contents: str) -> None: inifile = tmpdir.join(name) - inifile.write("[pytest]\n" if name != "setup.cfg" else "[tool:pytest]\n") + inifile.write(contents) a = tmpdir.mkdir("a") b = a.mkdir("b") @@ -975,9 +1055,10 @@ class TestRootdir: rootdir, parsed_inifile, _ = determine_setup(None, args) assert rootdir == tmpdir assert parsed_inifile == inifile - rootdir, parsed_inifile, _ = determine_setup(None, [str(b), str(a)]) + rootdir, parsed_inifile, ini_config = determine_setup(None, [str(b), str(a)]) assert rootdir == tmpdir assert parsed_inifile == inifile + assert ini_config == {"x": "10"} @pytest.mark.parametrize("name", "setup.cfg tox.ini".split()) def test_pytestini_overrides_empty_other(self, tmpdir: py.path.local, name) -> None: @@ -1004,10 +1085,26 @@ class TestRootdir: assert inifile is None assert inicfg == {} - def test_with_specific_inifile(self, tmpdir: py.path.local) -> None: - inifile = tmpdir.ensure("pytest.ini") - rootdir, _, _ = determine_setup(str(inifile), [str(tmpdir)]) + @pytest.mark.parametrize( + "name, contents", + [ + # pytest.param("pytest.ini", "[pytest]\nx=10", id="pytest.ini"), + pytest.param( + "pyproject.toml", "[tool.pytest.ini_options]\nx=10", id="pyproject.toml" + ), + # pytest.param("tox.ini", "[pytest]\nx=10", id="tox.ini"), + # pytest.param("setup.cfg", "[tool:pytest]\nx=10", id="setup.cfg"), + ], + ) + def test_with_specific_inifile( + self, tmpdir: py.path.local, name: str, contents: str + ) -> None: + p = tmpdir.ensure(name) + p.write(contents) + rootdir, inifile, ini_config = determine_setup(str(p), [str(tmpdir)]) assert rootdir == tmpdir + assert inifile == p + assert ini_config == {"x": "10"} def test_with_arg_outside_cwd_without_inifile(self, tmpdir, monkeypatch) -> None: monkeypatch.chdir(str(tmpdir)) diff --git a/testing/test_findpaths.py b/testing/test_findpaths.py new file mode 100644 index 000000000..3de2ea218 --- /dev/null +++ b/testing/test_findpaths.py @@ -0,0 +1,110 @@ +from textwrap import dedent + +import py + +import pytest +from _pytest.config.findpaths import get_common_ancestor +from _pytest.config.findpaths import load_config_dict_from_file + + +class TestLoadConfigDictFromFile: + def test_empty_pytest_ini(self, tmpdir): + """pytest.ini files are always considered for configuration, even if empty""" + fn = tmpdir.join("pytest.ini") + fn.write("") + assert load_config_dict_from_file(fn) == {} + + def test_pytest_ini(self, tmpdir): + """[pytest] section in pytest.ini files is read correctly""" + fn = tmpdir.join("pytest.ini") + fn.write("[pytest]\nx=1") + assert load_config_dict_from_file(fn) == {"x": "1"} + + def test_custom_ini(self, tmpdir): + """[pytest] section in any .ini file is read correctly""" + fn = tmpdir.join("custom.ini") + fn.write("[pytest]\nx=1") + assert load_config_dict_from_file(fn) == {"x": "1"} + + def test_custom_ini_without_section(self, tmpdir): + """Custom .ini files without [pytest] section are not considered for configuration""" + fn = tmpdir.join("custom.ini") + fn.write("[custom]") + assert load_config_dict_from_file(fn) is None + + def test_custom_cfg_file(self, tmpdir): + """Custom .cfg files without [tool:pytest] section are not considered for configuration""" + fn = tmpdir.join("custom.cfg") + fn.write("[custom]") + assert load_config_dict_from_file(fn) is None + + def test_valid_cfg_file(self, tmpdir): + """Custom .cfg files with [tool:pytest] section are read correctly""" + fn = tmpdir.join("custom.cfg") + fn.write("[tool:pytest]\nx=1") + assert load_config_dict_from_file(fn) == {"x": "1"} + + def test_unsupported_pytest_section_in_cfg_file(self, tmpdir): + """.cfg files with [pytest] section are no longer supported and should fail to alert users""" + fn = tmpdir.join("custom.cfg") + fn.write("[pytest]") + with pytest.raises(pytest.fail.Exception): + load_config_dict_from_file(fn) + + def test_invalid_toml_file(self, tmpdir): + """.toml files without [tool.pytest.ini_options] are not considered for configuration.""" + fn = tmpdir.join("myconfig.toml") + fn.write( + dedent( + """ + [build_system] + x = 1 + """ + ) + ) + assert load_config_dict_from_file(fn) is None + + def test_valid_toml_file(self, tmpdir): + """.toml files with [tool.pytest.ini_options] are read correctly, including changing + data types to str/list for compatibility with other configuration options.""" + fn = tmpdir.join("myconfig.toml") + fn.write( + dedent( + """ + [tool.pytest.ini_options] + x = 1 + y = 20.0 + values = ["tests", "integration"] + name = "foo" + """ + ) + ) + assert load_config_dict_from_file(fn) == { + "x": "1", + "y": "20.0", + "values": ["tests", "integration"], + "name": "foo", + } + + +class TestCommonAncestor: + def test_has_ancestor(self, tmpdir): + fn1 = tmpdir.join("foo/bar/test_1.py").ensure(file=1) + fn2 = tmpdir.join("foo/zaz/test_2.py").ensure(file=1) + assert get_common_ancestor([fn1, fn2]) == tmpdir.join("foo") + assert get_common_ancestor([py.path.local(fn1.dirname), fn2]) == tmpdir.join( + "foo" + ) + assert get_common_ancestor( + [py.path.local(fn1.dirname), py.path.local(fn2.dirname)] + ) == tmpdir.join("foo") + assert get_common_ancestor([fn1, py.path.local(fn2.dirname)]) == tmpdir.join( + "foo" + ) + + def test_single_dir(self, tmpdir): + assert get_common_ancestor([tmpdir]) == tmpdir + + def test_single_file(self, tmpdir): + fn = tmpdir.join("foo.py").ensure(file=1) + assert get_common_ancestor([fn]) == tmpdir diff --git a/testing/test_terminal.py b/testing/test_terminal.py index 7d7c82ad6..e8402079b 100644 --- a/testing/test_terminal.py +++ b/testing/test_terminal.py @@ -706,10 +706,10 @@ class TestTerminalFunctional: result = testdir.runpytest() result.stdout.fnmatch_lines(["rootdir: *test_header0"]) - # with inifile + # with configfile testdir.makeini("""[pytest]""") result = testdir.runpytest() - result.stdout.fnmatch_lines(["rootdir: *test_header0, inifile: tox.ini"]) + result.stdout.fnmatch_lines(["rootdir: *test_header0, configfile: tox.ini"]) # with testpaths option, and not passing anything in the command-line testdir.makeini( @@ -720,12 +720,12 @@ class TestTerminalFunctional: ) result = testdir.runpytest() result.stdout.fnmatch_lines( - ["rootdir: *test_header0, inifile: tox.ini, testpaths: tests, gui"] + ["rootdir: *test_header0, configfile: tox.ini, testpaths: tests, gui"] ) # with testpaths option, passing directory in command-line: do not show testpaths then result = testdir.runpytest("tests") - result.stdout.fnmatch_lines(["rootdir: *test_header0, inifile: tox.ini"]) + result.stdout.fnmatch_lines(["rootdir: *test_header0, configfile: tox.ini"]) def test_showlocals(self, testdir): p1 = testdir.makepyfile( diff --git a/tox.ini b/tox.ini index 8e1a51ca7..affb4a7a9 100644 --- a/tox.ini +++ b/tox.ini @@ -152,48 +152,6 @@ deps = pypandoc commands = python scripts/publish-gh-release-notes.py {posargs} - -[pytest] -minversion = 2.0 -addopts = -rfEX -p pytester --strict-markers -rsyncdirs = tox.ini doc src testing -python_files = test_*.py *_test.py testing/*/*.py -python_classes = Test Acceptance -python_functions = test -# NOTE: "doc" is not included here, but gets tested explicitly via "doctesting". -testpaths = testing -norecursedirs = testing/example_scripts -xfail_strict=true -filterwarnings = - error - default:Using or importing the ABCs:DeprecationWarning:unittest2.* - default:the imp module is deprecated in favour of importlib:DeprecationWarning:nose.* - ignore:Module already imported so cannot be rewritten:pytest.PytestWarning - # produced by python3.6/site.py itself (3.6.7 on Travis, could not trigger it with 3.6.8). - ignore:.*U.*mode is deprecated:DeprecationWarning:(?!(pytest|_pytest)) - # produced by pytest-xdist - ignore:.*type argument to addoption.*:DeprecationWarning - # produced by python >=3.5 on execnet (pytest-xdist) - ignore:.*inspect.getargspec.*deprecated, use inspect.signature.*:DeprecationWarning - # pytest's own futurewarnings - ignore::pytest.PytestExperimentalApiWarning - # Do not cause SyntaxError for invalid escape sequences in py37. - # Those are caught/handled by pyupgrade, and not easy to filter with the - # module being the filename (with .py removed). - default:invalid escape sequence:DeprecationWarning - # ignore use of unregistered marks, because we use many to test the implementation - ignore::_pytest.warning_types.PytestUnknownMarkWarning -pytester_example_dir = testing/example_scripts -markers = - # dummy markers for testing - foo - bar - baz - # conftest.py reorders tests moving slow ones to the end of the list - slow - # experimental mark for all tests using pexpect - uses_pexpect - [flake8] max-line-length = 120 extend-ignore = E203