From c6f90c25e311e195eeede499911b0d7987d583a5 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:20:40 -0200 Subject: [PATCH 01/13] Remove finalize_options override from goodpractices This is not required in latest versions of `setuptools`, and `self.test_args` is a read-only attribute in some of the versions of the 18.X series. Fix #1134 --- doc/en/goodpractises.rst | 5 ----- 1 file changed, 5 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index dba521c50..cef754ec9 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -276,11 +276,6 @@ get started with setuptools integration:: TestCommand.initialize_options(self) self.pytest_args = [] - def finalize_options(self): - TestCommand.finalize_options(self) - self.test_args = [] - self.test_suite = True - def run_tests(self): #import here, cause outside the eggs aren't loaded import pytest From b1f6dc23da3f7603d5b20a9f53af3b17d7c6b657 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:27:27 -0200 Subject: [PATCH 02/13] Moved "conventions for Python test discovery to the top" --- doc/en/goodpractises.rst | 46 +++++++++++++++++++++------------------- 1 file changed, 24 insertions(+), 22 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index cef754ec9..0d46b56a3 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -4,6 +4,30 @@ Good Integration Practices ================================================= + +.. _`test discovery`: +.. _`Python test discovery`: + +Conventions for Python test discovery +------------------------------------------------- + +``pytest`` implements the following standard test discovery: + +* collection starts from paths specified in :confval:`testpaths` if configured, + otherwise from initial command line arguments which may be directories, + filenames or test ids. If :confval:`testpaths` is not configured and no + directories or files were given in the command line, start collection from + the current directory. +* recurse into directories, unless they match :confval:`norecursedirs` +* ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. +* ``Test`` prefixed test classes (without an ``__init__`` method) +* ``test_`` prefixed test functions or methods are test items + +For examples of how to customize your test discovery :doc:`example/pythoncollection`. + +Within Python modules, ``pytest`` also discovers tests using the standard +:ref:`unittest.TestCase ` subclassing technique. + Work with virtual environments ----------------------------------------------------------- @@ -306,27 +330,5 @@ is equivalent to running ``py.test --durations=5``. For a more powerful solution, take a look at the `pytest-runner `_ plugin. -.. _`test discovery`: -.. _`Python test discovery`: - -Conventions for Python test discovery -------------------------------------------------- - -``pytest`` implements the following standard test discovery: - -* collection starts from paths specified in :confval:`testpaths` if configured, - otherwise from initial command line arguments which may be directories, - filenames or test ids. If :confval:`testpaths` is not configured and no - directories or files were given in the command line, start collection from - the current directory. -* recurse into directories, unless they match :confval:`norecursedirs` -* ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. -* ``Test`` prefixed test classes (without an ``__init__`` method) -* ``test_`` prefixed test functions or methods are test items - -For examples of how to customize your test discovery :doc:`example/pythoncollection`. - -Within Python modules, ``pytest`` also discovers tests using the standard -:ref:`unittest.TestCase ` subclassing technique. .. include:: links.inc From 4f17d56ecbfd82982fe5fb0cd7f99bf4da93691a Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:30:08 -0200 Subject: [PATCH 03/13] Move deprecated genscript method to the bottom of the document --- doc/en/goodpractises.rst | 49 ++++++++++++++++++++-------------------- 1 file changed, 24 insertions(+), 25 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 0d46b56a3..b227aefc0 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -175,31 +175,6 @@ glitches. If you want to use Jenkins_ you can use the ``--junitxml=PATH`` option to create a JUnitXML file that Jenkins_ can pick up and generate reports. -.. _standalone: -.. _`genscript method`: - -(deprecated) Create a pytest standalone script ------------------------------------------------ - -If you are a maintainer or application developer and want people -who don't deal with python much to easily run tests you may generate -a standalone ``pytest`` script:: - - py.test --genscript=runtests.py - -This generates a ``runtests.py`` script which is a fully functional basic -``pytest`` script, running unchanged under Python2 and Python3. -You can tell people to download the script and then e.g. run it like this:: - - python runtests.py - -.. note:: - - You must have pytest and its dependencies installed as an sdist, not - as wheels because genscript need the source code for generating a - standalone script. - - Integrating with setuptools / ``python setup.py test`` / ``pytest-runner`` -------------------------------------------------------------------------- @@ -330,5 +305,29 @@ is equivalent to running ``py.test --durations=5``. For a more powerful solution, take a look at the `pytest-runner `_ plugin. +.. _standalone: +.. _`genscript method`: + +(deprecated) Create a pytest standalone script +----------------------------------------------- + +If you are a maintainer or application developer and want people +who don't deal with python much to easily run tests you may generate +a standalone ``pytest`` script:: + + py.test --genscript=runtests.py + +This generates a ``runtests.py`` script which is a fully functional basic +``pytest`` script, running unchanged under Python2 and Python3. +You can tell people to download the script and then e.g. run it like this:: + + python runtests.py + +.. note:: + + You must have pytest and its dependencies installed as an sdist, not + as wheels because genscript need the source code for generating a + standalone script. + .. include:: links.inc From 0704fcacd750db9f13a84a94a176966c06ad5f48 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:32:07 -0200 Subject: [PATCH 04/13] Removed setuptools/genscript session --- doc/en/goodpractises.rst | 48 ++-------------------------------------- 1 file changed, 2 insertions(+), 46 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index b227aefc0..617333650 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -209,52 +209,6 @@ required for calling the test command. You can also pass additional arguments to py.test such as your test directory or other options using ``--addopts``. -Integrating with setuptools / ``python setup.py test`` / ``genscript`` ----------------------------------------------------------------------- - -You can integrate test runs into your -setuptools based project. Use the `genscript method`_ -to generate a standalone ``pytest`` script:: - - py.test --genscript=runtests.py - -and make this script part of your distribution and then add -this to your ``setup.py`` file:: - - from distutils.core import setup, Command - # you can also import from setuptools - - class PyTest(Command): - user_options = [] - def initialize_options(self): - pass - - def finalize_options(self): - pass - - def run(self): - import subprocess - import sys - errno = subprocess.call([sys.executable, 'runtests.py']) - raise SystemExit(errno) - - - setup( - #..., - cmdclass = {'test': PyTest}, - #..., - ) - -If you now type:: - - python setup.py test - -this will execute your tests using ``runtests.py``. As this is a -standalone version of ``pytest`` no prior installation whatsoever is -required for calling the test command. You can also pass additional -arguments to the subprocess-calls such as your test directory or other -options. - Integration with setuptools test commands ---------------------------------------------------- @@ -311,6 +265,8 @@ is equivalent to running ``py.test --durations=5``. (deprecated) Create a pytest standalone script ----------------------------------------------- +.. deprecated:: 2.7 + If you are a maintainer or application developer and want people who don't deal with python much to easily run tests you may generate a standalone ``pytest`` script:: From d851a8fd078a3cedba5e68bce5f7c0ee0d26780a Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:43:11 -0200 Subject: [PATCH 05/13] Fixed some formatting --- doc/en/goodpractises.rst | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 617333650..d196d3d5f 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -31,8 +31,8 @@ Within Python modules, ``pytest`` also discovers tests using the standard Work with virtual environments ----------------------------------------------------------- -We recommend to use virtualenv_ environments and use pip_ -(or easy_install_) for installing your application and any dependencies +We recommend to use virtualenv_ environments and pip_ + for installing your application and any dependencies as well as the ``pytest`` package itself. This way you will get an isolated and reproducible environment. Given you have installed virtualenv_ and execute it from the command line, here is an example session for unix @@ -159,8 +159,8 @@ required configurations. .. _`use tox`: -Use tox and Continuous Integration servers -------------------------------------------------- +Tox +--- If you frequently release code and want to make sure that your actual package passes all tests you may want to look into `tox`_, the @@ -180,9 +180,11 @@ Integrating with setuptools / ``python setup.py test`` / ``pytest-runner`` -------------------------------------------------------------------------- You can integrate test runs into your setuptools based project -with pytest-runner. +with the `pytest-runner `_ plugin. -Add this to ``setup.py`` file:: +Add this to ``setup.py`` file: + +.. code-block:: python from distutils.core import setup # you can also import from setuptools @@ -194,7 +196,10 @@ Add this to ``setup.py`` file:: #..., ) -And create an alias into ``setup.cfg``file:: +And create an alias into ``setup.cfg``file: + + +.. code-block:: ini [aliases] test=pytest @@ -210,12 +215,13 @@ arguments to py.test such as your test directory or other options using ``--addopts``. -Integration with setuptools test commands ----------------------------------------------------- +Manual Integration +^^^^^^^^^^^^^^^^^^ -Setuptools supports writing our own Test command for invoking pytest. -Most often it is better to use tox_ instead, but here is how you can -get started with setuptools integration:: +If for some reason you don't want/can't use ``pytest-runner``, you can write +your own setuptools Test command for invoking pytest. + +.. code-block:: python import sys @@ -254,10 +260,6 @@ using the ``--pytest-args`` or ``-a`` command-line option. For example:: is equivalent to running ``py.test --durations=5``. -.. seealso:: - - For a more powerful solution, take a look at the - `pytest-runner `_ plugin. .. _standalone: .. _`genscript method`: From 69ef0ab189e4f4e5ba0bda750025c1c18bcefb6b Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:47:36 -0200 Subject: [PATCH 06/13] Merged virtual env into the Tox section Nowadays virtualenv use is widespread so we don't need to devote a how-to section in pytest's docs --- doc/en/goodpractises.rst | 34 ++++++++-------------------------- 1 file changed, 8 insertions(+), 26 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index d196d3d5f..94631476e 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -28,29 +28,6 @@ For examples of how to customize your test discovery :doc:`example/pythoncollect Within Python modules, ``pytest`` also discovers tests using the standard :ref:`unittest.TestCase ` subclassing technique. -Work with virtual environments ------------------------------------------------------------ - -We recommend to use virtualenv_ environments and pip_ - for installing your application and any dependencies -as well as the ``pytest`` package itself. This way you will get an isolated -and reproducible environment. Given you have installed virtualenv_ -and execute it from the command line, here is an example session for unix -or windows:: - - virtualenv . # create a virtualenv directory in the current directory - - source bin/activate # on unix - - scripts/activate # on Windows - -We can now install pytest:: - - pip install pytest - -Due to the ``activate`` step above the ``pip`` will come from -the virtualenv directory and install any package into the isolated -virtual environment. Choosing a test layout / import rules ------------------------------------------ @@ -160,7 +137,12 @@ required configurations. .. _`use tox`: Tox ---- +------ + +For development, we recommend to use virtualenv_ environments and pip_ + for installing your application and any dependencies +as well as the ``pytest`` package itself. This ensures your code and +dependencies are isolated from the system Python installation. If you frequently release code and want to make sure that your actual package passes all tests you may want to look into `tox`_, the @@ -172,8 +154,8 @@ options. It will run tests against the installed package and not against your source code checkout, helping to detect packaging glitches. -If you want to use Jenkins_ you can use the ``--junitxml=PATH`` option -to create a JUnitXML file that Jenkins_ can pick up and generate reports. +Continuous integration services such as Jenkins_ can make use of the +``--junitxml=PATH`` option to create a JUnitXML file and generate reports. Integrating with setuptools / ``python setup.py test`` / ``pytest-runner`` From 095abfd0352a5b67cc51e6371dea80071d00e9be Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:52:22 -0200 Subject: [PATCH 07/13] Fix formatting errors --- doc/en/goodpractises.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 94631476e..fe3f41bd7 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -140,7 +140,7 @@ Tox ------ For development, we recommend to use virtualenv_ environments and pip_ - for installing your application and any dependencies +for installing your application and any dependencies as well as the ``pytest`` package itself. This ensures your code and dependencies are isolated from the system Python installation. @@ -178,7 +178,8 @@ Add this to ``setup.py`` file: #..., ) -And create an alias into ``setup.cfg``file: + +And create an alias into ``setup.cfg`` file: .. code-block:: ini From 5040dde0c546f28a439af4e0c598bab613d32863 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Tue, 1 Dec 2015 22:57:01 -0200 Subject: [PATCH 08/13] Fix genscript deprecation version and document reasons for such --- doc/en/goodpractises.rst | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index fe3f41bd7..2359c562c 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -250,7 +250,21 @@ is equivalent to running ``py.test --durations=5``. (deprecated) Create a pytest standalone script ----------------------------------------------- -.. deprecated:: 2.7 +.. deprecated:: 2.8 + +.. note:: + + ``genscript`` has been deprecated because: + + * It cannot support plugins, rendering its usefulness extremely limited; + * Tooling has become much better since ``genscript`` was introduced; + * It is possible to build a zipped ``pytest`` application without the + shortcomings above (this will be documented in more detail in a later + version of this documentation). + + There's no planned version in which this command will be removed + at the moment of this writing, but its use is discouraged for new + applications. If you are a maintainer or application developer and want people who don't deal with python much to easily run tests you may generate From d9b78f2a9568a39471f149d7ce667abc9422c077 Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Wed, 2 Dec 2015 18:30:12 -0200 Subject: [PATCH 09/13] Remove reference to distutils --- doc/en/goodpractises.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 2359c562c..944fe6caf 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -168,8 +168,7 @@ Add this to ``setup.py`` file: .. code-block:: python - from distutils.core import setup - # you can also import from setuptools + from setuptools import setup setup( #..., From 464e16decaf7f2dcd551ec2e79f8aa6e8834d89a Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Wed, 2 Dec 2015 18:33:53 -0200 Subject: [PATCH 10/13] Removed incorrect note about genscript requiring wheels --- doc/en/goodpractises.rst | 6 ------ 1 file changed, 6 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 944fe6caf..7fef93ae1 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -277,11 +277,5 @@ You can tell people to download the script and then e.g. run it like this:: python runtests.py -.. note:: - - You must have pytest and its dependencies installed as an sdist, not - as wheels because genscript need the source code for generating a - standalone script. - .. include:: links.inc From 9475cd3fb839eba17f9eafba445c47859d30a50a Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Wed, 2 Dec 2015 18:39:32 -0200 Subject: [PATCH 11/13] Replace "--assertmode=off" by "--assert=plain" Fix #1214 --- doc/en/faq.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/en/faq.rst b/doc/en/faq.rst index f91c953f9..dbcaf3fd4 100644 --- a/doc/en/faq.rst +++ b/doc/en/faq.rst @@ -76,7 +76,7 @@ be the same, confusing the reinterpreter and obfuscating the initial error (this is also explained at the command line if it happens). You can also turn off all assertion interaction using the -``--assertmode=off`` option. +``--assert=plain`` option. .. _`py namespaces`: index.html .. _`py/__init__.py`: http://bitbucket.org/hpk42/py-trunk/src/trunk/py/__init__.py From bb4771cedf0b624891ea9a04049bd4e87daec27c Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Thu, 3 Dec 2015 00:13:16 -0200 Subject: [PATCH 12/13] Remove promise about documenting how to create a zipped pytest --- doc/en/goodpractises.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index 7fef93ae1..ae93e0c2a 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -258,8 +258,7 @@ is equivalent to running ``py.test --durations=5``. * It cannot support plugins, rendering its usefulness extremely limited; * Tooling has become much better since ``genscript`` was introduced; * It is possible to build a zipped ``pytest`` application without the - shortcomings above (this will be documented in more detail in a later - version of this documentation). + shortcomings above. There's no planned version in which this command will be removed at the moment of this writing, but its use is discouraged for new From 8c74bb0d25e7fbff970d7a53989ad1a83b3843fc Mon Sep 17 00:00:00 2001 From: Bruno Oliveira Date: Thu, 3 Dec 2015 01:01:34 -0200 Subject: [PATCH 13/13] Improve description on how pytest starts test collection in goodpractises --- doc/en/customize.rst | 2 +- doc/en/goodpractises.rst | 8 +++----- 2 files changed, 4 insertions(+), 6 deletions(-) diff --git a/doc/en/customize.rst b/doc/en/customize.rst index 1c1655697..34e319c24 100644 --- a/doc/en/customize.rst +++ b/doc/en/customize.rst @@ -162,7 +162,7 @@ Builtin configuration file options .. versionadded:: 2.8 Sets list of directories that should be searched for tests when - no specific directories or files are given in the command line when + no specific directories, files or test ids are given in the command line when executing pytest from the :ref:`rootdir ` directory. Useful when all project tests are in a known location to speed up test collection and to avoid picking up undesired tests by accident. diff --git a/doc/en/goodpractises.rst b/doc/en/goodpractises.rst index ae93e0c2a..b2c34bae9 100644 --- a/doc/en/goodpractises.rst +++ b/doc/en/goodpractises.rst @@ -13,11 +13,9 @@ Conventions for Python test discovery ``pytest`` implements the following standard test discovery: -* collection starts from paths specified in :confval:`testpaths` if configured, - otherwise from initial command line arguments which may be directories, - filenames or test ids. If :confval:`testpaths` is not configured and no - directories or files were given in the command line, start collection from - the current directory. +* If no arguments are specified then collection starts from :confval:`testpaths` + (if configured) or the current directory. Alternatively, command line arguments + can be used in any combination of directories, file names or node ids. * recurse into directories, unless they match :confval:`norecursedirs` * ``test_*.py`` or ``*_test.py`` files, imported by their `test package name`_. * ``Test`` prefixed test classes (without an ``__init__`` method)