close issue240 - rework "good practises" document and discuss
discuss the two common test directory layouts in more detail, including an explicit note on how it interacts with PEP420-namespace packages.
This commit is contained in:
holger krekel
parent
dd0da4643a
commit
cd8e69e33c
|
|
@ -26,6 +26,10 @@ Unreleased
|
|||
with repeated same values (sometimes useful to to test if calling
|
||||
a second time works as with the first time).
|
||||
|
||||
- close issue240 - document precisely how pytest module importing
|
||||
works, discuss the two common test directory layouts, and how it
|
||||
interacts with PEP420-namespace packages.
|
||||
|
||||
- fix issue246 fix finalizer order to be LIFO on independent fixtures
|
||||
depending on a parametrized higher-than-function scoped fixture.
|
||||
(was quite some effort so please bear with the complexity of this sentence :)
|
||||
|
|
|
|||
|
|
@ -8,24 +8,151 @@ Good Integration Practises
|
|||
Work with virtual environments
|
||||
-----------------------------------------------------------
|
||||
|
||||
We recommend to use virtualenv_ environments and use easy_install_
|
||||
(or pip_) for installing your application dependencies as well as
|
||||
the ``pytest`` package itself. This way you will get a much more reproducible
|
||||
environment. A good tool to help you automate test runs against multiple
|
||||
dependency configurations or Python interpreters is `tox`_.
|
||||
We recommend to use virtualenv_ environments and use pip_
|
||||
(or easy_install_) for installing your application and any dependencies
|
||||
as well as the ``pytest`` package itself. This way you will get an isolated
|
||||
and reproducible environment. Given you have installed virtualenv_
|
||||
and execute it from the command line, here is an example session for unix
|
||||
or windows::
|
||||
|
||||
virtualenv . # create a virtualenv directory in the current directory
|
||||
|
||||
source bin/activate # on unix
|
||||
|
||||
scripts/activate # on Windows
|
||||
|
||||
We can now install pytest::
|
||||
|
||||
pip install pytest
|
||||
|
||||
Due to the ``activate`` step above the ``pip`` will come from
|
||||
the virtualenv directory and install any package into the isolated
|
||||
virtual environment.
|
||||
|
||||
Choosing a test layout / import rules
|
||||
------------------------------------------
|
||||
|
||||
py.test supports two common test layouts:
|
||||
|
||||
* putting tests into an extra directory outside your actual application
|
||||
code, useful if you have many functional tests or for other reasons
|
||||
want to keep tests separate from actual application code (often a good
|
||||
idea)::
|
||||
|
||||
setup.py # your distutils/setuptools Python package metadata
|
||||
mypkg/
|
||||
__init__.py
|
||||
appmodule.py
|
||||
tests/
|
||||
test_app.py
|
||||
...
|
||||
|
||||
|
||||
* inlining test directories into your application package, useful if you
|
||||
have direct relation between (unit-)test and application modules and
|
||||
want to distribute your tests along with your application::
|
||||
|
||||
setup.py # your distutils/setuptools Python package metadata
|
||||
mypkg/
|
||||
__init__.py
|
||||
appmodule.py
|
||||
...
|
||||
test/
|
||||
test_app.py
|
||||
...
|
||||
|
||||
Important notes relating to both schemes:
|
||||
|
||||
- **make sure that "mypkg" is importable**, for example by typing once::
|
||||
|
||||
pip install -e . # install package using setup.py in editable mode
|
||||
|
||||
- **avoid "__init__.py" files in your test directories**.
|
||||
This way your tests can run easily against an installed version
|
||||
of ``mypkg``, independently from if the installed version contains
|
||||
the tests or not.
|
||||
|
||||
- With inlined tests you might put ``__init__.py`` into test
|
||||
directories and make them installable as part of your application.
|
||||
Using the ``py.test --pyargs mypkg`` invocation pytest will
|
||||
discover where mypkg is installed and collect tests from there.
|
||||
With the "external" test you can still distribute tests but they
|
||||
will not be installed or become importable.
|
||||
|
||||
Typically you can run tests by pointing to test directories or modules::
|
||||
|
||||
py.test tests/test_app.py # for external test dirs
|
||||
py.test mypkg/test/test_app.py # for inlined test dirs
|
||||
py.test mypkg # run tests in all below test directories
|
||||
py.test # run all tests below current dir
|
||||
...
|
||||
|
||||
Because of the above ``editable install`` mode you can change your
|
||||
source code (both tests and the app) and rerun tests at will.
|
||||
Once you are done with your work, you can `use tox`_ to make sure
|
||||
that the package is really correct and tests pass in all
|
||||
required configurations.
|
||||
|
||||
.. note::
|
||||
|
||||
You can use Python3 namespace packages (PEP420) for your application
|
||||
but pytest will still perform `package name`_ discovery based on the
|
||||
presence of ``__init__.py`` files. If you use one of the above
|
||||
two recommended file system layouts but leave away the ``__init__.py``
|
||||
files it should just work on Python3.3 and above. When using
|
||||
"inlined tests", however, you will need to use absolute imports for
|
||||
getting at your application code because the test modules will be
|
||||
imported directly, without any application context. The latter allows
|
||||
your tests to run against an installed version of your package.
|
||||
|
||||
.. _`package name`:
|
||||
|
||||
.. note::
|
||||
|
||||
If py.test finds a "a/b/test_module.py" test file while
|
||||
recursing into the filesystem it determines the import name
|
||||
as follows:
|
||||
|
||||
* determine ``basedir``: this is the first "upward" (towards the root)
|
||||
directory not containing an ``__init__.py``. If e.g. both ``a``
|
||||
and ``b`` contain an ``__init__.py`` file then the parent directory
|
||||
of ``a`` will become the ``basedir``.
|
||||
|
||||
* perform ``sys.path.insert(0, basedir)`` to make the test module
|
||||
importable under the fully qualified import name.
|
||||
|
||||
* ``import a.b.test_module`` where the path is determined
|
||||
by converting path separators ``/`` into "." characters. This means
|
||||
you must follow the convention of having directory and file
|
||||
names map directly to the import names.
|
||||
|
||||
The reason for this somewhat evolved importing technique is
|
||||
that in larger projects multiple test modules might import
|
||||
from each other and thus deriving a canonical import name helps
|
||||
to avoid surprises such as a test modules getting imported twice.
|
||||
|
||||
|
||||
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
|
||||
.. _`buildout`: http://www.buildout.org/
|
||||
.. _pip: http://pypi.python.org/pypi/pip
|
||||
|
||||
.. _`use tox`:
|
||||
|
||||
Use tox and Continuous Integration servers
|
||||
-------------------------------------------------
|
||||
|
||||
If you frequently release code to the public you
|
||||
may want to look into `tox`_, the virtualenv test automation
|
||||
tool and its `pytest support <http://testrun.org/tox/latest/example/pytest.html>`_.
|
||||
The basic idea is to generate a JUnitXML file through the ``--junitxml=PATH`` option and have a continuous integration server like Jenkins_ pick it up
|
||||
and generate reports.
|
||||
If you frequently release code and want to make sure that your actual
|
||||
package passes all tests you may want to look into `tox`_, the
|
||||
virtualenv test automation tool and its `pytest support
|
||||
<http://testrun.org/tox/latest/example/pytest.html>`_.
|
||||
Tox helps you to setup virtualenv environments with pre-defined
|
||||
dependencies and then executing a pre-configured test command with
|
||||
options. It will run tests against the installed package and not
|
||||
against your source code checkout, helping to detect packaging
|
||||
glitches.
|
||||
|
||||
If you want to use Jenkins_ you can use the ``--junitxml=PATH`` option
|
||||
to create a JUnitXML file that Jenkins_ can pick up and generate reports.
|
||||
|
||||
.. _standalone:
|
||||
.. _`genscript method`:
|
||||
|
|
@ -33,21 +160,19 @@ and generate reports.
|
|||
Create a py.test standalone script
|
||||
-------------------------------------------
|
||||
|
||||
If you are a maintainer or application developer and want others
|
||||
to easily run tests you can generate a completely standalone "py.test"
|
||||
script::
|
||||
If you are a maintainer or application developer and want people
|
||||
who don't deal with python much to easily run tests you may generate
|
||||
a standalone "py.test" script::
|
||||
|
||||
py.test --genscript=runtests.py
|
||||
|
||||
generates a ``runtests.py`` script which is a fully functional basic
|
||||
This generates a ``runtests.py`` script which is a fully functional basic
|
||||
``py.test`` script, running unchanged under Python2 and Python3.
|
||||
You can tell people to download the script and then e.g. run it like this::
|
||||
|
||||
python runtests.py
|
||||
|
||||
|
||||
|
||||
|
||||
Integrating with distutils / ``python setup.py test``
|
||||
--------------------------------------------------------
|
||||
|
||||
|
|
@ -93,8 +218,9 @@ options.
|
|||
Integration with setuptools test commands
|
||||
----------------------------------------------------
|
||||
|
||||
Setuptools supports writing our own Test command for invoking
|
||||
pytest::
|
||||
Setuptools supports writing our own Test command for invoking pytest.
|
||||
Most often it is better to use tox_ instead, but here is how you can
|
||||
get started with setuptools integration::
|
||||
|
||||
from setuptools.command.test import test as TestCommand
|
||||
import sys
|
||||
|
|
@ -143,69 +269,4 @@ For examples of how to customize your test discovery :doc:`example/pythoncollect
|
|||
Within Python modules, py.test also discovers tests using the standard
|
||||
:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
|
||||
|
||||
Choosing a test layout / import rules
|
||||
------------------------------------------
|
||||
|
||||
py.test supports common test layouts:
|
||||
|
||||
* inlining test directories into your application package, useful if you want to
|
||||
keep (unit) tests and actually tested code close together::
|
||||
|
||||
mypkg/
|
||||
__init__.py
|
||||
appmodule.py
|
||||
...
|
||||
test/
|
||||
test_app.py
|
||||
...
|
||||
|
||||
* putting tests into an extra directory outside your actual application
|
||||
code, useful if you have many functional tests or want to keep
|
||||
tests separate from actual application code::
|
||||
|
||||
mypkg/
|
||||
__init__.py
|
||||
appmodule.py
|
||||
tests/
|
||||
test_app.py
|
||||
...
|
||||
|
||||
In both cases you usually need to make sure that ``mypkg`` is importable,
|
||||
for example by using the setuptools ``python setup.py develop`` method.
|
||||
|
||||
You can run your tests by pointing to it::
|
||||
|
||||
py.test tests/test_app.py # for external test dirs
|
||||
py.test mypkg/test/test_app.py # for inlined test dirs
|
||||
py.test mypkg # run tests in all below test directories
|
||||
py.test # run all tests below current dir
|
||||
...
|
||||
|
||||
.. _`package name`:
|
||||
|
||||
.. note::
|
||||
|
||||
If py.test finds a "a/b/test_module.py" test file while
|
||||
recursing into the filesystem it determines the import name
|
||||
as follows:
|
||||
|
||||
* find ``basedir`` -- this is the first "upward" (towards the root)
|
||||
directory not containing an ``__init__.py``. If both the ``a``
|
||||
and ``b`` directories contain an ``__init__.py`` the basedir will
|
||||
be the parent dir of ``a``.
|
||||
|
||||
* perform ``sys.path.insert(0, basedir)`` to make the test module
|
||||
importable under the fully qualified import name.
|
||||
|
||||
* ``import a.b.test_module`` where the path is determined
|
||||
by converting path separators ``/`` into "." characters. This means
|
||||
you must follow the convention of having directory and file
|
||||
names map directly to the import names.
|
||||
|
||||
The reason for this somewhat evolved importing technique is
|
||||
that in larger projects multiple test modules might import
|
||||
from each other and thus deriving a canonical import name helps
|
||||
to avoid surprises such as a test modules getting imported twice.
|
||||
|
||||
|
||||
.. include:: links.inc
|
||||
|
|
|
|||
Loading…
Reference in New Issue