jiuyuan
/
test_ok1
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

close issue240 - rework "good practises" document and discuss 

discuss the two common test directory layouts in more detail, including
an explicit note on how it interacts with PEP420-namespace packages.
This commit is contained in:
holger krekel 2013-12-10 14:54:13 +01:00
parent dd0da4643a
commit cd8e69e33c
2 changed files with 148 additions and 83 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
CHANGELOG
View File

@ -26,6 +26,10 @@ Unreleased
with repeated same values (sometimes useful to to test if calling with repeated same values (sometimes useful to to test if calling
a second time works as with the first time). a second time works as with the first time).
- close issue240 - document precisely how pytest module importing
works, discuss the two common test directory layouts, and how it
interacts with PEP420-namespace packages.
- fix issue246 fix finalizer order to be LIFO on independent fixtures - fix issue246 fix finalizer order to be LIFO on independent fixtures
depending on a parametrized higher-than-function scoped fixture. depending on a parametrized higher-than-function scoped fixture.
(was quite some effort so please bear with the complexity of this sentence :) (was quite some effort so please bear with the complexity of this sentence :)

227
doc/en/goodpractises.txt
View File

@ -8,24 +8,151 @@ Good Integration Practises
Work with virtual environments Work with virtual environments
----------------------------------------------------------- -----------------------------------------------------------
We recommend to use virtualenv_ environments and use easy_install_ We recommend to use virtualenv_ environments and use pip_
(or pip_) for installing your application dependencies as well as (or easy_install_) for installing your application and any dependencies
the ``pytest`` package itself. This way you will get a much more reproducible as well as the ``pytest`` package itself. This way you will get an isolated
environment. A good tool to help you automate test runs against multiple and reproducible environment. Given you have installed virtualenv_
dependency configurations or Python interpreters is `tox`_. and execute it from the command line, here is an example session for unix
or windows::
virtualenv . # create a virtualenv directory in the current directory
source bin/activate # on unix
scripts/activate # on Windows
We can now install pytest::
pip install pytest
Due to the ``activate`` step above the ``pip`` will come from
the virtualenv directory and install any package into the isolated
virtual environment.
Choosing a test layout / import rules
------------------------------------------
py.test supports two common test layouts:
* putting tests into an extra directory outside your actual application
code, useful if you have many functional tests or for other reasons
want to keep tests separate from actual application code (often a good
idea)::
setup.py # your distutils/setuptools Python package metadata
mypkg/
__init__.py
appmodule.py
tests/
test_app.py
...
* inlining test directories into your application package, useful if you
have direct relation between (unit-)test and application modules and
want to distribute your tests along with your application::
setup.py # your distutils/setuptools Python package metadata
mypkg/
__init__.py
appmodule.py
...
test/
test_app.py
...
Important notes relating to both schemes:
- **make sure that "mypkg" is importable**, for example by typing once::
pip install -e . # install package using setup.py in editable mode
- **avoid "__init__.py" files in your test directories**.
This way your tests can run easily against an installed version
of ``mypkg``, independently from if the installed version contains
the tests or not.
- With inlined tests you might put ``__init__.py`` into test
directories and make them installable as part of your application.
Using the ``py.test --pyargs mypkg`` invocation pytest will
discover where mypkg is installed and collect tests from there.
With the "external" test you can still distribute tests but they
will not be installed or become importable.
Typically you can run tests by pointing to test directories or modules::
py.test tests/test_app.py # for external test dirs
py.test mypkg/test/test_app.py # for inlined test dirs
py.test mypkg # run tests in all below test directories
py.test # run all tests below current dir
...
Because of the above ``editable install`` mode you can change your
source code (both tests and the app) and rerun tests at will.
Once you are done with your work, you can `use tox`_ to make sure
that the package is really correct and tests pass in all
required configurations.
.. note::
You can use Python3 namespace packages (PEP420) for your application
but pytest will still perform `package name`_ discovery based on the
presence of ``__init__.py`` files. If you use one of the above
two recommended file system layouts but leave away the ``__init__.py``
files it should just work on Python3.3 and above. When using
"inlined tests", however, you will need to use absolute imports for
getting at your application code because the test modules will be
imported directly, without any application context. The latter allows
your tests to run against an installed version of your package.
.. _`package name`:
.. note::
If py.test finds a "a/b/test_module.py" test file while
recursing into the filesystem it determines the import name
as follows:
* determine ``basedir``: this is the first "upward" (towards the root)
directory not containing an ``__init__.py``. If e.g. both ``a``
and ``b`` contain an ``__init__.py`` file then the parent directory
of ``a`` will become the ``basedir``.
* perform ``sys.path.insert(0, basedir)`` to make the test module
importable under the fully qualified import name.
* ``import a.b.test_module`` where the path is determined
by converting path separators ``/`` into "." characters. This means
you must follow the convention of having directory and file
names map directly to the import names.
The reason for this somewhat evolved importing technique is
that in larger projects multiple test modules might import
from each other and thus deriving a canonical import name helps
to avoid surprises such as a test modules getting imported twice.
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv .. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
.. _`buildout`: http://www.buildout.org/ .. _`buildout`: http://www.buildout.org/
.. _pip: http://pypi.python.org/pypi/pip .. _pip: http://pypi.python.org/pypi/pip
.. _`use tox`:
Use tox and Continuous Integration servers Use tox and Continuous Integration servers
------------------------------------------------- -------------------------------------------------
If you frequently release code to the public you If you frequently release code and want to make sure that your actual
may want to look into `tox`_, the virtualenv test automation package passes all tests you may want to look into `tox`_, the
tool and its `pytest support <http://testrun.org/tox/latest/example/pytest.html>`_. virtualenv test automation tool and its `pytest support
The basic idea is to generate a JUnitXML file through the ``--junitxml=PATH`` option and have a continuous integration server like Jenkins_ pick it up <http://testrun.org/tox/latest/example/pytest.html>`_.
and generate reports. Tox helps you to setup virtualenv environments with pre-defined
dependencies and then executing a pre-configured test command with
options. It will run tests against the installed package and not
against your source code checkout, helping to detect packaging
glitches.
If you want to use Jenkins_ you can use the ``--junitxml=PATH`` option
to create a JUnitXML file that Jenkins_ can pick up and generate reports.
.. _standalone: .. _standalone:
.. _`genscript method`: .. _`genscript method`:
@ -33,21 +160,19 @@ and generate reports.
Create a py.test standalone script Create a py.test standalone script
------------------------------------------- -------------------------------------------
If you are a maintainer or application developer and want others If you are a maintainer or application developer and want people
to easily run tests you can generate a completely standalone "py.test" who don't deal with python much to easily run tests you may generate
script:: a standalone "py.test" script::
py.test --genscript=runtests.py py.test --genscript=runtests.py
generates a ``runtests.py`` script which is a fully functional basic This generates a ``runtests.py`` script which is a fully functional basic
``py.test`` script, running unchanged under Python2 and Python3. ``py.test`` script, running unchanged under Python2 and Python3.
You can tell people to download the script and then e.g. run it like this:: You can tell people to download the script and then e.g. run it like this::
python runtests.py python runtests.py
Integrating with distutils / ``python setup.py test`` Integrating with distutils / ``python setup.py test``
-------------------------------------------------------- --------------------------------------------------------
@ -93,8 +218,9 @@ options.
Integration with setuptools test commands Integration with setuptools test commands
---------------------------------------------------- ----------------------------------------------------
Setuptools supports writing our own Test command for invoking Setuptools supports writing our own Test command for invoking pytest.
pytest:: Most often it is better to use tox_ instead, but here is how you can
get started with setuptools integration::
from setuptools.command.test import test as TestCommand from setuptools.command.test import test as TestCommand
import sys import sys
@ -143,69 +269,4 @@ For examples of how to customize your test discovery :doc:`example/pythoncollect
Within Python modules, py.test also discovers tests using the standard Within Python modules, py.test also discovers tests using the standard
:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique. :ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
Choosing a test layout / import rules
------------------------------------------
py.test supports common test layouts:
* inlining test directories into your application package, useful if you want to
keep (unit) tests and actually tested code close together::
mypkg/
__init__.py
appmodule.py
...
test/
test_app.py
...
* putting tests into an extra directory outside your actual application
code, useful if you have many functional tests or want to keep
tests separate from actual application code::
mypkg/
__init__.py
appmodule.py
tests/
test_app.py
...
In both cases you usually need to make sure that ``mypkg`` is importable,
for example by using the setuptools ``python setup.py develop`` method.
You can run your tests by pointing to it::
py.test tests/test_app.py # for external test dirs
py.test mypkg/test/test_app.py # for inlined test dirs
py.test mypkg # run tests in all below test directories
py.test # run all tests below current dir
...
.. _`package name`:
.. note::
If py.test finds a "a/b/test_module.py" test file while
recursing into the filesystem it determines the import name
as follows:
* find ``basedir`` -- this is the first "upward" (towards the root)
directory not containing an ``__init__.py``. If both the ``a``
and ``b`` directories contain an ``__init__.py`` the basedir will
be the parent dir of ``a``.
* perform ``sys.path.insert(0, basedir)`` to make the test module
importable under the fully qualified import name.
* ``import a.b.test_module`` where the path is determined
by converting path separators ``/`` into "." characters. This means
you must follow the convention of having directory and file
names map directly to the import names.
The reason for this somewhat evolved importing technique is
that in larger projects multiple test modules might import
from each other and thus deriving a canonical import name helps
to avoid surprises such as a test modules getting imported twice.
.. include:: links.inc .. include:: links.inc