214 lines
6.9 KiB
Plaintext
214 lines
6.9 KiB
Plaintext
|
|
.. highlightlang:: python
|
|
.. _`goodpractises`:
|
|
|
|
Good Integration Practises
|
|
=================================================
|
|
|
|
Work with virtual environments
|
|
-----------------------------------------------------------
|
|
|
|
We recommend to use virtualenv_ environments and use easy_install_
|
|
(or pip_) for installing your application dependencies as well as
|
|
the ``pytest`` package itself. This way you will get a much more reproducible
|
|
environment. A good tool to help you automate test runs against multiple
|
|
dependency configurations or Python interpreters is `tox`_.
|
|
|
|
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
|
|
.. _`buildout`: http://www.buildout.org/
|
|
.. _pip: http://pypi.python.org/pypi/pip
|
|
|
|
Use tox and Continuous Integration servers
|
|
-------------------------------------------------
|
|
|
|
If you frequently release code to the public you
|
|
may want to look into `tox`_, the virtualenv test automation
|
|
tool and its `pytest support <http://codespeak.net/tox/example/pytest.html>`_.
|
|
The basic idea is to generate a JUnitXML file through the ``--junitxml=PATH`` option and have a continuous integration server like Jenkins_ pick it up
|
|
and generate reports.
|
|
|
|
.. _standalone:
|
|
.. _`genscript method`:
|
|
|
|
Create a py.test standalone script
|
|
-------------------------------------------
|
|
|
|
If you are a maintainer or application developer and want others
|
|
to easily run tests you can generate a completely standalone "py.test"
|
|
script::
|
|
|
|
py.test --genscript=runtests.py
|
|
|
|
generates a ``runtests.py`` script which is a fully functional basic
|
|
``py.test`` script, running unchanged under Python2 and Python3.
|
|
You can tell people to download the script and then e.g. run it like this::
|
|
|
|
python runtests.py
|
|
|
|
|
|
.. _`Distribute for installation`: http://pypi.python.org/pypi/distribute#installation-instructions
|
|
.. _`distribute installation`: http://pypi.python.org/pypi/distribute
|
|
|
|
|
|
Integrating with distutils / ``python setup.py test``
|
|
--------------------------------------------------------
|
|
|
|
You can integrate test runs into your distutils or
|
|
setuptools based project. Use the `genscript method`_
|
|
to generate a standalone py.test script::
|
|
|
|
py.test --genscript=runtests.py
|
|
|
|
and make this script part of your distribution and then add
|
|
this to your ``setup.py`` file::
|
|
|
|
from distutils.core import setup, Command
|
|
# you can also import from setuptools
|
|
|
|
class PyTest(Command):
|
|
user_options = []
|
|
def initialize_options(self):
|
|
pass
|
|
def finalize_options(self):
|
|
pass
|
|
def run(self):
|
|
import sys,subprocess
|
|
errno = subprocess.call([sys.executable, 'runtest.py'])
|
|
raise SystemExit(errno)
|
|
setup(
|
|
#...,
|
|
cmdclass = {'test': PyTest},
|
|
#...,
|
|
)
|
|
|
|
If you now type::
|
|
|
|
python setup.py test
|
|
|
|
this will execute your tests using ``runtest.py``. As this is a
|
|
standalone version of ``py.test`` no prior installation whatsoever is
|
|
required for calling the test command. You can also pass additional
|
|
arguments to the subprocess-calls such as your test directory or other
|
|
options.
|
|
|
|
.. _`test discovery`:
|
|
.. _`Python test discovery`:
|
|
|
|
|
|
Integration with setuptools/distribute test commands
|
|
----------------------------------------------------
|
|
|
|
Distribute/Setuptools support test requirements,
|
|
which means its really easy to extend its test command
|
|
to support running a pytest from test requirements::
|
|
|
|
from setuptools.command.test import test as TestCommand
|
|
|
|
class PyTest(TestCommand):
|
|
def finalize_options(self):
|
|
TestCommand.finalize_options(self)
|
|
self.test_args = []
|
|
self.test_suite = True
|
|
def run_tests(self):
|
|
#import here, cause outside the eggs aren't loaded
|
|
import pytest
|
|
pytest.main(self.test_args)
|
|
|
|
setup(
|
|
#...,
|
|
tests_require=['pytest'],
|
|
cmdclass = {'test': pytest},
|
|
)
|
|
|
|
Now if you run::
|
|
|
|
python setup.py test
|
|
|
|
this will download py.test if needed and then run py.test
|
|
as you would expect it to.
|
|
|
|
|
|
Conventions for Python test discovery
|
|
-------------------------------------------------
|
|
|
|
``py.test`` implements the following standard test discovery:
|
|
|
|
* collection starts from the initial command line arguments
|
|
which may be directories, filenames or test ids.
|
|
* recurse into directories, unless they match :confval:`norecursedirs`
|
|
* ``test_*.py`` or ``*_test.py`` files, imported by their `package name`_.
|
|
* ``Test`` prefixed test classes (without an ``__init__`` method)
|
|
* ``test_`` prefixed test functions or methods are test items
|
|
|
|
For examples of how to customize your test discovery :doc:`example/pythoncollection`.
|
|
|
|
Within Python modules, py.test also discovers tests using the standard
|
|
:ref:`unittest.TestCase <unittest.TestCase>` subclassing technique.
|
|
|
|
Choosing a test layout / import rules
|
|
------------------------------------------
|
|
|
|
py.test supports common test layouts:
|
|
|
|
* inlining test directories into your application package, useful if you want to
|
|
keep (unit) tests and actually tested code close together::
|
|
|
|
mypkg/
|
|
__init__.py
|
|
appmodule.py
|
|
...
|
|
test/
|
|
test_app.py
|
|
...
|
|
|
|
* putting tests into an extra directory outside your actual application
|
|
code, useful if you have many functional tests or want to keep
|
|
tests separate from actual application code::
|
|
|
|
mypkg/
|
|
__init__.py
|
|
appmodule.py
|
|
tests/
|
|
test_app.py
|
|
...
|
|
|
|
In both cases you usually need to make sure that ``mypkg`` is importable,
|
|
for example by using the setuptools ``python setup.py develop`` method.
|
|
|
|
You can run your tests by pointing to it::
|
|
|
|
py.test tests/test_app.py # for external test dirs
|
|
py.test mypkg/test/test_app.py # for inlined test dirs
|
|
py.test mypkg # run tests in all below test directories
|
|
py.test # run all tests below current dir
|
|
...
|
|
|
|
.. _`package name`:
|
|
|
|
.. note::
|
|
|
|
If py.test finds a "a/b/test_module.py" test file while
|
|
recursing into the filesystem it determines the import name
|
|
as follows:
|
|
|
|
* find ``basedir`` -- this is the first "upward" (towards the root)
|
|
directory not containing an ``__init__.py``. If both the ``a``
|
|
and ``b`` directories contain an ``__init__.py`` the basedir will
|
|
be the parent dir of ``a``.
|
|
|
|
* perform ``sys.path.insert(0, basedir)`` to make the test module
|
|
importable under the fully qualified import name.
|
|
|
|
* ``import a.b.test_module`` where the path is determined
|
|
by converting path separators ``/`` into "." characters. This means
|
|
you must follow the convention of having directory and file
|
|
names map directly to the import names.
|
|
|
|
The reason for this somewhat evolved importing technique is
|
|
that in larger projects multiple test modules might import
|
|
from each other and thus deriving a canonical import name helps
|
|
to avoid surprises such as a test modules getting imported twice.
|
|
|
|
|
|
.. include:: links.inc
|