2010-11-02 07:53:53 +08:00
|
|
|
Some Issues and Questions
|
2009-08-19 01:04:57 +08:00
|
|
|
==================================
|
2009-09-08 15:57:19 +08:00
|
|
|
|
2010-11-02 07:53:53 +08:00
|
|
|
.. note::
|
2010-10-14 01:30:00 +08:00
|
|
|
|
2012-07-02 19:13:48 +08:00
|
|
|
If you don't find an answer here, you may checkout
|
|
|
|
`pytest Q&A at Stackoverflow <http://stackoverflow.com/search?q=pytest>`_
|
|
|
|
or other :ref:`contact channels` to get help.
|
2010-10-14 01:30:00 +08:00
|
|
|
|
2010-11-25 19:11:10 +08:00
|
|
|
On naming, nosetests, licensing and magic
|
2010-10-11 05:45:45 +08:00
|
|
|
------------------------------------------------
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2011-03-04 06:40:38 +08:00
|
|
|
How does py.test relate to nose and unittest?
|
2010-10-11 05:45:45 +08:00
|
|
|
+++++++++++++++++++++++++++++++++++++++++++++++++
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2009-08-19 23:12:02 +08:00
|
|
|
py.test and nose_ share basic philosophy when it comes
|
2011-03-04 06:40:38 +08:00
|
|
|
to running and writing Python tests. In fact, you can run many tests
|
|
|
|
written for nose with py.test. nose_ was originally created
|
2010-07-27 03:15:15 +08:00
|
|
|
as a clone of ``py.test`` when py.test was in the ``0.8`` release
|
2011-03-04 06:40:38 +08:00
|
|
|
cycle. Note that starting with pytest-2.0 support for running unittest
|
2012-06-17 16:59:30 +08:00
|
|
|
test suites is majorly improved.
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2012-06-17 16:59:30 +08:00
|
|
|
how does py.test relate to twisted's trial?
|
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
|
|
|
|
Since some time py.test has builtin support for supporting tests
|
|
|
|
written using trial. It does not itself start a reactor, however,
|
|
|
|
and does not handle Deferreds returned from a test. Someone using
|
|
|
|
these features might eventually write a dedicated ``pytest-twisted``
|
|
|
|
plugin which will surely see strong support from the pytest development
|
|
|
|
team.
|
|
|
|
|
|
|
|
how does py.test work with Django?
|
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2012-06-17 16:59:30 +08:00
|
|
|
In 2012, some work is going into the `pytest-django plugin <http://pypi.python.org/pypi/pytest-django>`_. It substitutes the usage of Django's
|
|
|
|
``manage.py test`` and allows to use all pytest features_ most of which
|
|
|
|
are not available from Django directly.
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2012-07-02 19:13:48 +08:00
|
|
|
.. _features: features.html
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2011-03-04 06:40:38 +08:00
|
|
|
|
2012-06-17 16:59:30 +08:00
|
|
|
What's this "magic" with py.test? (historic notes)
|
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
|
|
|
|
Around 2007 (version ``0.8``) some people thought that py.test
|
|
|
|
was using too much "magic". It had been part of the `pylib`_ which
|
|
|
|
contains a lot of unreleated python library code. Around 2010 there
|
|
|
|
was a major cleanup refactoring, which removed unused or deprecated code
|
|
|
|
and resulted in the new ``pytest`` PyPI package which strictly contains
|
|
|
|
only test-related code. This relese also brought a complete pluginification
|
|
|
|
such that the core is around 300 lines of code and everything else is
|
|
|
|
implemented in plugins. Thus ``pytest`` today is a small, universally runnable
|
|
|
|
and customizable testing framework for Python. Note, however, that
|
|
|
|
``pytest`` uses metaprogramming techniques and reading its source is
|
|
|
|
thus likely not something for Python beginners.
|
|
|
|
|
|
|
|
A second "magic" issue was the assert statement debugging feature.
|
|
|
|
Nowadays, py.test explicitely rewrites assert statements in test modules
|
|
|
|
in order to provide more useful :ref:`assert feedback <assertfeedback>`.
|
|
|
|
This completely avoids previous issues of confusing assertion-reporting.
|
|
|
|
It also means, that you can use Python's ``-O`` optimization without loosing
|
|
|
|
assertions in test modules.
|
|
|
|
|
2012-06-18 17:47:57 +08:00
|
|
|
py.test contains a second mostly obsolete assert debugging technique,
|
|
|
|
invoked via ``--assert=reinterpret``, activated by default on
|
|
|
|
Python-2.5: When an ``assert`` statement fails, py.test re-interprets
|
|
|
|
the expression part to show intermediate values. This technique suffers
|
|
|
|
from a caveat that the rewriting does not: If your expression has side
|
|
|
|
effects (better to avoid them anyway!) the intermediate values may not
|
|
|
|
be the same, confusing the reinterpreter and obfuscating the initial
|
|
|
|
error (this is also explained at the command line if it happens).
|
2012-06-17 16:59:30 +08:00
|
|
|
|
|
|
|
You can also turn off all assertion interaction using the
|
|
|
|
``--assertmode=off`` option.
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2009-08-19 01:04:57 +08:00
|
|
|
.. _`py namespaces`: index.html
|
2009-11-05 10:18:55 +08:00
|
|
|
.. _`py/__init__.py`: http://bitbucket.org/hpk42/py-trunk/src/trunk/py/__init__.py
|
|
|
|
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2012-06-17 16:59:30 +08:00
|
|
|
Why a ``py.test`` instead of a ``pytest`` command?
|
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++
|
|
|
|
|
|
|
|
Some of the reasons are historic, others are practical. ``py.test``
|
|
|
|
used to be part of the ``py`` package which provided several developer
|
|
|
|
utilities, all starting with ``py.<TAB>``, thus providing nice
|
|
|
|
TAB-completion. If
|
|
|
|
you install ``pip install pycmd`` you get these tools from a separate
|
|
|
|
package. These days the command line tool could be called ``pytest``
|
|
|
|
but since many people have gotten used to the old name and there
|
|
|
|
is another tool named "pytest" we just decided to stick with
|
|
|
|
``py.test`` for now.
|
|
|
|
|
2011-09-06 17:43:42 +08:00
|
|
|
Function arguments, parametrized tests and setup
|
2010-10-11 05:45:45 +08:00
|
|
|
-------------------------------------------------------
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2012-07-02 19:13:48 +08:00
|
|
|
.. _funcargs: funcargs.html
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2010-11-02 07:53:53 +08:00
|
|
|
Is using funcarg- versus xUnit setup a style question?
|
2010-10-11 05:45:45 +08:00
|
|
|
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2010-11-07 17:19:58 +08:00
|
|
|
For simple applications and for people experienced with nose_ or
|
2011-03-04 06:40:38 +08:00
|
|
|
unittest-style test setup using `xUnit style setup`_ probably
|
2010-10-11 05:45:45 +08:00
|
|
|
feels natural. For larger test suites, parametrized testing
|
2010-11-18 05:12:16 +08:00
|
|
|
or setup of complex test resources using funcargs_ may feel more natural.
|
2010-07-27 03:15:15 +08:00
|
|
|
Moreover, funcargs are ideal for writing advanced test support
|
|
|
|
code (like e.g. the monkeypatch_, the tmpdir_ or capture_ funcargs)
|
|
|
|
because the support code can register setup/teardown functions
|
|
|
|
in a managed class/module/function scope.
|
2009-11-05 10:18:55 +08:00
|
|
|
|
2012-07-02 19:13:48 +08:00
|
|
|
.. _monkeypatch: monkeypatch.html
|
|
|
|
.. _tmpdir: tmpdir.html
|
|
|
|
.. _capture: capture.html
|
2009-08-19 01:04:57 +08:00
|
|
|
|
|
|
|
.. _`why pytest_pyfuncarg__ methods?`:
|
|
|
|
|
2010-07-27 03:15:15 +08:00
|
|
|
Why the ``pytest_funcarg__*`` name for funcarg factories?
|
2010-10-11 05:45:45 +08:00
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2011-03-04 06:40:38 +08:00
|
|
|
We like `Convention over Configuration`_ and didn't see much point
|
|
|
|
in allowing a more flexible or abstract mechanism. Moreover,
|
2011-12-05 18:10:48 +08:00
|
|
|
it is nice to be able to search for ``pytest_funcarg__MYARG`` in
|
|
|
|
source code and safely find all factory functions for
|
2011-03-04 06:40:38 +08:00
|
|
|
the ``MYARG`` function argument.
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2012-08-13 18:58:08 +08:00
|
|
|
.. note::
|
|
|
|
|
2012-10-05 16:21:35 +08:00
|
|
|
With pytest-2.3 you can use the :ref:`@pytest.fixture` decorator
|
2012-08-13 18:58:08 +08:00
|
|
|
to mark a function as a funcarg factory.
|
|
|
|
|
2009-08-19 01:04:57 +08:00
|
|
|
.. _`Convention over Configuration`: http://en.wikipedia.org/wiki/Convention_over_Configuration
|
|
|
|
|
2010-11-18 05:12:16 +08:00
|
|
|
Can I yield multiple values from a funcarg factory function?
|
2010-10-11 05:45:45 +08:00
|
|
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2010-05-06 02:19:02 +08:00
|
|
|
There are two conceptual reasons why yielding from a factory function
|
2010-07-27 03:15:15 +08:00
|
|
|
is not possible:
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2010-07-27 03:15:15 +08:00
|
|
|
* Calling factories for obtaining test function arguments
|
|
|
|
is part of setting up and running a test. At that
|
2009-08-19 01:04:57 +08:00
|
|
|
point it is not possible to add new test calls to
|
2010-07-27 03:15:15 +08:00
|
|
|
the test collection anymore.
|
2009-08-19 01:04:57 +08:00
|
|
|
|
|
|
|
* If multiple factories yielded values there would
|
2010-07-27 03:15:15 +08:00
|
|
|
be no natural place to determine the combination
|
2009-08-19 01:04:57 +08:00
|
|
|
policy - in real-world examples some combinations
|
2010-07-27 03:15:15 +08:00
|
|
|
often should not run.
|
2009-08-19 01:04:57 +08:00
|
|
|
|
2012-10-05 16:21:35 +08:00
|
|
|
However, with pytest-2.3 you can use the :ref:`@pytest.fixture` decorator
|
2012-08-13 18:58:08 +08:00
|
|
|
and specify ``params`` so that all tests depending on the factory-created
|
|
|
|
resource will run multiple times with different parameters.
|
|
|
|
|
|
|
|
You can also use the `pytest_generate_tests`_ hook to
|
|
|
|
implement the `parametrization scheme of your choice`_.
|
2009-08-19 01:04:57 +08:00
|
|
|
|
|
|
|
.. _`pytest_generate_tests`: test/funcargs.html#parametrizing-tests
|
2010-07-27 03:15:15 +08:00
|
|
|
.. _`parametrization scheme of your choice`: http://tetamap.wordpress.com/2009/05/13/parametrizing-python-tests-generalized/
|
2009-11-02 20:00:48 +08:00
|
|
|
|
|
|
|
py.test interaction with other packages
|
2010-10-11 05:45:45 +08:00
|
|
|
---------------------------------------------------
|
2009-11-02 20:00:48 +08:00
|
|
|
|
2010-07-27 03:15:15 +08:00
|
|
|
Issues with py.test, multiprocess and setuptools?
|
2010-10-11 05:45:45 +08:00
|
|
|
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
2009-11-02 20:00:48 +08:00
|
|
|
|
2010-07-27 03:15:15 +08:00
|
|
|
On windows the multiprocess package will instantiate sub processes
|
2011-02-17 21:46:40 +08:00
|
|
|
by pickling and thus implicitly re-import a lot of local modules.
|
|
|
|
Unfortunately, setuptools-0.6.11 does not ``if __name__=='__main__'``
|
2009-11-02 20:00:48 +08:00
|
|
|
protect its generated command line script. This leads to infinite
|
|
|
|
recursion when running a test that instantiates Processes.
|
|
|
|
|
2010-11-18 05:12:16 +08:00
|
|
|
A good solution is to `install Distribute`_ as a drop-in replacement
|
|
|
|
for setuptools and then re-install ``pytest``. Otherwise you could
|
2009-11-02 21:52:54 +08:00
|
|
|
fix the script that is created by setuptools by inserting an
|
2010-07-27 03:15:15 +08:00
|
|
|
``if __name__ == '__main__'``. Or you can create a "pytest.py"
|
2009-11-02 21:52:54 +08:00
|
|
|
script with this content and invoke that with the python version::
|
|
|
|
|
2010-11-18 05:12:16 +08:00
|
|
|
import pytest
|
2009-11-02 21:52:54 +08:00
|
|
|
if __name__ == '__main__':
|
2010-11-18 05:12:16 +08:00
|
|
|
pytest.main()
|
2009-11-02 20:00:48 +08:00
|
|
|
|
|
|
|
.. _`install distribute`: http://pypi.python.org/pypi/distribute#installation-instructions
|
|
|
|
|
2010-11-01 06:28:31 +08:00
|
|
|
.. include:: links.inc
|