Review doctest docs (#5183)

Review doctest docs
This commit is contained in:
Bruno Oliveira 2019-04-29 08:02:59 -03:00 committed by GitHub
commit 204004c8b8
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 122 additions and 85 deletions

View File

@ -10,29 +10,63 @@ can change the pattern by issuing:
pytest --doctest-glob='*.rst'
on the command line. Since version ``2.9``, ``--doctest-glob``
can be given multiple times in the command-line.
on the command line. ``--doctest-glob`` can be given multiple times in the command-line.
.. versionadded:: 3.1
If you then have a text file like this:
You can specify the encoding that will be used for those doctest files
using the ``doctest_encoding`` ini option:
.. code-block:: text
.. code-block:: ini
# content of test_example.txt
# content of pytest.ini
[pytest]
doctest_encoding = latin1
hello this is a doctest
>>> x = 3
>>> x
3
The default encoding is UTF-8.
then you can just invoke ``pytest`` directly:
You can also trigger running of doctests
from docstrings in all python modules (including regular
python test modules):
.. code-block:: pytest
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 1 item
test_example.txt . [100%]
========================= 1 passed in 0.12 seconds =========================
By default, pytest will collect ``test*.txt`` files looking for doctest directives, but you
can pass additional globs using the ``--doctest-glob`` option (multi-allowed).
In addition to text files, you can also execute doctests directly from docstrings of your classes
and functions, including from test modules:
.. code-block:: python
# content of mymodule.py
def something():
""" a doctest in a docstring
>>> something()
42
"""
return 42
.. code-block:: bash
pytest --doctest-modules
$ pytest --doctest-modules
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR
collected 2 items
mymodule.py . [ 50%]
test_example.txt . [100%]
========================= 2 passed in 0.12 seconds =========================
You can make these changes permanent in your project by
putting them into a pytest.ini file like this:
@ -43,57 +77,37 @@ putting them into a pytest.ini file like this:
[pytest]
addopts = --doctest-modules
If you then have a text file like this:
.. note::
.. code-block:: text
The builtin pytest doctest supports only ``doctest`` blocks, but if you are looking
for more advanced checking over *all* your documentation,
including doctests, ``.. codeblock:: python`` Sphinx directive support,
and any other examples your documentation may include, you may wish to
consider `Sybil <https://sybil.readthedocs.io/en/latest/index.html>`__.
It provides pytest integration out of the box.
# content of example.rst
hello this is a doctest
>>> x = 3
>>> x
3
Encoding
--------
and another like this::
The default encoding is **UTF-8**, but you can specify the encoding
that will be used for those doctest files using the
``doctest_encoding`` ini option:
# content of mymodule.py
def something():
""" a doctest in a docstring
>>> something()
42
"""
return 42
.. code-block:: ini
then you can just invoke ``pytest`` without command line options:
# content of pytest.ini
[pytest]
doctest_encoding = latin1
.. code-block:: pytest
Using 'doctest' options
-----------------------
$ pytest
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-4.x.y, py-1.x.y, pluggy-0.x.y
cachedir: $PYTHON_PREFIX/.pytest_cache
rootdir: $REGENDOC_TMPDIR, inifile: pytest.ini
collected 1 item
The standard ``doctest`` module provides some `options <https://docs.python.org/3/library/doctest.html#option-flags>`__
to configure the strictness of doctest tests. In pytest, you can enable those flags using the
configuration file.
mymodule.py . [100%]
========================= 1 passed in 0.12 seconds =========================
It is possible to use fixtures using the ``getfixture`` helper:
.. code-block:: text
# content of example.rst
>>> tmp = getfixture('tmpdir')
>>> ...
>>>
Also, :ref:`usefixtures` and :ref:`autouse` fixtures are supported
when executing text doctest files.
The standard ``doctest`` module provides some setting flags to configure the
strictness of doctest tests. In pytest, you can enable those flags using the
configuration file. To make pytest ignore trailing whitespaces and ignore
For example, to make pytest ignore trailing whitespaces and ignore
lengthy exception stack traces you can just write:
.. code-block:: ini
@ -110,16 +124,7 @@ Python 3 unchanged:
* ``ALLOW_BYTES``: when enabled, the ``b`` prefix is stripped from byte strings
in expected doctest output.
As with any other option flag, these flags can be enabled in ``pytest.ini`` using
the ``doctest_optionflags`` ini option:
.. code-block:: ini
[pytest]
doctest_optionflags = ALLOW_UNICODE ALLOW_BYTES
Alternatively, it can be enabled by an inline comment in the doc test
Alternatively, options can be enabled by an inline comment in the doc test
itself:
.. code-block:: rst
@ -128,7 +133,7 @@ itself:
>>> get_unicode_greeting() # doctest: +ALLOW_UNICODE
'Hello'
By default, pytest would report only the first failure for a given doctest. If
By default, pytest would report only the first failure for a given doctest. If
you want to continue the test even when you have failures, do:
.. code-block:: bash
@ -136,12 +141,50 @@ you want to continue the test even when you have failures, do:
pytest --doctest-modules --doctest-continue-on-failure
Output format
-------------
You can change the diff output format on failure for your doctests
by using one of standard doctest modules format in options
(see :data:`python:doctest.REPORT_UDIFF`, :data:`python:doctest.REPORT_CDIFF`,
:data:`python:doctest.REPORT_NDIFF`, :data:`python:doctest.REPORT_ONLY_FIRST_FAILURE`):
.. code-block:: bash
pytest --doctest-modules --doctest-report none
pytest --doctest-modules --doctest-report udiff
pytest --doctest-modules --doctest-report cdiff
pytest --doctest-modules --doctest-report ndiff
pytest --doctest-modules --doctest-report only_first_failure
pytest-specific features
------------------------
Some features are provided to make writing doctests easier or with better integration with
your existing test suite. Keep in mind however that by using those features you will make
your doctests incompatible with the standard ``doctests`` module.
Using fixtures
^^^^^^^^^^^^^^
It is possible to use fixtures using the ``getfixture`` helper:
.. code-block:: text
# content of example.rst
>>> tmp = getfixture('tmpdir')
>>> ...
>>>
Also, :ref:`usefixtures` and :ref:`autouse` fixtures are supported
when executing text doctest files.
.. _`doctest_namespace`:
The 'doctest_namespace' fixture
-------------------------------
.. versionadded:: 3.0
'doctest_namespace' fixture
^^^^^^^^^^^^^^^^^^^^^^^^^^^
The ``doctest_namespace`` fixture can be used to inject items into the
namespace in which your doctests run. It is intended to be used within
@ -171,20 +214,14 @@ Note that like the normal ``conftest.py``, the fixtures are discovered in the di
Meaning that if you put your doctest with your source code, the relevant conftest.py needs to be in the same directory tree.
Fixtures will not be discovered in a sibling directory tree!
Output format
-------------
Skipping tests dynamically
^^^^^^^^^^^^^^^^^^^^^^^^^^
.. versionadded:: 3.0
.. versionadded:: 4.4
You can change the diff output format on failure for your doctests
by using one of standard doctest modules format in options
(see :data:`python:doctest.REPORT_UDIFF`, :data:`python:doctest.REPORT_CDIFF`,
:data:`python:doctest.REPORT_NDIFF`, :data:`python:doctest.REPORT_ONLY_FIRST_FAILURE`):
You can use ``pytest.skip`` to dynamically skip doctests. For example::
.. code-block:: bash
pytest --doctest-modules --doctest-report none
pytest --doctest-modules --doctest-report udiff
pytest --doctest-modules --doctest-report cdiff
pytest --doctest-modules --doctest-report ndiff
pytest --doctest-modules --doctest-report only_first_failure
>>> import sys, pytest
>>> if sys.platform.startswith('win'):
... pytest.skip('this doctest does not work on Windows')
...