label code blocks

This commit is contained in:
boris 2019-08-06 13:25:54 -07:00
parent cf7761f91f
commit 7f90e74e02
23 changed files with 194 additions and 1 deletions

View File

@ -164,5 +164,7 @@ For information about fixtures, see :ref:`fixtures`. To see a complete list of a
You can also interactively ask for help, e.g. by typing on the Python interactive prompt something like::
.. code-block:: python
import pytest
help(pytest)

View File

@ -35,6 +35,8 @@ Rerunning only failures or failures first
First, let's create 50 test invocation of which only 2 fail::
.. code-block:: python
# content of test_50.py
import pytest
@ -185,6 +187,8 @@ pytest ``config`` object. Here is a basic example plugin which
implements a :ref:`fixture` which re-uses previously created state
across pytest invocations::
.. code-block:: python
# content of test_caching.py
import pytest
import time

View File

@ -51,6 +51,8 @@ Using print statements for debugging
One primary benefit of the default capturing of stdout/stderr output
is that you can use print statements for debugging::
.. code-block:: python
# content of test_module.py
def setup_function(function):

View File

@ -457,6 +457,8 @@ Internal classes accessed through ``Node``
Access of ``Module``, ``Function``, ``Class``, ``Instance``, ``File`` and ``Item`` through ``Node`` instances now issue
this warning::
.. code-block:: text
usage of Function.Module is deprecated, please use pytest.Module instead
Users should just ``import pytest`` and access those objects using the ``pytest`` module.

View File

@ -193,6 +193,8 @@ your own fixtures to provide the tests that use them with context.
``doctest_namespace`` is a standard ``dict`` object into which you
place the objects you want to appear in the doctest namespace::
.. code-block:: python
# content of conftest.py
import numpy
@pytest.fixture(autouse=True)
@ -201,6 +203,8 @@ place the objects you want to appear in the doctest namespace::
which can then be used in your doctests directly::
.. code-block:: python
# content of numpy.py
def arange():
"""
@ -221,6 +225,8 @@ Skipping tests dynamically
You can use ``pytest.skip`` to dynamically skip doctests. For example::
.. code-block:: text
>>> import sys, pytest
>>> if sys.platform.startswith('win'):
... pytest.skip('this doctest does not work on Windows')

View File

@ -21,6 +21,8 @@ Let's say we want to execute a test with different computation
parameters and the parameter range shall be determined by a command
line argument. Let's first write a simple (do-nothing) computation test::
.. code-block:: python
# content of test_compute.py
def test_compute(param1):
@ -28,6 +30,8 @@ line argument. Let's first write a simple (do-nothing) computation test::
Now we add a test configuration like this::
.. code-block:: python
# content of conftest.py
def pytest_addoption(parser):
@ -85,6 +89,8 @@ Numbers, strings, booleans and None will have their usual string representation
used in the test ID. For other objects, pytest will make a string based on
the argument name::
.. code-block:: python
# content of test_time.py
import pytest
@ -173,6 +179,8 @@ an add-on from Robert Collins for the standard unittest framework. We
only have to work a bit to construct the correct arguments for pytest's
:py:func:`Metafunc.parametrize`::
.. code-block:: python
# content of test_scenarios.py
def pytest_generate_tests(metafunc):
@ -246,6 +254,8 @@ connections or subprocess only when the actual test is run.
Here is a simple example how you can achieve that, first
the actual test requiring a ``db`` object::
.. code-block:: python
# content of test_backends.py
import pytest
@ -258,6 +268,8 @@ We can now add a test configuration that generates two invocations of
the ``test_db_initialized`` function and also implements a factory that
creates a database object for the actual test invocations::
.. code-block:: python
# content of conftest.py
import pytest
@ -329,6 +341,8 @@ two fixtures: ``x`` and ``y``. Here we give to indirect the list, which contains
fixture ``x``. The indirect parameter will be applied to this argument only, and the value ``a``
will be passed to respective fixture function::
.. code-block:: python
# content of test_indirect_list.py
import pytest
@ -372,6 +386,8 @@ Here is an example ``pytest_generate_tests`` function implementing a
parametrization scheme similar to Michael Foord's `unittest
parametrizer`_ but in a lot less code::
.. code-block:: python
# content of ./test_parametrize.py
import pytest
@ -449,6 +465,8 @@ and get skipped in case the implementation is not importable/available. Let's
say we have a "base" implementation and the other (possibly optimized ones)
need to provide similar results::
.. code-block:: python
# content of conftest.py
import pytest
@ -463,18 +481,24 @@ need to provide similar results::
And then a base implementation of a simple function::
.. code-block:: python
# content of base.py
def func1():
return 1
And an optimized version::
.. code-block:: python
# content of opt1.py
def func1():
return 1.0001
And finally a little test module::
.. code-block:: python
# content of test_module.py
def test_func1(basemod, optmod):
@ -581,6 +605,8 @@ in which some tests raise exceptions and others do not.
It is helpful to define a no-op context manager ``does_not_raise`` to serve
as a complement to ``raises``. For example::
.. code-block:: python
from contextlib import contextmanager
import pytest
@ -606,12 +632,18 @@ while the fourth should raise ``ZeroDivisionError``.
If you're only supporting Python 3.7+, you can simply use ``nullcontext``
to define ``does_not_raise``::
.. code-block:: python
from contextlib import nullcontext as does_not_raise
Or, if you're supporting Python 3.3+ you can use::
.. code-block:: python
from contextlib import ExitStack as does_not_raise
Or, if desired, you can ``pip install contextlib2`` and use::
.. code-block:: python
from contextlib2 import ExitStack as does_not_raise

View File

@ -133,6 +133,8 @@ This would make ``pytest`` look for tests in files that match the ``check_*
.py`` glob-pattern, ``Check`` prefixes in classes, and functions and methods
that match ``*_check``. For example, if we have::
.. code-block:: python
# content of check_myapp.py
class CheckMyApp(object):
def simple_check(self):
@ -240,6 +242,8 @@ imported. Moreover, there may files only importable by a specific python
version. For such cases you can dynamically define files to be ignored by
listing them in a ``conftest.py`` file::
.. code-block:: python
# content of conftest.py
import sys
@ -249,6 +253,8 @@ listing them in a ``conftest.py`` file::
and then if you have a module file like this::
.. code-block:: python
# content of pkg/module_py2.py
def test_only_on_python2():
try:
@ -258,6 +264,8 @@ and then if you have a module file like this::
and a ``setup.py`` dummy file like this::
.. code-block:: python
# content of setup.py
0/0 # will raise exception if imported
@ -297,6 +305,8 @@ The following example ``conftest.py`` ignores the file ``setup.py`` and in
addition all files that end with ``*_py2.py`` when executed with a Python 3
interpreter::
.. code-block:: python
# content of conftest.py
import sys

View File

@ -7,6 +7,8 @@ function which walks all collected tests and looks
if their test class defines a ``callme`` method and
calls it::
.. code-block:: python
# content of conftest.py
import pytest
@ -26,6 +28,8 @@ calls it::
test classes may now define a ``callme`` method which
will be called ahead of running any tests::
.. code-block:: python
# content of test_module.py
class TestHello(object):

View File

@ -17,6 +17,8 @@ After pulling the code into your development space using some
flavor of version control and (optionally) setting up a virtualenv
you will want to run::
.. code-block:: bash
cd <repository>
pip install -e . # Environment dependent alternatives include
# 'python setup.py develop' and 'conda develop'

View File

@ -51,6 +51,8 @@ the fixture object. Fixture functions are registered by marking them with
self-contained test module containing a fixture and a test function
using it::
.. code-block:: python
# content of ./test_smtpsimple.py
import pytest
@ -182,6 +184,8 @@ The next example puts the fixture function into a separate ``conftest.py`` file
so that tests from multiple test modules in the directory can
access the fixture function::
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@ -195,6 +199,8 @@ result by listing the name ``smtp_connection`` as an input parameter in any
test or fixture function (in or below the directory where ``conftest.py`` is
located)::
.. code-block:: python
# content of test_module.py
def test_ehlo(smtp_connection):
@ -479,6 +485,8 @@ to introspect the "requesting" test function, class or module context.
Further extending the previous ``smtp_connection`` fixture example, let's
read an optional server URL from the test module which uses our fixture::
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@ -505,6 +513,8 @@ again, nothing much has changed:
Let's quickly create another test module that actually sets the
server URL in its module namespace::
.. code-block:: python
# content of test_anothersmtp.py
smtpserver = "mail.python.org" # will be read by smtp fixture
@ -542,6 +552,8 @@ This function can then be called multiple times in the test.
Factories can have parameters as needed::
.. code-block:: python
@pytest.fixture
def make_customer_record():
@ -561,6 +573,8 @@ Factories can have parameters as needed::
If the data created by the factory requires managing, the fixture can take care of that::
.. code-block:: python
@pytest.fixture
def make_customer_record():
@ -600,6 +614,8 @@ Extending the previous example, we can flag the fixture to create two
to run twice. The fixture function gets access to each parameter
through the special :py:class:`request <FixtureRequest>` object::
.. code-block:: python
# content of conftest.py
import pytest
import smtplib
@ -692,6 +708,8 @@ make a string based on the argument name. It is possible to customise
the string used in a test ID for a certain fixture value by using the
``ids`` keyword argument::
.. code-block:: python
# content of test_ids.py
import pytest
@ -756,6 +774,8 @@ that they can be used with :ref:`@pytest.mark.parametrize <@pytest.mark.parametr
Example::
.. code-block:: python
# content of test_fixture_marks.py
import pytest
@pytest.fixture(params=[0, 1, pytest.param(2, marks=pytest.mark.skip)])
@ -794,6 +814,8 @@ many projects. As a simple example, we can extend the previous example
and instantiate an object ``app`` where we stick the already defined
``smtp_connection`` resource into it::
.. code-block:: python
# content of test_appsetup.py
import pytest
@ -856,6 +878,8 @@ The following example uses two parametrized fixtures, one of which is
scoped on a per-module basis, and all the functions perform ``print`` calls
to show the setup/teardown flow::
.. code-block:: python
# content of test_module.py
import pytest
@ -955,6 +979,8 @@ directory. Here is how you can use the standard `tempfile
achieve it. We separate the creation of the fixture into a conftest.py
file::
.. code-block:: python
# content of conftest.py
import pytest
@ -968,6 +994,8 @@ file::
and declare its use in a test module via a ``usefixtures`` marker::
.. code-block:: python
# content of test_setenv.py
import os
import pytest
@ -1052,6 +1080,8 @@ begin/rollback/commit architecture and we want to automatically surround
each test method by a transaction and a rollback. Here is a dummy
self-contained implementation of this idea::
.. code-block:: python
# content of test_db_transact.py
import pytest
@ -1118,6 +1148,8 @@ you want to make available in your project without having it generally
active. The canonical way to do that is to put the transact definition
into a conftest.py file **without** using ``autouse``::
.. code-block:: python
# content of conftest.py
@pytest.fixture
def transact(request, db):
@ -1127,6 +1159,8 @@ into a conftest.py file **without** using ``autouse``::
and then e.g. have a TestClass using it by declaring the need::
.. code-block:: python
@pytest.mark.usefixtures("transact")
class TestClass(object):
def test_method1(self):

View File

@ -23,6 +23,8 @@ the ``request.cached_setup()`` helper to manage caching of
resources. Here is a basic example how we could implement
a per-session Database object::
.. code-block:: python
# content of conftest.py
class Database(object):
def __init__(self):
@ -70,6 +72,8 @@ Instead of calling cached_setup() with a cache scope, you can use the
:ref:`@pytest.fixture <pytest.fixture>` decorator and directly state
the scope::
.. code-block:: python
@pytest.fixture(scope="session")
def db(request):
# factory will only be invoked once per session -
@ -92,6 +96,8 @@ or implement a ``pytest_generate_tests`` hook to perform
parametrization, i.e. calling a test multiple times with different value
sets. pytest-2.3 introduces a decorator for use on the factory itself::
.. code-block:: python
@pytest.fixture(params=["mysql", "pg"])
def db(request):
... # use request.param
@ -109,6 +115,8 @@ parametrized via
Of course it's perfectly fine to combine parametrization and scoping::
.. code-block:: python
@pytest.fixture(scope="session", params=["mysql", "pg"])
def db(request):
if request.param == "mysql":
@ -130,6 +138,8 @@ When using the ``@fixture`` decorator the name of the function
denotes the name under which the resource can be accessed as a function
argument::
.. code-block:: python
@pytest.fixture()
def db(request):
...
@ -139,6 +149,8 @@ The name under which the funcarg resource can be requested is ``db``.
You can still use the "old" non-decorator way of specifying funcarg factories
aka::
.. code-block:: python
def pytest_funcarg__db(request):
...

View File

@ -37,6 +37,8 @@ Create your first test
Create a simple test function with just four lines of code::
.. code-block:: python
# content of test_sample.py
def func(x):
return x + 1
@ -85,6 +87,8 @@ Assert that a certain exception is raised
Use the :ref:`raises <assertraises>` helper to assert that some code raises an exception::
.. code-block:: python
# content of test_sysexit.py
import pytest
def f():
@ -107,6 +111,8 @@ Group multiple tests in a class
Once you develop multiple tests, you may want to group them into a class. pytest makes it easy to create a class containing more than one test::
.. code-block:: python
# content of test_class.py
class TestClass(object):
def test_one(self):
@ -144,6 +150,8 @@ Request a unique temporary directory for functional tests
``pytest`` provides `Builtin fixtures/function arguments <https://docs.pytest.org/en/latest/builtin.html>`_ to request arbitrary resources, like a unique temporary directory::
.. code-block:: python
# content of test_tmpdir.py
def test_needsfiles(tmpdir):
print(tmpdir)

View File

@ -14,12 +14,16 @@ This ensures your code and dependencies are isolated from your system Python ins
Next, place a ``setup.py`` file in the root of your package with the following minimum content::
.. code-block:: python
from setuptools import setup, find_packages
setup(name="PACKAGENAME", packages=find_packages())
Where ``PACKAGENAME`` is the name of your package. You can then install your package in "editable" mode by running from the same directory::
.. code-block:: bash
pip install -e .
which lets you change your source code (both tests and application) and rerun tests at will.
@ -62,6 +66,8 @@ Putting tests into an extra directory outside your actual application code
might be useful if you have many functional tests or for other reasons want
to keep tests separate from actual application code (often a good idea)::
.. code-block:: text
setup.py
mypkg/
__init__.py
@ -94,6 +100,8 @@ be imported as ``test_app`` and ``test_view`` top-level modules by adding ``test
If you need to have test modules with the same name, you might add ``__init__.py`` files to your
``tests`` folder and subfolders, changing them to packages::
.. code-block:: text
setup.py
mypkg/
...
@ -116,6 +124,8 @@ because you want to test the *installed* version of your package, not the local
In this situation, it is **strongly** suggested to use a ``src`` layout where application root package resides in a
sub-directory of your root::
.. code-block:: text
setup.py
src/
mypkg/
@ -142,6 +152,8 @@ Inlining test directories into your application package
is useful if you have direct relation between tests and application modules and
want to distribute them along with your application::
.. code-block:: text
setup.py
mypkg/
__init__.py
@ -155,6 +167,8 @@ want to distribute them along with your application::
In this scheme, it is easy to run your tests using the ``--pyargs`` option::
.. code-block:: bash
pytest --pyargs mypkg
``pytest`` will discover where ``mypkg`` is installed and collect tests from there.

View File

@ -72,6 +72,8 @@ caplog fixture
Inside tests it is possible to change the log level for the captured log
messages. This is supported by the ``caplog`` fixture::
.. code-block:: python
def test_foo(caplog):
caplog.set_level(logging.INFO)
pass
@ -80,6 +82,8 @@ By default the level is set on the root logger,
however as a convenience it is also possible to set the log level of any
logger::
.. code-block:: python
def test_foo(caplog):
caplog.set_level(logging.CRITICAL, logger='root.baz')
pass
@ -89,6 +93,8 @@ The log levels set are restored automatically at the end of the test.
It is also possible to use a context manager to temporarily change the log
level inside a ``with`` block::
.. code-block:: python
def test_bar(caplog):
with caplog.at_level(logging.INFO):
pass
@ -96,6 +102,8 @@ level inside a ``with`` block::
Again, by default the level of the root logger is affected but the level of any
logger can be changed instead with::
.. code-block:: python
def test_bar(caplog):
with caplog.at_level(logging.CRITICAL, logger='root.baz'):
pass
@ -104,6 +112,8 @@ Lastly all the logs sent to the logger during the test run are made available on
the fixture in the form of both the ``logging.LogRecord`` instances and the final log text.
This is useful for when you want to assert on the contents of a message::
.. code-block:: python
def test_baz(caplog):
func_under_test()
for record in caplog.records:
@ -117,6 +127,8 @@ You can also resort to ``record_tuples`` if all you want to do is to ensure,
that certain messages have been logged under a given logger name with a given
severity and message::
.. code-block:: python
def test_foo(caplog):
logging.getLogger().info('boo %s', 'arg')
@ -126,6 +138,8 @@ severity and message::
You can call ``caplog.clear()`` to reset the captured log records in a test::
.. code-block:: python
def test_something_with_clearing_records(caplog):
some_method_that_creates_log_records()
caplog.clear()

View File

@ -10,6 +10,8 @@ For writing your own plugins, please refer to :ref:`writing-plugins`.
Installing a third party plugin can be easily done with ``pip``::
.. code-block:: bash
pip install pytest-NAME
pip uninstall pytest-NAME
@ -97,6 +99,8 @@ Finding out which plugins are active
If you want to find out which plugins are active in your
environment you can type::
.. code-block:: bash
pytest --trace-config
and will get an extended test header which shows activated plugins
@ -110,6 +114,8 @@ Deactivating / unregistering a plugin by name
You can prevent plugins from loading or unregister them::
.. code-block:: bash
pytest -p no:NAME
This means that any subsequent try to activate/load the named

View File

@ -24,6 +24,8 @@ Consider this file and directory layout::
When executing::
.. code-block:: bash
pytest root/
@ -56,6 +58,8 @@ Consider this file and directory layout::
When executing::
.. code-block:: bash
pytest root/
pytest will find ``foo/bar/tests/test_foo.py`` and realize it is NOT part of a package given that

View File

@ -471,6 +471,8 @@ test plugins.
To use it, include in your top-most ``conftest.py`` file::
.. code-block:: python
pytest_plugins = 'pytester'
@ -1001,6 +1003,8 @@ passed multiple times. The expected format is ``name=value``. For example::
issuing ``pytest test_hello.py`` actually means::
.. code-block:: bash
pytest --maxfail=2 -rf test_hello.py
Default is to add no options.

View File

@ -182,12 +182,16 @@ Skipping on a missing import dependency
You can use the following helper at module level
or within a test or test setup function::
.. code-block:: python
docutils = pytest.importorskip("docutils")
If ``docutils`` cannot be imported here, this will lead to a
skip outcome of the test. You can also skip based on the
version number of a library::
.. code-block:: python
docutils = pytest.importorskip("docutils", minversion="0.3")
The version will be read from the specified
@ -225,6 +229,8 @@ XFail: mark test functions as expected to fail
You can use the ``xfail`` marker to indicate that you
expect a test to fail::
.. code-block:: python
@pytest.mark.xfail
def test_function():
...

View File

@ -92,6 +92,8 @@ created in the `base temporary directory`_.
``tmpdir`` is a `py.path.local`_ object which offers ``os.path`` methods
and more. Here is an example test usage::
.. code-block:: python
# content of test_tmpdir.py
import os
def test_create_file(tmpdir):

View File

@ -12,6 +12,8 @@ the test suite to take full advantage of pytest's features.
To run an existing ``unittest``-style test suite using ``pytest``, type::
.. code-block:: bash
pytest tests
@ -80,6 +82,8 @@ let's jump-start into an example that integrates a pytest ``db_class``
fixture, setting up a class-cached database object, and then reference
it from a unittest-style test::
.. code-block:: python
# content of conftest.py
# we define a fixture function below and it will be "used" by
@ -105,6 +109,8 @@ code and allows re-use of the fixture by a minimal reference, the fixture
name. So let's write an actual ``unittest.TestCase`` class using our
fixture definition::
.. code-block:: python
# content of test_unittest_db.py
import unittest
@ -181,6 +187,8 @@ pre-initialized ``samplefile.ini``. Our ``initdir`` fixture itself uses
the pytest builtin :ref:`tmpdir <tmpdir>` fixture to delegate the
creation of a per-test temporary directory::
.. code-block:: python
# content of test_unittest_cleandir.py
import pytest
import unittest

View File

@ -756,16 +756,22 @@ Calling pytest from Python code
You can invoke ``pytest`` from Python code directly::
.. code-block:: python
pytest.main()
this acts as if you would call "pytest" from the command line.
It will not raise ``SystemExit`` but return the exitcode instead.
You can pass in options and arguments::
.. code-block:: python
pytest.main(['-x', 'mytestdir'])
You can specify additional plugins to ``pytest.main``::
.. code-block:: python
# content of myinvoke.py
import pytest
class MyPlugin(object):

View File

@ -180,6 +180,7 @@ This will ignore all warnings of type ``DeprecationWarning`` where the start of
the regular expression ``".*U.*mode is deprecated"``.
.. note::
If warnings are configured at the interpreter level, using
the `PYTHONWARNINGS <https://docs.python.org/3/using/cmdline.html#envvar-PYTHONWARNINGS>`_ environment variable or the
``-W`` command-line option, pytest will not configure any filters by default.
@ -279,6 +280,8 @@ argument ``match`` to assert that the exception matches a text or regex::
You can also call ``pytest.warns`` on a function or code string::
.. code-block:: python
pytest.warns(expected_warning, func, *args, **kwargs)
pytest.warns(expected_warning, "func(*args, **kwargs)")
@ -411,7 +414,7 @@ These warnings might be filtered using the same builtin mechanisms used to filte
Please read our :ref:`backwards-compatibility` to learn how we proceed about deprecating and eventually removing
features.
The following warning types ares used by pytest and are part of the public API:
The following warning types are used by pytest and are part of the public API:
.. autoclass:: pytest.PytestWarning

View File

@ -29,6 +29,8 @@ If you have multiple test functions and test classes in a single
module you can optionally implement the following fixture methods
which will usually be called once for all the functions::
.. code-block:: python
def setup_module(module):
""" setup any state specific to the execution of the given module."""
@ -45,6 +47,8 @@ Class level setup/teardown
Similarly, the following methods are called at class level before
and after all test methods of the class are called::
.. code-block:: python
@classmethod
def setup_class(cls):
""" setup any state specific to the execution of the given class (which
@ -62,6 +66,8 @@ Method and function level setup/teardown
Similarly, the following methods are called around each method invocation::
.. code-block:: python
def setup_method(self, method):
""" setup any state tied to the execution of the given method in a
class. setup_method is invoked for every test method of a class.
@ -77,6 +83,8 @@ As of pytest-3.0, the ``method`` parameter is optional.
If you would rather define test functions directly at module level
you can also use the following functions to implement fixtures::
.. code-block:: python
def setup_function(function):
""" setup any state tied to the execution of the given function.
Invoked for every test function in the module.