test_ok1/doc/assert.txt

The writing and reporting of assertions in tests
==================================================

.. _`assert with the assert statement`:

assert with the ``assert`` statement
---------------------------------------------------------

``py.test`` allows you to use the standard python ``assert`` for verifying
expectations and values in Python tests.  For example, you can write the
following::

    # content of test_assert1.py
    def f():
        return 3

    def test_function():
        assert f() == 4

to assert that your function returns a certain value. If this assertion fails
you will see the value of ``x``::

    $ py.test test_assert1.py
    =========================== test session starts ============================
    platform linux2 -- Python 2.6.6 -- pytest-2.0.3
    collecting ... collected 1 items
    
    test_assert1.py F
    
    ================================= FAILURES =================================
    ______________________________ test_function _______________________________
    
        def test_function():
    >       assert f() == 4
    E       assert 3 == 4
    E        +  where 3 = f()
    
    test_assert1.py:5: AssertionError
    ========================= 1 failed in 0.02 seconds =========================

py.test has support for showing the values of the most common subexpressions
including calls, attributes, comparisons, and binary and unary operators. This
allows you to use the idiomatic python constructs without boilerplate code while
not losing introspection information.

See :ref:`assert-details` for more information on assertion introspection.


assertions about expected exceptions
------------------------------------------

In order to write assertions about raised exceptions, you can use
``pytest.raises`` as a context manager like this::

    import pytest
    with pytest.raises(ZeroDivisionError):
        1 / 0

and if you need to have access to the actual exception info you may use::

    with pytest.raises(RuntimeError) as excinfo:
        def f():
            f()
        f()

    # do checks related to excinfo.type, excinfo.value, excinfo.traceback

If you want to write test code that works on Python2.4 as well,
you may also use two other ways to test for an expected exception::

    pytest.raises(ExpectedException, func, *args, **kwargs)
    pytest.raises(ExpectedException, "func(*args, **kwargs)")

both of which execute the specified function with args and kwargs and
asserts that the given ``ExpectedException`` is raised.  The reporter will
provide you with helpful output in case of failures such as *no
exception* or *wrong exception*.

.. _newreport:

Making use of context-sensitive comparisons
-------------------------------------------------

.. versionadded:: 2.0

py.test has rich support for providing context-sensitive information
when it encounters comparisons.  For example::

    # content of test_assert2.py

    def test_set_comparison():
        set1 = set("1308")
        set2 = set("8035")
        assert set1 == set2

if you run this module::

    $ py.test test_assert2.py
    =========================== test session starts ============================
    platform linux2 -- Python 2.6.6 -- pytest-2.0.3
    collecting ... collected 1 items
    
    test_assert2.py F
    
    ================================= FAILURES =================================
    ___________________________ test_set_comparison ____________________________
    
        def test_set_comparison():
            set1 = set("1308")
            set2 = set("8035")
    >       assert set1 == set2
    E       assert set(['0', '1', '3', '8']) == set(['0', '3', '5', '8'])
    E         Extra items in the left set:
    E         '1'
    E         Extra items in the right set:
    E         '5'
    
    test_assert2.py:5: AssertionError
    ========================= 1 failed in 0.03 seconds =========================

Special comparisons are done for a number of cases:

* comparing long strings: a context diff is shown
* comparing long sequences: first failing indices
* comparing dicts: different entries

See the :ref:`reporting demo <tbreportdemo>` for many more examples.


.. _assert-details:

Assertion introspection details
-------------------------------

Reporting details about the failing assertion is achieved either by rewriting
assert statements before they are run or re-evaluating the assert expression and
recording the intermediate values. Which technique is used depends on the
location of the assert, py.test's configuration, and Python version being used
to run py.test.

By default, if the Python version is greater than or equal to 2.6, py.test
rewrites assert statements in test modules. Rewritten assert statements put
introspection information into the assertion failure message. Note py.test only
rewrites test modules directly discovered by its test collection process, so
asserts in supporting modules will not be rewritten.

.. note::

   py.test rewrites test modules as it collects tests from them. It does this by
   writing a new pyc file which Python loads when the test module is
   imported. If the module has already been loaded (it is in sys.modules),
   though, Python will not load the rewritten module. This means if a test
   module imports another test module which has not already been rewritten, then
   py.test will not be able to rewrite the second module.

If an assert statement has not been rewritten or the Python version is less than
2.6, py.test falls back on assert reinterpretation. In assert reinterpretation,
py.test walks the frame of the function containing the assert statement to
discover sub-expression results of the failing assert statement. You can force
py.test to always use assertion reinterpretation by passing the
``--assertmode=old`` option.

Assert reinterpretation has a caveat not present with assert rewriting: If
evaluating the assert expression has side effects you may get a warning that the
intermediate values could not be determined safely.  A common example of this
issue is an assertion which reads from a file::

        assert f.read() != '...'

If this assertion fails then the re-evaluation will probably succeed!
This is because ``f.read()`` will return an empty string when it is
called the second time during the re-evaluation.  However, it is
easy to rewrite the assertion and avoid any trouble::

        content = f.read()
        assert content != '...'

All assert introspection can be turned off by passing ``--assertmode=off``.

.. versionadded:: 2.1
   Add assert rewriting as an alternate introspection technique.

.. versionchanged:: 2.1
   Introduce the ``--assertmode`` option. Deprecate ``--no-assert`` and
   ``--nomagic``.

..
    Defining your own comparison
    ----------------------------------------------