================================================== py.test feature overview ================================================== .. contents:: :local: :depth: 1 mature command line testing tool ==================================================== py.test is a command line tool to collect and run automated tests. It runs well on Linux, Windows and OSX Python 2.4 through to 2.6 versions. It can distribute a single test run to multiple machines. It is used in many projects, ranging from running 10 thousands of tests integrated with buildbot to a few inlined tests on a command line script. .. _`autocollect`: automatically collects and executes tests =============================================== py.test discovers tests automatically by looking at specified directories and its files for common naming patterns. As ``py.test`` operates as a separate cmdline tool you can easily have a command line utility and some tests in the same file. supports many testing practises and methods ================================================================== py.test supports many testing methods conventionally used in the Python community. It runs traditional `unittest.py`_, `doctest.py`_, supports `xUnit style setup`_ and nose_ specific setups and test suites. It offers minimal no-boilerplate model for configuring and deploying tests written as simple Python functions or methods. It also integrates `coverage testing with figleaf`_ or `Javasript unit- and functional testing`_. .. _`Javasript unit- and functional testing`: plugin/oejskit.html .. _`coverage testing with figleaf`: plugin/figleaf.html no-boilerplate test functions with Python =================================================== automatic Python test discovery ------------------------------------ By default, all python modules with a ``test_*.py`` filename are inspected for finding tests: * functions with a name beginning with ``test_`` * classes with a leading ``Test`` name and ``test`` prefixed methods. * ``unittest.TestCase`` subclasses test functions can run with different argument sets ----------------------------------------------------------- py.test offers the unique `funcargs mechanism`_ for setting up and passing project-specific objects to Python test functions. Test Parametrization happens by triggering a call to the same test functions with different argument values. per-test capturing of output, including subprocesses ---------------------------------------------------- By default, ``py.test`` captures all writes to stdout/stderr. Output from ``print`` statements as well as from subprocesses is captured_. When a test fails, the associated captured outputs are shown. This allows you to put debugging print statements in your code without being overwhelmed by all the output that might be generated by tests that do not fail. .. _captured: plugin/capture.html assert with the ``assert`` statement ---------------------------------------- ``py.test`` allows to use the standard python ``assert statement`` for verifying expectations and values in Python tests. For example, you can write the following in your tests:: assert hasattr(x, 'attribute') to state that your object has a certain ``attribute``. In case this assertion fails you will see the value of ``x``. Intermediate values are computed by executing the assert expression a second time. If you execute code with side effects, e.g. read from a file like this:: assert f.read() != '...' then you may get a warning from pytest if that assertions first failed and then succeeded. asserting expected exceptions ---------------------------------------- In order to write assertions about exceptions, you use one of two forms:: py.test.raises(Exception, func, *args, **kwargs) py.test.raises(Exception, "func(*args, **kwargs)") both of which execute the specified function with args and kwargs and asserts that the given ``Exception`` is raised. The reporter will provide you with helpful output in case of failures such as *no exception* or *wrong exception*. information-rich tracebacks, PDB introspection ------------------------------------------------------- .. _`example tracebacks`: http://paste.pocoo.org/show/134814/ A lot of care is taken to present useful failure information and in particular nice and concise Python tracebacks. This is especially useful if you need to regularly look at failures from nightly runs, i.e. are detached from the actual test running session. Here are `example tracebacks`_ for a number of failing test functions. You can modify traceback printing styles through the command line. Using the `--pdb`` option you can automatically activate a PDB `Python debugger`_ when a test fails. advanced skipping of tests ------------------------------- If you want to skip tests you can use ``py.test.skip`` within test or setup functions. Example:: def test_hello(): if sys.platform != "win32": py.test.skip("only win32 supported") You can also use a helper to skip on a failing import:: docutils = py.test.importorskip("docutils") or to skip if a library does not have the right version:: docutils = py.test.importorskip("docutils", minversion="0.3") The version will be read from the specified module's ``__version__`` attribute. .. _`funcargs mechanism`: funcargs.html .. _`unittest.py`: http://docs.python.org/library/unittest.html .. _`doctest.py`: http://docs.python.org/library/doctest.html .. _`xUnit style setup`: xunit_setup.html .. _`pytest_nose`: plugin/nose.html load-balance test runs to multiple CPUs ======================================== For large test suites you can distribute your tests to multiple CPUs by issuing for example:: py.test -n 3 Read more on `distributed testing`_. .. _`distributed testing`: dist.html ad-hoc run tests cross-platform ================================================== py.test supports the sending of tests to remote ssh-accounts, socket servers. It can `ad-hoc run your test on multiple platforms one a single test run`. Ad-hoc means that there are **no installation requirements whatsoever** on the remote side. .. _`ad-hoc run your test on multiple platforms one a single test run`: dist.html#atonce advanced test selection and running modes ========================================================= .. _`selection by keyword`: ``py.test --looponfailing`` allows to run a test suite, memorize all failures and then loop over the failing set of tests until they all pass. It will re-start running the tests when it detects file changes in your project. You can selectively run tests by specifiying a keyword on the command line. Examples:: py.test -k test_simple py.test -k "-test_simple" will run all tests matching (or not matching) the "test_simple" keyword. Note that you need to quote the keyword if "-" is recognized as an indicator for a commandline option. Lastly, you may use:: py.test. -k "test_simple:" which will run all tests after the expression has *matched once*, i.e. all tests that are seen after a test that matches the "test_simple" keyword. By default, all filename parts and class/function names of a test function are put into the set of keywords for a given test. You can specify additional kewords like this:: @py.test.mark.webtest def test_send_http(): ... and then use those keywords to select tests. See the `pytest_keyword`_ plugin for more information. .. _`pytest_keyword`: plugin/keyword.html easy to extend ========================================= py.test has advanced `extension mechanisms`_ with a growing `list of default plugins`_. One can can easily modify or add aspects for for purposes such as: * reporting extensions * customizing collection and execution of tests * running and managing non-python tests * managing domain-specific test state setup .. _`list of default plugins`: plugin/index.html .. _`extension mechanisms`: customize.html#extensions .. _`reStructured Text`: http://docutils.sourceforge.net .. _`Python debugger`: http://docs.python.org/lib/module-pdb.html .. _nose: http://somethingaboutorange.com/mrl/projects/nose/