major refinements to documentation and examples
--HG-- branch : trunk
This commit is contained in:
parent
9925ac883e
commit
2e4391d28e
|
@ -13,8 +13,6 @@ py.test reference documentation
|
||||||
skipping.txt
|
skipping.txt
|
||||||
mark.txt
|
mark.txt
|
||||||
recwarn.txt
|
recwarn.txt
|
||||||
reporting.txt
|
|
||||||
debugging.txt
|
|
||||||
doctest.txt
|
doctest.txt
|
||||||
unittest.txt
|
unittest.txt
|
||||||
|
|
||||||
|
|
|
@ -0,0 +1,58 @@
|
||||||
|
.. _`setuptools installation`: http://pypi.python.org/pypi/setuptools
|
||||||
|
|
||||||
|
|
||||||
|
Basic usage
|
||||||
|
==================
|
||||||
|
|
||||||
|
We assume you have done an :ref:`install` like this::
|
||||||
|
|
||||||
|
easy_install -U pytest # or
|
||||||
|
pip install -U pytest
|
||||||
|
|
||||||
|
Ensure that this installed the ``py.test`` script::
|
||||||
|
|
||||||
|
py.test --version
|
||||||
|
|
||||||
|
|
||||||
|
Writing a first test
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
Let's create a small file with the following content::
|
||||||
|
|
||||||
|
# content of test_sample.py
|
||||||
|
def func(x):
|
||||||
|
return x + 1
|
||||||
|
def test_answer():
|
||||||
|
assert func(3) == 5
|
||||||
|
|
||||||
|
That's it. Now you can already execute the test function::
|
||||||
|
|
||||||
|
$ py.test test_sample.py
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: test_sample.py
|
||||||
|
|
||||||
|
test_sample.py F
|
||||||
|
|
||||||
|
================================= FAILURES =================================
|
||||||
|
_______________________________ test_answer ________________________________
|
||||||
|
|
||||||
|
def test_answer():
|
||||||
|
> assert func(3) == 5
|
||||||
|
E assert 4 == 5
|
||||||
|
E + where 4 = func(3)
|
||||||
|
|
||||||
|
test_sample.py:4: AssertionError
|
||||||
|
========================= 1 failed in 0.02 seconds =========================
|
||||||
|
|
||||||
|
We got a failure because our little ``func(3)`` call did not return ``5``.
|
||||||
|
A few notes on this little test invocation:
|
||||||
|
|
||||||
|
* ``test_answer`` was identified as a test function because of the
|
||||||
|
``test_`` prefix,
|
||||||
|
|
||||||
|
* we conveniently used the standard `assert statement`_ and the failure
|
||||||
|
report shows us the intermediate values.
|
||||||
|
|
||||||
|
.. _`assert statement`: http://docs.python.org/reference/simple_stmts.html#the-assert-statement
|
||||||
|
|
|
@ -0,0 +1,111 @@
|
||||||
|
|
||||||
|
Using the interactive command line
|
||||||
|
===============================================
|
||||||
|
|
||||||
|
Getting help on version, option names, environment vars
|
||||||
|
-----------------------------------------------------------
|
||||||
|
|
||||||
|
::
|
||||||
|
|
||||||
|
py.test --version # shows where pytest was imported from
|
||||||
|
py.test --funcargs # show available builtin function arguments
|
||||||
|
py.test --help-config # show configuration values
|
||||||
|
|
||||||
|
py.test -h | --help # show help
|
||||||
|
|
||||||
|
|
||||||
|
Stopping after the first (or N) failures
|
||||||
|
---------------------------------------------------
|
||||||
|
|
||||||
|
To stop the testing process after the first (N) failures::
|
||||||
|
|
||||||
|
py.test -x # stop after first failure
|
||||||
|
py.test -maxfail=2 # stop after two failures
|
||||||
|
|
||||||
|
Modifying Python traceback printing
|
||||||
|
----------------------------------------------
|
||||||
|
|
||||||
|
Examples for modifying traceback printing::
|
||||||
|
|
||||||
|
py.test --showlocals # show local variables in tracebacks
|
||||||
|
py.test -l # show local variables (shortcut)
|
||||||
|
|
||||||
|
py.test --tb=long # the default informative traceback formatting
|
||||||
|
py.test --tb=native # the Python standard library formatting
|
||||||
|
py.test --tb=short # a shorter traceback format
|
||||||
|
py.test --tb=line # only one line per failure
|
||||||
|
|
||||||
|
Dropping to PDB (Python Debugger) on failures
|
||||||
|
----------------------------------------------
|
||||||
|
|
||||||
|
.. _PDB: http://docs.python.org/library/pdb.html
|
||||||
|
|
||||||
|
Python comes with a builtin Python debugger called PDB_. ``py.test``
|
||||||
|
allows to drop into the PDB prompt via a command line option::
|
||||||
|
|
||||||
|
py.test --pdb
|
||||||
|
|
||||||
|
This will invoke the Python debugger on every failure. Often you might
|
||||||
|
only want to do this for the first failing test to understand a certain
|
||||||
|
failure situation::
|
||||||
|
|
||||||
|
py.test -x --pdb # drop to PDB on first failure, then end test session
|
||||||
|
py.test --pdb --maxfail=3 # drop to PDB for the first three failures
|
||||||
|
|
||||||
|
|
||||||
|
Setting a breakpoint / aka ``set_trace()``
|
||||||
|
----------------------------------------------------
|
||||||
|
|
||||||
|
If you want to set a breakpoint and enter the ``pdb.set_trace()`` you
|
||||||
|
can use a helper::
|
||||||
|
|
||||||
|
def test_function():
|
||||||
|
...
|
||||||
|
py.test.set_trace() # invoke PDB debugger and tracing
|
||||||
|
|
||||||
|
.. versionadded: 2.0.0
|
||||||
|
|
||||||
|
In previous versions you could only enter PDB tracing if
|
||||||
|
you :ref:`disable capturing`.
|
||||||
|
|
||||||
|
creating JUnitXML format files
|
||||||
|
----------------------------------------------------
|
||||||
|
|
||||||
|
To create result files which can be read by Hudson_ or other Continous
|
||||||
|
integration servers, use this invocation::
|
||||||
|
|
||||||
|
py.test --junitxml=path
|
||||||
|
|
||||||
|
to create an XML file at ``path``.
|
||||||
|
|
||||||
|
creating resultlog format files
|
||||||
|
----------------------------------------------------
|
||||||
|
|
||||||
|
To create plain-text machine-readable result files you can issue::
|
||||||
|
|
||||||
|
py.test --resultlog=path
|
||||||
|
|
||||||
|
and look at the content at the ``path`` location. Such files are used e.g.
|
||||||
|
by the `PyPy-test`_ web page to show test results over several revisions.
|
||||||
|
|
||||||
|
.. _`PyPy-test`: http://codespeak.net:8099/summary
|
||||||
|
|
||||||
|
|
||||||
|
send test report to pocoo pastebin service
|
||||||
|
-----------------------------------------------------
|
||||||
|
|
||||||
|
**Creating a URL for each test failure**::
|
||||||
|
|
||||||
|
py.test --pastebin=failed
|
||||||
|
|
||||||
|
This will submit test run information to a remote Paste service and
|
||||||
|
provide a URL for each failure. You may select tests as usual or add
|
||||||
|
for example ``-x`` if you only want to send one particular failure.
|
||||||
|
|
||||||
|
**Creating a URL for a whole test session log**::
|
||||||
|
|
||||||
|
py.test --pastebin=all
|
||||||
|
|
||||||
|
Currently only pasting to the http://paste.pocoo.org service is implemented.
|
||||||
|
|
||||||
|
.. include:: links.inc
|
|
@ -64,7 +64,7 @@ release = '2.0.0dev0'
|
||||||
|
|
||||||
# List of patterns, relative to source directory, that match files and
|
# List of patterns, relative to source directory, that match files and
|
||||||
# directories to ignore when looking for source files.
|
# directories to ignore when looking for source files.
|
||||||
exclude_patterns = ['_build', 'example', 'test', 'announce'] # XXX
|
exclude_patterns = ['_build', 'test', 'announce'] # XXX
|
||||||
|
|
||||||
# The reST default role (used for this markup: `text`) to use for all documents.
|
# The reST default role (used for this markup: `text`) to use for all documents.
|
||||||
#default_role = None
|
#default_role = None
|
||||||
|
|
|
@ -1,59 +0,0 @@
|
||||||
|
|
||||||
debugging Python failures
|
|
||||||
=================================================================
|
|
||||||
|
|
||||||
Stopping after the first (or N) failures
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
To stop the testing process after the first (N) failures::
|
|
||||||
|
|
||||||
py.test -x # stop after first failure
|
|
||||||
py.test -maxfail=2 # stop after two failures
|
|
||||||
|
|
||||||
Modifying traceback printing
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
Examples for modifying traceback printing::
|
|
||||||
|
|
||||||
py.test --showlocals # show local variables in tracebacks
|
|
||||||
py.test -l # show local variables (shortcut)
|
|
||||||
|
|
||||||
py.test --tb=long # the default informative traceback formatting
|
|
||||||
py.test --tb=native # the Python standard library formatting
|
|
||||||
py.test --tb=short # a shorter traceback format
|
|
||||||
py.test --tb=line # only one line per failure
|
|
||||||
|
|
||||||
Dropping to PDB (Python Debugger) on failures
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
.. _PDB: http://docs.python.org/library/pdb.html
|
|
||||||
|
|
||||||
Python comes with a builtin Python debugger called PDB_. ``py.test``
|
|
||||||
allows to drop into the PDB prompt via a command line option::
|
|
||||||
|
|
||||||
py.test --pdb
|
|
||||||
|
|
||||||
This will invoke the Python debugger on every failure. Often you might
|
|
||||||
only want to do this for the first failing test to understand a certain
|
|
||||||
failure situation::
|
|
||||||
|
|
||||||
py.test -x --pdb # drop to PDB on first failure, then end test session
|
|
||||||
py.test --pdb --maxfail=3 # drop to PDB for the first three failures
|
|
||||||
|
|
||||||
|
|
||||||
Setting a breakpoint / aka ``set_trace()``
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
If you want to set a breakpoint and enter the ``pdb.set_trace()`` you
|
|
||||||
can use a helper::
|
|
||||||
|
|
||||||
def test_function():
|
|
||||||
...
|
|
||||||
py.test.set_trace() # invoke PDB debugger and tracing
|
|
||||||
|
|
||||||
.. versionadded: 2.0.0
|
|
||||||
|
|
||||||
In previous versions you could only enter PDB tracing if
|
|
||||||
you :ref:`disable capturing`.
|
|
||||||
|
|
||||||
|
|
|
@ -6,7 +6,6 @@ Feedback and contribute to py.test
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
contact.txt
|
contact.txt
|
||||||
faq.txt
|
|
||||||
|
|
||||||
.. _checkout:
|
.. _checkout:
|
||||||
|
|
||||||
|
|
|
@ -1,14 +0,0 @@
|
||||||
|
|
||||||
Using the command line
|
|
||||||
===============================================
|
|
||||||
|
|
||||||
Example command line usages::
|
|
||||||
|
|
||||||
py.test --version # shows where pytest was imported from
|
|
||||||
py.test --funcargs # show available builtin function arguments
|
|
||||||
py.test --help-config # show configuration values
|
|
||||||
|
|
||||||
py.test -h | --help # show help
|
|
||||||
|
|
||||||
|
|
||||||
which tells you both version and import location of the tool.
|
|
|
@ -0,0 +1,60 @@
|
||||||
|
|
||||||
|
Customizing test function through marks and hooks
|
||||||
|
====================================================
|
||||||
|
|
||||||
|
.. _`retrieved by hooks as item keywords`:
|
||||||
|
|
||||||
|
control skipping of tests according to command line option
|
||||||
|
--------------------------------------------------------------
|
||||||
|
|
||||||
|
Here is a ``conftest.py`` file adding a ``--runslow`` command
|
||||||
|
line option to control skipping of ``slow`` marked tests::
|
||||||
|
|
||||||
|
# content of conftest.py
|
||||||
|
|
||||||
|
import py
|
||||||
|
def pytest_addoption(parser):
|
||||||
|
parser.addoption("--runslow", action="store_true",
|
||||||
|
help="run slow tests")
|
||||||
|
|
||||||
|
def pytest_runtest_setup(item):
|
||||||
|
if 'slow' in item.keywords and not item.config.getvalue("runslow"):
|
||||||
|
py.test.skip("need --runslow option to run")
|
||||||
|
|
||||||
|
We can now write a test module like this::
|
||||||
|
|
||||||
|
# content of test_module.py
|
||||||
|
|
||||||
|
import py
|
||||||
|
slow = py.test.mark.slow
|
||||||
|
|
||||||
|
def test_func_fast():
|
||||||
|
pass
|
||||||
|
|
||||||
|
@slow
|
||||||
|
def test_func_slow():
|
||||||
|
pass
|
||||||
|
|
||||||
|
and when running it will see a skipped "slow" test::
|
||||||
|
|
||||||
|
$ py.test test_module.py -rs # "-rs" means report on the little 's'
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: test_module.py
|
||||||
|
|
||||||
|
test_module.py .s
|
||||||
|
========================= short test summary info ==========================
|
||||||
|
SKIP [1] /tmp/doc-exec-12/conftest.py:9: 'need --runslow option to run'
|
||||||
|
|
||||||
|
=================== 1 passed, 1 skipped in 0.02 seconds ====================
|
||||||
|
|
||||||
|
Or run it including the ``slow`` marked test::
|
||||||
|
|
||||||
|
$ py.test test_module.py --runslow
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: test_module.py
|
||||||
|
|
||||||
|
test_module.py ..
|
||||||
|
|
||||||
|
========================= 2 passed in 0.01 seconds =========================
|
|
@ -0,0 +1,138 @@
|
||||||
|
|
||||||
|
.. highlightlang:: python
|
||||||
|
|
||||||
|
mysetup pattern: application specific test fixtures
|
||||||
|
==========================================================
|
||||||
|
|
||||||
|
Here is a basic useful step-by-step example for managing and interacting
|
||||||
|
with application specific test setup. The goal is to have one place
|
||||||
|
where we have the glue and test support code for bootstrapping and
|
||||||
|
configuring application objects and allow test modules and test
|
||||||
|
functions to stay ignorant of involved details.
|
||||||
|
|
||||||
|
step1: implementing the test/app-specific ``mysetup`` pattern
|
||||||
|
-------------------------------------------------------------
|
||||||
|
|
||||||
|
Let's write a simple test function using a ``mysetup`` funcarg::
|
||||||
|
|
||||||
|
# content of test_sample.py
|
||||||
|
def test_answer(mysetup):
|
||||||
|
app = mysetup.myapp()
|
||||||
|
answer = app.question()
|
||||||
|
assert answer == 42
|
||||||
|
|
||||||
|
To run this test py.test needs to find and call a factory to
|
||||||
|
obtain the required ``mysetup`` function argument. To make
|
||||||
|
an according factory findable we write down a specifically named factory
|
||||||
|
method in a :ref:`local plugin`::
|
||||||
|
|
||||||
|
# content of conftest.py
|
||||||
|
from myapp import MyApp
|
||||||
|
|
||||||
|
def pytest_funcarg__mysetup(request): # "mysetup" factory function
|
||||||
|
return MySetup()
|
||||||
|
|
||||||
|
class MySetup: # instances of this are seen by test functions
|
||||||
|
def myapp(self):
|
||||||
|
return MyApp()
|
||||||
|
|
||||||
|
To run the example we stub out a a simple ``MyApp`` application object::
|
||||||
|
|
||||||
|
# content of myapp.py
|
||||||
|
class MyApp:
|
||||||
|
def question(self):
|
||||||
|
return 6 * 9
|
||||||
|
|
||||||
|
You can now run the test::
|
||||||
|
|
||||||
|
$ py.test test_sample.py
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: test_sample.py
|
||||||
|
|
||||||
|
test_sample.py F
|
||||||
|
|
||||||
|
================================= FAILURES =================================
|
||||||
|
_______________________________ test_answer ________________________________
|
||||||
|
|
||||||
|
mysetup = <conftest.MySetup instance at 0x1a09f38>
|
||||||
|
|
||||||
|
def test_answer(mysetup):
|
||||||
|
app = mysetup.myapp()
|
||||||
|
answer = app.question()
|
||||||
|
> assert answer == 42
|
||||||
|
E assert 54 == 42
|
||||||
|
|
||||||
|
test_sample.py:4: AssertionError
|
||||||
|
========================= 1 failed in 0.02 seconds =========================
|
||||||
|
|
||||||
|
This means that our ``mysetup`` object was successfully instantiated
|
||||||
|
and ``mysetup.app()`` returned an initialized ``MyApp`` instance.
|
||||||
|
We can ask it about the question and if you are confused as to what
|
||||||
|
the concrete question or answers actually mean, please see here_.
|
||||||
|
|
||||||
|
.. _here: http://uncyclopedia.wikia.com/wiki/The_Hitchhiker's_Guide_to_the_Galaxy
|
||||||
|
.. _`tut-cmdlineoption`:
|
||||||
|
|
||||||
|
step 2: checking a command line option and skipping tests
|
||||||
|
-----------------------------------------------------------
|
||||||
|
|
||||||
|
To add a command line option we update the ``conftest.py`` of
|
||||||
|
the previous example to add a command line option
|
||||||
|
and to offer a new mysetup method::
|
||||||
|
|
||||||
|
# content of ./conftest.py
|
||||||
|
import py
|
||||||
|
from myapp import MyApp
|
||||||
|
|
||||||
|
def pytest_funcarg__mysetup(request): # "mysetup" factory function
|
||||||
|
return MySetup(request)
|
||||||
|
|
||||||
|
def pytest_addoption(parser):
|
||||||
|
parser.addoption("--ssh", action="store", default=None,
|
||||||
|
help="specify ssh host to run tests with")
|
||||||
|
|
||||||
|
|
||||||
|
class MySetup:
|
||||||
|
def __init__(self, request):
|
||||||
|
self.config = request.config
|
||||||
|
|
||||||
|
def myapp(self):
|
||||||
|
return MyApp()
|
||||||
|
|
||||||
|
def getsshconnection(self):
|
||||||
|
host = self.config.option.ssh
|
||||||
|
if host is None:
|
||||||
|
py.test.skip("specify ssh host with --ssh")
|
||||||
|
return execnet.SshGateway(host)
|
||||||
|
|
||||||
|
|
||||||
|
Now any test function can use the ``mysetup.getsshconnection()`` method
|
||||||
|
like this::
|
||||||
|
|
||||||
|
# content of test_ssh.py
|
||||||
|
class TestClass:
|
||||||
|
def test_function(self, mysetup):
|
||||||
|
conn = mysetup.getsshconnection()
|
||||||
|
# work with conn
|
||||||
|
|
||||||
|
Running it yields::
|
||||||
|
|
||||||
|
$ py.test test_ssh.py -rs
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: test_ssh.py
|
||||||
|
|
||||||
|
test_ssh.py s
|
||||||
|
========================= short test summary info ==========================
|
||||||
|
SKIP [1] /tmp/doc-exec-9/conftest.py:22: 'specify ssh host with --ssh'
|
||||||
|
|
||||||
|
======================== 1 skipped in 0.02 seconds =========================
|
||||||
|
|
||||||
|
If you specify a command line option like ``py.test --ssh=python.org`` the test will execute as expected.
|
||||||
|
|
||||||
|
Note that neither the ``TestClass`` nor the ``test_function`` need to
|
||||||
|
know anything about how to setup the test state. It is handled separately
|
||||||
|
in your "test setup glue" code in the ``conftest.py`` file. It is easy
|
||||||
|
to extend the ``mysetup`` object for further needs in the test code - and for use by any other test functions in the files and directories below the ``conftest.py`` file.
|
||||||
|
|
|
@ -1,71 +0,0 @@
|
||||||
.. _`setuptools installation`: http://pypi.python.org/pypi/setuptools
|
|
||||||
|
|
||||||
|
|
||||||
==================
|
|
||||||
Quickstart
|
|
||||||
==================
|
|
||||||
|
|
||||||
.. _here: ../install.html
|
|
||||||
|
|
||||||
If you have the ``easy_install`` tool (otherwise see here_) just type:
|
|
||||||
|
|
||||||
|
|
||||||
easy_install -U py
|
|
||||||
|
|
||||||
Now create a file ``test_sample.py`` with the following content:
|
|
||||||
|
|
||||||
.. sourcecode:: python
|
|
||||||
|
|
||||||
# content of test_sample.py
|
|
||||||
def func(x):
|
|
||||||
return x + 1
|
|
||||||
def test_answer():
|
|
||||||
assert func(3) == 5
|
|
||||||
|
|
||||||
You can now run the test file like this::
|
|
||||||
|
|
||||||
py.test test_sample.py
|
|
||||||
|
|
||||||
and will see output like this:
|
|
||||||
|
|
||||||
.. sourcecode:: python
|
|
||||||
|
|
||||||
=========================== test session starts ============================
|
|
||||||
python: platform linux2 -- Python 2.6.2 -- pytest-1.1.0
|
|
||||||
test object 1: test_sample.py
|
|
||||||
|
|
||||||
test_sample.py F
|
|
||||||
|
|
||||||
================================= FAILURES =================================
|
|
||||||
_______________________________ test_answer ________________________________
|
|
||||||
|
|
||||||
def test_answer():
|
|
||||||
> assert func(3) == 5
|
|
||||||
E assert 4 == 5
|
|
||||||
E + where 4 = func(3)
|
|
||||||
|
|
||||||
test_sample.py:6: AssertionError
|
|
||||||
========================= 1 failed in 0.08 seconds =========================
|
|
||||||
|
|
||||||
This output contains Python interpreter information, a list of test objects,
|
|
||||||
a progress report and important details of the failure.
|
|
||||||
|
|
||||||
**Where to go from here**
|
|
||||||
|
|
||||||
`features`_: overview and description of test features
|
|
||||||
|
|
||||||
`plugins`_: a list of available plugins which each contain usage examples
|
|
||||||
|
|
||||||
`tutorials`_: some blog entries and starting points with code examples
|
|
||||||
|
|
||||||
`contact`_: if you want to feedback or have problems
|
|
||||||
|
|
||||||
.. _`contact`: ../contact.html
|
|
||||||
.. _`automatically collected`: features.html#autocollect
|
|
||||||
.. _install: ../install.html
|
|
||||||
.. _plugins: plugin/index.html
|
|
||||||
.. _features: features.html
|
|
||||||
.. _tutorials: talks.html
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -1,8 +1,9 @@
|
||||||
|
|
||||||
Examples for writing tests
|
Usages and Examples
|
||||||
===========================================
|
===========================================
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
|
||||||
example/funcarg_simple.txt
|
example/marking.txt
|
||||||
|
example/mysetup.txt
|
||||||
|
|
25
doc/faq.txt
25
doc/faq.txt
|
@ -1,6 +1,29 @@
|
||||||
Frequently asked Questions
|
Frequent Issues and Questions
|
||||||
==================================
|
==================================
|
||||||
|
|
||||||
|
.. _`installation issues`:
|
||||||
|
|
||||||
|
Installation issues
|
||||||
|
------------------------------
|
||||||
|
|
||||||
|
easy_install or py.test not found on Windows machine
|
||||||
|
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
|
||||||
|
|
||||||
|
- **Windows**: If "easy_install" or "py.test" are not found
|
||||||
|
please see here for preparing your environment for running
|
||||||
|
command line tools: `Python for Windows`_. You may alternatively
|
||||||
|
use an `ActivePython install`_ which makes command line tools
|
||||||
|
automatically available under Windows.
|
||||||
|
|
||||||
|
.. _`ActivePython install`: http://www.activestate.com/activepython/downloads
|
||||||
|
|
||||||
|
.. _`Jython does not create command line launchers`: http://bugs.jython.org/issue1491
|
||||||
|
|
||||||
|
- **Jython2.5.1 on Windows XP**: `Jython does not create command line launchers`_
|
||||||
|
so ``py.test`` will not work correctly. You may install py.test on
|
||||||
|
CPython and type ``py.test --genscript=mytest`` and then use
|
||||||
|
``jython mytest`` to run py.test for your tests to run in Jython.
|
||||||
|
|
||||||
On naming, nosetests, licensing and magic XXX
|
On naming, nosetests, licensing and magic XXX
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
|
|
||||||
|
|
|
@ -1,14 +1,14 @@
|
||||||
py.test Features
|
py.test Features
|
||||||
==================
|
==================
|
||||||
|
|
||||||
mature command line testing tool
|
no-boilerplate testing with Python
|
||||||
--------------------------------------
|
----------------------------------
|
||||||
|
|
||||||
- used in many projects, ranging from 10 to 10K tests
|
- automatic customizable Python test discovery
|
||||||
- simple well sorted command line options
|
- powerful parametrization of test functions
|
||||||
- runs on Unix, Windows from Python 2.4 up to Python 3.1 and 3.2
|
- use the ``assert`` statement for your assertions
|
||||||
- is itself tested extensively on a CI server
|
- rely on powerful traceback and assertion reporting
|
||||||
- keyword/testname based selection of tests
|
- use ``print`` or ``pdb`` debugging on failures
|
||||||
|
|
||||||
extensive plugin and customization system
|
extensive plugin and customization system
|
||||||
------------------------------------------------------
|
------------------------------------------------------
|
||||||
|
@ -20,6 +20,16 @@ extensive plugin and customization system
|
||||||
- it is `suprisingly easy`_ to add command line options or
|
- it is `suprisingly easy`_ to add command line options or
|
||||||
do other kind of add-ons and customizations.
|
do other kind of add-ons and customizations.
|
||||||
|
|
||||||
|
mature command line testing tool
|
||||||
|
--------------------------------------
|
||||||
|
|
||||||
|
- used in many projects, ranging from 10 to 10K tests
|
||||||
|
- autodiscovery of tests
|
||||||
|
- simple well sorted command line options
|
||||||
|
- runs on Unix, Windows from Python 2.4 up to Python 3.1 and 3.2
|
||||||
|
- is itself tested extensively on a CI server
|
||||||
|
- keyword/testname based selection of tests
|
||||||
|
|
||||||
integrates well with CI systems
|
integrates well with CI systems
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
|
@ -29,14 +39,6 @@ integrates well with CI systems
|
||||||
|
|
||||||
.. _`tox`: http://codespeak.net/tox
|
.. _`tox`: http://codespeak.net/tox
|
||||||
|
|
||||||
no-boilerplate testing with Python
|
|
||||||
----------------------------------
|
|
||||||
|
|
||||||
- automatic customizable Python test discovery
|
|
||||||
- powerful parametrization of test functions
|
|
||||||
- use the ``assert`` statement for your assertions
|
|
||||||
- rely on powerful traceback and assertion reporting
|
|
||||||
- use ``print`` or ``pdb`` debugging on failures
|
|
||||||
|
|
||||||
supports several testing practises and methods
|
supports several testing practises and methods
|
||||||
-----------------------------------------------------------
|
-----------------------------------------------------------
|
||||||
|
|
|
@ -1,5 +1,5 @@
|
||||||
==============================================================
|
==============================================================
|
||||||
funcargs: creating and managing test function arguments
|
creating and managing test function arguments
|
||||||
==============================================================
|
==============================================================
|
||||||
|
|
||||||
.. currentmodule:: pytest.plugin.python
|
.. currentmodule:: pytest.plugin.python
|
||||||
|
|
|
@ -0,0 +1,43 @@
|
||||||
|
|
||||||
|
Good Practises
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
Recommendation: install tool and dependencies virtually
|
||||||
|
-----------------------------------------------------------
|
||||||
|
|
||||||
|
We recommend to work with virtual environments
|
||||||
|
(e.g. virtualenv_ or buildout_ based) and use easy_install_
|
||||||
|
(or pip_) for installing py.test/pylib and any dependencies
|
||||||
|
you need to run your tests. Local virtual Python environments
|
||||||
|
(as opposed to system-wide "global" environments) make for a more
|
||||||
|
reproducible and reliable test environment.
|
||||||
|
|
||||||
|
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
|
||||||
|
.. _`buildout`: http://www.buildout.org/
|
||||||
|
.. _pip: http://pypi.python.org/pypi/pip
|
||||||
|
|
||||||
|
.. _standalone:
|
||||||
|
|
||||||
|
Generating a py.test standalone Script
|
||||||
|
-------------------------------------------
|
||||||
|
|
||||||
|
If you are a maintainer or application developer and want users
|
||||||
|
to run tests you can use a facility to generate a standalone
|
||||||
|
"py.test" script that you can tell users to run::
|
||||||
|
|
||||||
|
py.test --genscript=mytest
|
||||||
|
|
||||||
|
will generate a ``mytest`` script that is, in fact, a ``py.test`` under
|
||||||
|
disguise. You can tell people to download and then e.g. run it like this::
|
||||||
|
|
||||||
|
python mytest --pastebin=all
|
||||||
|
|
||||||
|
and ask them to send you the resulting URL. The resulting script has
|
||||||
|
all core features and runs unchanged under Python2 and Python3 interpreters.
|
||||||
|
|
||||||
|
.. _`Python for Windows`: http://www.imladris.com/Scripts/PythonForWindows.html
|
||||||
|
|
||||||
|
.. _`Distribute for installation`: http://pypi.python.org/pypi/distribute#installation-instructions
|
||||||
|
.. _`distribute installation`: http://pypi.python.org/pypi/distribute
|
||||||
|
|
||||||
|
.. include:: links.inc
|
|
@ -1,14 +1,7 @@
|
||||||
.. pytest documentation master file, created by
|
py.test: no-boilerplate testing with Python
|
||||||
sphinx-quickstart on Fri Oct 8 17:54:28 2010.
|
==============================================
|
||||||
You can adapt this file completely to your liking, but it should at least
|
|
||||||
contain the root `toctree` directive.
|
|
||||||
|
|
||||||
py.test: no-boilerplate rapid testing with Python
|
Welcome to ``py.test`` documentation:
|
||||||
======================================================
|
|
||||||
|
|
||||||
Welcome to ``py.test`` documentation.
|
|
||||||
|
|
||||||
Contents:
|
|
||||||
|
|
||||||
.. toctree::
|
.. toctree::
|
||||||
:maxdepth: 2
|
:maxdepth: 2
|
||||||
|
@ -25,7 +18,6 @@ Contents:
|
||||||
|
|
||||||
changelog.txt
|
changelog.txt
|
||||||
|
|
||||||
|
|
||||||
Indices and tables
|
Indices and tables
|
||||||
==================
|
==================
|
||||||
|
|
||||||
|
|
|
@ -13,76 +13,21 @@ installing py.test
|
||||||
Installation using easy_install
|
Installation using easy_install
|
||||||
----------------------------------------
|
----------------------------------------
|
||||||
|
|
||||||
You need setuptools_ or Distribute_ to be able to simply type::
|
If you have setuptools_ or Distribute_ type::
|
||||||
|
|
||||||
easy_install -U pytest
|
easy_install -U pytest
|
||||||
|
|
||||||
to install the latest release of ``py.test``. The ``-U`` switch
|
to install the latest release of ``py.test``. The ``-U`` switch
|
||||||
triggers an upgrade if you already have an older version installed.
|
triggers an upgrade if you already have an older version installed.
|
||||||
Note that setuptools works ok with Python2 interpreters while `Distribute`_
|
Note that setuptools works ok with Python2 interpreters while **Distribute
|
||||||
additionally works with Python3 and also avoid some issues on Windows.
|
works with Python3 and Python2** and also avoids some issues on Windows.
|
||||||
|
|
||||||
Known issues:
|
You should now be able to type::
|
||||||
|
|
||||||
- **Windows**: If "easy_install" or "py.test" are not found
|
$ py.test --version
|
||||||
please see here for preparing your environment for running
|
|
||||||
command line tools: `Python for Windows`_. You may alternatively
|
|
||||||
use an `ActivePython install`_ which makes command line tools
|
|
||||||
automatically available under Windows.
|
|
||||||
|
|
||||||
.. _`ActivePython install`: http://www.activestate.com/activepython/downloads
|
Refer to :ref:`installation issues` if you have issues.
|
||||||
|
|
||||||
.. _`Jython does not create command line launchers`: http://bugs.jython.org/issue1491
|
|
||||||
|
|
||||||
- **Jython2.5.1 on Windows XP**: `Jython does not create command line launchers`_
|
|
||||||
so ``py.test`` will not work correctly. You may install py.test on
|
|
||||||
CPython and type ``py.test --genscript=mytest`` and then use
|
|
||||||
``jython mytest`` to run py.test for your tests to run in Jython.
|
|
||||||
|
|
||||||
- **On Linux**: If ``easy_install`` fails because it needs to run
|
|
||||||
as the superuser you are trying to install things globally
|
|
||||||
and need to put ``sudo`` in front of the command.
|
|
||||||
|
|
||||||
|
|
||||||
.. _quickstart: test/quickstart.html
|
|
||||||
|
|
||||||
|
|
||||||
Recommendation: install tool and dependencies virtually
|
|
||||||
-----------------------------------------------------------
|
|
||||||
|
|
||||||
I recommend to work with virtual environments
|
|
||||||
(e.g. virtualenv_ or buildout_ based) and use easy_install_
|
|
||||||
(or pip_) for installing py.test/pylib and any dependencies
|
|
||||||
you need to run your tests. Local virtual Python environments
|
|
||||||
(as opposed to system-wide "global" environments) make for a more
|
|
||||||
reproducible and reliable test environment.
|
|
||||||
|
|
||||||
.. _`virtualenv`: http://pypi.python.org/pypi/virtualenv
|
|
||||||
.. _`buildout`: http://www.buildout.org/
|
|
||||||
.. _pip: http://pypi.python.org/pypi/pip
|
|
||||||
|
|
||||||
.. _standalone:
|
|
||||||
|
|
||||||
Generating a py.test standalone Script
|
|
||||||
-------------------------------------------
|
|
||||||
|
|
||||||
If you are a maintainer or application developer and want users
|
|
||||||
to run tests you can use a facility to generate a standalone
|
|
||||||
"py.test" script that you can tell users to run::
|
|
||||||
|
|
||||||
py.test --genscript=mytest
|
|
||||||
|
|
||||||
will generate a ``mytest`` script that is, in fact, a ``py.test`` under
|
|
||||||
disguise. You can tell people to download and then e.g. run it like this::
|
|
||||||
|
|
||||||
python mytest --pastebin=all
|
|
||||||
|
|
||||||
and ask them to send you the resulting URL. The resulting script has
|
|
||||||
all core features and runs unchanged under Python2 and Python3 interpreters.
|
|
||||||
|
|
||||||
.. _`Python for Windows`: http://www.imladris.com/Scripts/PythonForWindows.html
|
|
||||||
|
|
||||||
.. _`Distribute for installation`: http://pypi.python.org/pypi/distribute#installation-instructions
|
|
||||||
.. _`distribute installation`: http://pypi.python.org/pypi/distribute
|
|
||||||
|
|
||||||
.. include:: links.inc
|
.. include:: links.inc
|
||||||
|
|
||||||
|
|
||||||
|
|
37
doc/mark.txt
37
doc/mark.txt
|
@ -1,9 +1,11 @@
|
||||||
|
|
||||||
.. _mark:
|
.. _mark:
|
||||||
|
|
||||||
mark (attribute) python functions
|
mark test functions with attributes
|
||||||
=================================================================
|
=================================================================
|
||||||
|
|
||||||
|
.. currentmodule:: pytest.plugin.mark
|
||||||
|
|
||||||
By using the ``py.test.mark`` helper you can instantiate
|
By using the ``py.test.mark`` helper you can instantiate
|
||||||
decorators that will set named meta data on test functions.
|
decorators that will set named meta data on test functions.
|
||||||
|
|
||||||
|
@ -17,7 +19,7 @@ You can "mark" a test function with meta data like this::
|
||||||
def test_send_http():
|
def test_send_http():
|
||||||
...
|
...
|
||||||
|
|
||||||
This will set the function attribute ``webtest`` to a :py:meth:`MarkInfo`
|
This will set the function attribute ``webtest`` to a :py:class:`MarkInfo`
|
||||||
instance. You can also specify parametrized meta data like this::
|
instance. You can also specify parametrized meta data like this::
|
||||||
|
|
||||||
# content of test_mark.py
|
# content of test_mark.py
|
||||||
|
@ -79,15 +81,15 @@ You can also set a module level marker::
|
||||||
in which case it will be applied to all functions and
|
in which case it will be applied to all functions and
|
||||||
methods defined in the module.
|
methods defined in the module.
|
||||||
|
|
||||||
Using "-k MARKNAME" to select tests
|
Using ``-k TEXT`` to select tests
|
||||||
----------------------------------------------------
|
----------------------------------------------------
|
||||||
|
|
||||||
You can use the ``-k`` command line option to select tests::
|
You can use the ``-k`` command line option to select tests::
|
||||||
|
|
||||||
$ py.test -k webtest # will only run tests marked as webtest
|
$ py.test -k webtest # running with the above defined examples yields
|
||||||
=========================== test session starts ============================
|
=========================== test session starts ============================
|
||||||
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
test path 1: /tmp/doc-exec-335
|
test path 1: /tmp/doc-exec-11
|
||||||
|
|
||||||
test_mark.py ..
|
test_mark.py ..
|
||||||
test_mark_classlevel.py ..
|
test_mark_classlevel.py ..
|
||||||
|
@ -96,16 +98,35 @@ You can use the ``-k`` command line option to select tests::
|
||||||
|
|
||||||
And you can also run all tests except the ones that match the keyword::
|
And you can also run all tests except the ones that match the keyword::
|
||||||
|
|
||||||
$ py.test -k-webtest # "-" negates but be careful to have no space before
|
$ py.test -k-webtest
|
||||||
=========================== test session starts ============================
|
=========================== test session starts ============================
|
||||||
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
test path 1: /tmp/doc-exec-335
|
test path 1: /tmp/doc-exec-11
|
||||||
|
|
||||||
===================== 4 tests deselected by '-webtest' =====================
|
===================== 4 tests deselected by '-webtest' =====================
|
||||||
======================= 4 deselected in 0.01 seconds =======================
|
======================= 4 deselected in 0.01 seconds =======================
|
||||||
|
|
||||||
|
Or to only select the class::
|
||||||
|
|
||||||
|
$ py.test -kTestClass
|
||||||
|
=========================== test session starts ============================
|
||||||
|
platform linux2 -- Python 2.6.5 -- pytest-2.0.0dev0
|
||||||
|
test path 1: /tmp/doc-exec-11
|
||||||
|
|
||||||
|
test_mark_classlevel.py ..
|
||||||
|
|
||||||
|
==================== 2 tests deselected by 'TestClass' =====================
|
||||||
|
================== 2 passed, 2 deselected in 0.01 seconds ==================
|
||||||
|
|
||||||
API reference for mark related objects
|
API reference for mark related objects
|
||||||
------------------------------------------------
|
------------------------------------------------
|
||||||
|
|
||||||
.. autoclass:: pytest.plugin.mark.MarkInfo
|
.. autoclass:: MarkGenerator
|
||||||
|
:members:
|
||||||
|
|
||||||
|
.. autoclass:: MarkDecorator
|
||||||
|
:members:
|
||||||
|
|
||||||
|
.. autoclass:: MarkInfo
|
||||||
|
:members:
|
||||||
|
|
||||||
|
|
|
@ -7,4 +7,8 @@ Overview and Introduction
|
||||||
|
|
||||||
features.txt
|
features.txt
|
||||||
install.txt
|
install.txt
|
||||||
|
basic_usage.txt
|
||||||
|
cmdline.txt
|
||||||
|
goodpractises.txt
|
||||||
|
faq.txt
|
||||||
|
|
||||||
|
|
|
@ -1,44 +0,0 @@
|
||||||
XML, pastebin services and other reporting
|
|
||||||
=================================================================
|
|
||||||
|
|
||||||
creating JUnitXML format files
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
To create result files which can be read by Hudson_ or other Continous
|
|
||||||
integration servers, use this invocation::
|
|
||||||
|
|
||||||
py.test --junitxml=path
|
|
||||||
|
|
||||||
to create an XML file at ``path``.
|
|
||||||
|
|
||||||
creating resultlog format files
|
|
||||||
----------------------------------------------------
|
|
||||||
|
|
||||||
To create plain-text machine-readable result files you can issue::
|
|
||||||
|
|
||||||
py.test --resultlog=path
|
|
||||||
|
|
||||||
and look at the content at the ``path`` location. Such files are used e.g.
|
|
||||||
by the `PyPy-test`_ web page to show test results over several revisions.
|
|
||||||
|
|
||||||
.. _`PyPy-test`: http://codespeak.net:8099/summary
|
|
||||||
|
|
||||||
|
|
||||||
send test report to pocoo pastebin service
|
|
||||||
-----------------------------------------------------
|
|
||||||
|
|
||||||
**Creating a URL for each test failure**::
|
|
||||||
|
|
||||||
py.test --pastebin=failed
|
|
||||||
|
|
||||||
This will submit test run information to a remote Paste service and
|
|
||||||
provide a URL for each failure. You may select tests as usual or add
|
|
||||||
for example ``-x`` if you only want to send one particular failure.
|
|
||||||
|
|
||||||
**Creating a URL for a whole test session log**::
|
|
||||||
|
|
||||||
py.test --pastebin=all
|
|
||||||
|
|
||||||
Currently only pasting to the http://paste.pocoo.org service is implemented.
|
|
||||||
|
|
||||||
.. include:: links.inc
|
|
|
@ -70,16 +70,37 @@ def matchonekeyword(key, chain):
|
||||||
return False
|
return False
|
||||||
|
|
||||||
class MarkGenerator:
|
class MarkGenerator:
|
||||||
""" non-underscore attributes of this object can be used as decorators for
|
""" Factory for :class:`MarkDecorator` objects - exposed as
|
||||||
marking test functions. Example: @py.test.mark.slowtest in front of a
|
a ``py.test.mark`` singleton instance. Example::
|
||||||
function will set the 'slowtest' marker object on it. """
|
|
||||||
|
import py
|
||||||
|
@py.test.mark.slowtest
|
||||||
|
def test_function():
|
||||||
|
pass
|
||||||
|
|
||||||
|
will set a 'slowtest' :class:`MarkInfo` object
|
||||||
|
on the ``test_function`` object. """
|
||||||
|
|
||||||
def __getattr__(self, name):
|
def __getattr__(self, name):
|
||||||
if name[0] == "_":
|
if name[0] == "_":
|
||||||
raise AttributeError(name)
|
raise AttributeError(name)
|
||||||
return MarkDecorator(name)
|
return MarkDecorator(name)
|
||||||
|
|
||||||
class MarkDecorator:
|
class MarkDecorator:
|
||||||
""" decorator for setting function attributes. """
|
""" A decorator for test functions and test classes. When applied
|
||||||
|
it will create :class:`MarkInfo` objects which may be
|
||||||
|
:ref:`retrieved by hooks as item keywords` MarkDecorator instances
|
||||||
|
are usually created by writing::
|
||||||
|
|
||||||
|
mark1 = py.test.mark.NAME # simple MarkDecorator
|
||||||
|
mark2 = py.test.mark.NAME(name1=value) # parametrized MarkDecorator
|
||||||
|
|
||||||
|
and can then be applied as decorators to test functions::
|
||||||
|
|
||||||
|
@mark2
|
||||||
|
def test_function():
|
||||||
|
pass
|
||||||
|
"""
|
||||||
def __init__(self, name):
|
def __init__(self, name):
|
||||||
self.markname = name
|
self.markname = name
|
||||||
self.kwargs = {}
|
self.kwargs = {}
|
||||||
|
@ -121,9 +142,13 @@ class MarkDecorator:
|
||||||
return self
|
return self
|
||||||
|
|
||||||
class MarkInfo:
|
class MarkInfo:
|
||||||
|
""" Marking object created by :class:`MarkDecorator` instances. """
|
||||||
def __init__(self, name, args, kwargs):
|
def __init__(self, name, args, kwargs):
|
||||||
self._name = name
|
#: name of attribute
|
||||||
|
self.name = name
|
||||||
|
#: positional argument list, empty if none specified
|
||||||
self.args = args
|
self.args = args
|
||||||
|
#: keyword argument dictionary, empty if nothing specified
|
||||||
self.kwargs = kwargs
|
self.kwargs = kwargs
|
||||||
|
|
||||||
def __repr__(self):
|
def __repr__(self):
|
||||||
|
|
|
@ -560,14 +560,12 @@ class FuncargRequest:
|
||||||
|
|
||||||
|
|
||||||
def applymarker(self, marker):
|
def applymarker(self, marker):
|
||||||
""" apply a marker to a test function invocation.
|
""" apply a marker to a single test function invocation.
|
||||||
Usually markers can be used as decorators for test functions or
|
This method is useful if you don't want to have a keyword/marker
|
||||||
classes. However, with parametrized testing a single
|
on all function invocations.
|
||||||
test function may be called multiple times and ``applymarker``
|
|
||||||
allows to mark only a single invocation.
|
|
||||||
|
|
||||||
:param marker: The ``pytest.mark.*`` object to be applied to the test invocation.
|
|
||||||
|
|
||||||
|
:arg marker: a :py:class:`pytest.plugin.mark.MarkDecorator` object
|
||||||
|
created by a call to ``py.test.mark.NAME(...)``.
|
||||||
"""
|
"""
|
||||||
if not isinstance(marker, py.test.mark.XYZ.__class__):
|
if not isinstance(marker, py.test.mark.XYZ.__class__):
|
||||||
raise ValueError("%r is not a py.test.mark.* object")
|
raise ValueError("%r is not a py.test.mark.* object")
|
||||||
|
|
Loading…
Reference in New Issue