jiuyuan
/
test_ok1
2
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

recreated plugin docs 

--HG--
branch : trunk
This commit is contained in:
holger krekel 2010-09-14 17:35:01 +02:00
parent 489faf26f2
commit bb6e9848b3
9 changed files with 261 additions and 248 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
doc/announce/release-1.3.4.txt
View File

@ -1,16 +1,17 @@
py.test/pylib 1.3.4: fixes and new native traceback option py.test/pylib 1.3.4: fixes and new native traceback option
=========================================================================== ===========================================================================
pylib/py.test 1.3.4 is a minor bugfix release featuring small fixes. See pylib/py.test 1.3.4 is a minor maintenance release mostly containing bug fixes
below and the changelog_ for full history. and a new "--tb=native" traceback option to show "normal" Python standard
tracebacks instead of the py.test enhanced tracebacks. See below for more
change info and http://pytest.org for more general information on features
and configuration of the testing tool.
Particular thanks to the issue reporters and Ronny Pfannschmidt for general help. Thanks to the issue reporters and generally to Ronny Pfannschmidt for help.
have fun, cheers,
holger krekel holger krekel
.. _changelog: ../changelog.html
Changes between 1.3.3 and 1.3.4 Changes between 1.3.3 and 1.3.4
================================================== ==================================================

297
doc/test/plugin/cov.txt
View File

@ -6,27 +6,39 @@ produce code coverage reports using the 'coverage' package, including support fo
.. contents:: .. contents::
:local: :local:
This plugin produces coverage reports using the coverage package. It This plugin produces coverage reports. It supports centralised testing and distributed testing in
supports centralised testing and distributed testing in both load and both load and each modes. It also supports coverage of subprocesses.
each modes.
All features offered by the coverage package should be available, All features offered by the coverage package should be available, either through pytest-cov or
either through this plugin or through coverage's own config file. through coverage's config file.
Installation Installation
------------ ------------
The `pytest-cov pypi`_ package may be installed / uninstalled with pip:: The `pytest-cov`_ package may be installed with pip or easy_install::
pip install pytest-cov pip install pytest-cov
pip uninstall pytest-cov
Alternatively easy_install can be used::
easy_install pytest-cov easy_install pytest-cov
.. _`pytest-cov pypi`: http://pypi.python.org/pypi/pytest-cov/ .. _`pytest-cov`: http://pypi.python.org/pypi/pytest-cov/
Uninstallation
--------------
Uninstalling packages is supported by pip::
pip uninstall pytest-cov
However easy_install does not provide an uninstall facility.
.. IMPORTANT::
Ensure that you manually delete the init_cov_core.pth file in your site-packages directory.
This file starts coverage collection of subprocesses if appropriate during site initialisation
at python startup.
Usage Usage
@ -35,6 +47,9 @@ Usage
Centralised Testing Centralised Testing
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~
Centralised testing will report on the combined coverage of the main process and all of it's
subprocesses.
Running centralised testing:: Running centralised testing::
py.test --cov myproj tests/ py.test --cov myproj tests/
@ -42,150 +57,149 @@ Running centralised testing::
Shows a terminal report:: Shows a terminal report::
-------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- -------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name Stmts Exec Cover Missing Name Stmts Miss Cover
-------------------------------------------------- ----------------------------------------
myproj/__init__ 2 2 100% myproj/__init__ 2 0 100%
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370 myproj/myproj 257 13 94%
myproj/feature4286 94 87 92% 183-188, 197 myproj/feature4286 94 7 92%
-------------------------------------------------- ----------------------------------------
TOTAL 353 333 94% TOTAL 353 20 94%
Distributed Testing Distributed Testing: Load
~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
Distributed testing with dist mode set to load:: Distributed testing with dist mode set to load will report on the combined coverage of all slaves.
The slaves may be spread out over any number of hosts and each slave may be located anywhere on the
file system. Each slave will have it's subprocesses measured.
Running distributed testing with dist mode set to load::
py.test --cov myproj -n 2 tests/ py.test --cov myproj -n 2 tests/
The results from the slaves will be combined like so:: Shows a terminal report::
-------------------- coverage: platform linux2, python 2.6.4-final-0 --------------------- -------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name Stmts Exec Cover Missing Name Stmts Miss Cover
-------------------------------------------------- ----------------------------------------
myproj/__init__ 2 2 100% myproj/__init__ 2 0 100%
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370 myproj/myproj 257 13 94%
myproj/feature4286 94 87 92% 183-188, 197 myproj/feature4286 94 7 92%
-------------------------------------------------- ----------------------------------------
TOTAL 353 333 94% TOTAL 353 20 94%
Distributed testing in each mode:: Again but spread over different hosts and different directories::
py.test --cov myproj --dist=each py.test --cov myproj --dist load
--tx=popen//python=/usr/local/python265/bin/python --tx ssh=memedough@host1//chdir=testenv1
--tx=popen//python=/usr/local/python27b1/bin/python --tx ssh=memedough@host2//chdir=/tmp/testenv2//python=/tmp/env1/bin/python
--rsyncdir myproj --rsyncdir tests --rsync examples
tests/ tests/
Will produce a report for each slave:: Shows a terminal report::
-------------------- coverage: platform linux2, python 2.6.5-final-0 --------------------- -------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name Stmts Exec Cover Missing Name Stmts Miss Cover
-------------------------------------------------- ----------------------------------------
myproj/__init__ 2 2 100% myproj/__init__ 2 0 100%
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370 myproj/myproj 257 13 94%
myproj/feature4286 94 87 92% 183-188, 197 myproj/feature4286 94 7 92%
-------------------------------------------------- ----------------------------------------
TOTAL 353 333 94% TOTAL 353 20 94%
--------------------- coverage: platform linux2, python 2.7.0-beta-1 ---------------------
Name Stmts Exec Cover Missing
--------------------------------------------------
myproj/__init__ 2 2 100%
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286 94 87 92% 183-188, 197
--------------------------------------------------
TOTAL 353 333 94%
Distributed testing in each mode can also produce a single combined Distributed Testing: Each
report. This is useful to get coverage information spanning things ~~~~~~~~~~~~~~~~~~~~~~~~~
such as all python versions::
py.test --cov myproj --cov-combine-each --dist=each Distributed testing with dist mode set to each will report on the combined coverage of all slaves.
--tx=popen//python=/usr/local/python265/bin/python Since each slave is running all tests this allows generating a combined coverage report for multiple
--tx=popen//python=/usr/local/python27b1/bin/python environments.
Running distributed testing with dist mode set to each::
py.test --cov myproj --dist each
--tx popen//chdir=/tmp/testenv3//python=/usr/local/python27/bin/python
--tx ssh=memedough@host2//chdir=/tmp/testenv4//python=/tmp/env2/bin/python
--rsyncdir myproj --rsyncdir tests --rsync examples
tests/ tests/
Which looks like:: Shows a terminal report::
---------------------------------------- coverage ---------------------------------------- ---------------------------------------- coverage ----------------------------------------
platform linux2, python 2.6.5-final-0 platform linux2, python 2.6.5-final-0
platform linux2, python 2.7.0-beta-1 platform linux2, python 2.7.0-final-0
Name Stmts Exec Cover Missing Name Stmts Miss Cover
-------------------------------------------------- ----------------------------------------
myproj/__init__ 2 2 100% myproj/__init__ 2 0 100%
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370 myproj/myproj 257 13 94%
myproj/feature4286 94 87 92% 183-188, 197 myproj/feature4286 94 7 92%
-------------------------------------------------- ----------------------------------------
TOTAL 353 333 94% TOTAL 353 20 94%
Reporting Reporting
--------- ---------
By default a terminal report is output. This report can be disabled It is possible to generate any combination of the reports for a single test run.
if desired, such as when results are going to a continuous integration
system and the terminal output won't be seen.
In addition and without rerunning tests it is possible to generate The available reports are terminal (with or without missing line numbers shown), HTML, XML and
annotated source code, a html report and an xml report. annotated source code.
The directories for annotated source code and html reports can be The terminal report without line numbers (default)::
specified as can the file name for the xml report.
Since testing often takes a non trivial amount of time at the end of py.test --cov-report term --cov myproj tests/
testing any / all of the reports may be generated.
-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name Stmts Miss Cover
----------------------------------------
myproj/__init__ 2 0 100%
myproj/myproj 257 13 94%
myproj/feature4286 94 7 92%
----------------------------------------
TOTAL 353 20 94%
The terminal report with line numbers::
py.test --cov-report term-missing --cov myproj tests/
-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
Name Stmts Miss Cover Missing
--------------------------------------------------
myproj/__init__ 2 0 100%
myproj/myproj 257 13 94% 24-26, 99, 149, 233-236, 297-298, 369-370
myproj/feature4286 94 7 92% 183-188, 197
--------------------------------------------------
TOTAL 353 20 94%
The remaining three reports output to files without showing anything on the terminal (useful for
when the output is going to a continuous integration server)::
py.test --cov-report html --cov-report xml --cov-report annotate --cov myproj tests/
Coverage Data File Coverage Data File
------------------ ------------------
During testing there may be many data files with coverage data. These The data file is erased at the beginning of testing to ensure clean data for each test run.
will have unique suffixes and will be combined at the end of testing.
Upon completion, for --dist=load (and also for --dist=each when the The data file is left at the end of testing so that it is possible to use normal coverage tools to
--cov-combine-each option is used) there will only be one data file. examine it.
For --dist=each there may be many data files where each one will have
the platform / python version info appended to the name.
These data files are left at the end of testing so that it is possible
to use normal coverage tools to examine them.
At the beginning of testing any data files that are about to be used
will first be erased so ensure the data is clean for each test run.
It is possible to set the name of the data file. If needed the
platform / python version will be appended automatically to this name.
Coverage Config File
--------------------
Coverage by default will read its own config file. An alternative
file name may be specified or reading config can be disabled entirely.
Care has been taken to ensure that the coverage env vars and config
file options work the same under this plugin as they do under coverage
itself.
Since options may be specified in different ways the order of
precedence between pytest-cov and coverage from highest to lowest is:
1. pytest command line
2. pytest env var
3. pytest conftest
4. coverage env var
5. coverage config file
6. coverage default
Limitations Limitations
----------- -----------
For distributed testing the slaves must have the pytest-cov package For distributed testing the slaves must have the pytest-cov package installed. This is needed since
installed. This is needed since the plugin must be registered through the plugin must be registered through setuptools / distribute for pytest to start the plugin on the
setuptools / distribute for pytest to start the plugin on the slave. slave.
For subprocess measurement environment variables must make it from the main process to the
subprocess. The python used by the subprocess must have pytest-cov installed. The subprocess must
do normal site initialisation so that the environment variables can be detected and coverage
started.
Acknowledgements Acknowledgements
@ -193,14 +207,11 @@ Acknowledgements
Holger Krekel for pytest with its distributed testing support. Holger Krekel for pytest with its distributed testing support.
Ned Batchelder for coverage and its ability to combine the coverage Ned Batchelder for coverage and its ability to combine the coverage results of parallel runs.
results of parallel runs.
Whilst this plugin has been built fresh from the ground up to support Whilst this plugin has been built fresh from the ground up to support distributed testing it has
distributed testing it has been influenced by the work done on been influenced by the work done on pytest-coverage (Ross Lawley, James Mills, Holger Krekel) and
pytest-coverage (Ross Lawley, James Mills, Holger Krekel) and nose-cover (Jason Pellerin) which are other coverage plugins for pytest and nose respectively.
nose-cover (Jason Pellerin) which are other coverage plugins for
pytest and nose respectively.
No doubt others have contributed to these tools as well. No doubt others have contributed to these tools as well.
@ -208,43 +219,11 @@ command line options
-------------------- --------------------
``--cov-on`` ``--cov=path``
enable coverage, only needed if not specifying any --cov options measure coverage for filesystem path (multi-allowed)
``--cov=package`` ``--cov-report=type``
collect coverage for the specified package (multi-allowed) type of report to generate: term, term-missing, annotate, html, xml (multi-allowed)
``--cov-no-terminal`` ``--cov-config=path``
disable printing a report on the terminal config file for coverage, default: .coveragerc
``--cov-annotate``
generate an annotated source code report
``--cov-html``
generate a html report
``--cov-xml``
generate an xml report
``--cov-annotate-dir=dir``
directory for the annotate report, default: %default
``--cov-html-dir=dir``
directory for the html report, default: coverage_html
``--cov-xml-file=path``
file for the xml report, default: coverage.xml
``--cov-data-file=path``
file containing coverage data, default: .coverage
``--cov-combine-each``
for dist=each mode produce a single combined report
``--cov-branch``
enable branch coverage
``--cov-pylib``
enable python library coverage
``--cov-timid``
enable slower and simpler tracing
``--cov-no-missing-lines``
disable showing missing lines, only relevant to the terminal report
``--cov-no-missing-files``
disable showing message about missing source files
``--cov-omit=prefix1,prefix2,...``
ignore files with these prefixes
``--cov-no-config``
disable coverage reading its config file
``--cov-config-file=path``
config file for coverage, default: %default
.. include:: links.txt .. include:: links.txt

26
doc/test/plugin/coverage.txt
View File

@ -6,10 +6,27 @@ Write and report coverage data with the 'coverage' package.
.. contents:: .. contents::
:local: :local:
Original code by Ross Lawley. Note: Original code by Ross Lawley.
Requires Ned Batchelder's excellent coverage: Install
http://nedbatchelder.com/code/coverage/ --------------
Use pip to (un)install::
pip install pytest-coverage
pip uninstall pytest-coverage
or alternatively use easy_install to install::
easy_install pytest-coverage
Usage
-------------
To get full test coverage reports for a particular package type::
py.test --cover-report=report
command line options command line options
-------------------- --------------------
@ -21,8 +38,11 @@ command line options
html: Directory for html output. html: Directory for html output.
report: Output a text report. report: Output a text report.
annotate: Annotate your source code for which lines were executed and which were not. annotate: Annotate your source code for which lines were executed and which were not.
xml: Output an xml report compatible with the cobertura plugin for hudson.
``--cover-directory=DIRECTORY`` ``--cover-directory=DIRECTORY``
Directory for the reports (html / annotate results) defaults to ./coverage Directory for the reports (html / annotate results) defaults to ./coverage
``--cover-xml-file=XML_FILE``
File for the xml report defaults to ./coverage.xml
``--cover-show-missing`` ``--cover-show-missing``
Show missing files Show missing files
``--cover-ignore-errors=IGNORE_ERRORS`` ``--cover-ignore-errors=IGNORE_ERRORS``

19
doc/test/plugin/figleaf.txt
View File

@ -6,16 +6,29 @@ report test coverage using the 'figleaf' package.
.. contents:: .. contents::
:local: :local:
Install
---------------
To install the plugin issue::
easy_install pytest-figleaf # or
pip install pytest-figleaf
and if you are using pip you can also uninstall::
pip uninstall pytest-figleaf
Usage Usage
--------------- ---------------
after pip or easy_install mediated installation of ``pytest-figleaf`` you can type:: After installation you can simply type::
py.test --figleaf [...] py.test --figleaf [...]
to enable figleaf coverage in your test run. A default ".figleaf" data file to enable figleaf coverage in your test run. A default ".figleaf" data file
and "html" directory will be created. You can use ``--fig-data`` and "html" directory will be created. You can use command line options
and ``fig-html`` to modify the paths. to control where data and html files are created.
command line options command line options
-------------------- --------------------

36
doc/test/plugin/links.txt
View File

@ -1,47 +1,47 @@
.. _`helpconfig`: helpconfig.html .. _`helpconfig`: helpconfig.html
.. _`pytest_recwarn.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_recwarn.py .. _`pytest_recwarn.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_recwarn.py
.. _`unittest`: unittest.html .. _`unittest`: unittest.html
.. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_monkeypatch.py .. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_monkeypatch.py
.. _`pytest_genscript.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_genscript.py .. _`pytest_genscript.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_genscript.py
.. _`pastebin`: pastebin.html .. _`pastebin`: pastebin.html
.. _`skipping`: skipping.html .. _`skipping`: skipping.html
.. _`genscript`: genscript.html .. _`genscript`: genscript.html
.. _`plugins`: index.html .. _`plugins`: index.html
.. _`mark`: mark.html .. _`mark`: mark.html
.. _`tmpdir`: tmpdir.html .. _`tmpdir`: tmpdir.html
.. _`pytest_doctest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_doctest.py .. _`pytest_doctest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_doctest.py
.. _`capture`: capture.html .. _`capture`: capture.html
.. _`pytest_nose.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_nose.py .. _`pytest_nose.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_nose.py
.. _`pytest_restdoc.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_restdoc.py .. _`pytest_restdoc.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_restdoc.py
.. _`restdoc`: restdoc.html .. _`restdoc`: restdoc.html
.. _`xdist`: xdist.html .. _`xdist`: xdist.html
.. _`pytest_pastebin.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_pastebin.py .. _`pytest_pastebin.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_pastebin.py
.. _`pytest_tmpdir.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_tmpdir.py .. _`pytest_tmpdir.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_tmpdir.py
.. _`terminal`: terminal.html .. _`terminal`: terminal.html
.. _`pytest_hooklog.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_hooklog.py .. _`pytest_hooklog.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_hooklog.py
.. _`capturelog`: capturelog.html .. _`capturelog`: capturelog.html
.. _`junitxml`: junitxml.html .. _`junitxml`: junitxml.html
.. _`pytest_skipping.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_skipping.py .. _`pytest_skipping.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_skipping.py
.. _`checkout the py.test development version`: ../../install.html#checkout .. _`checkout the py.test development version`: ../../install.html#checkout
.. _`pytest_helpconfig.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_helpconfig.py .. _`pytest_helpconfig.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_helpconfig.py
.. _`oejskit`: oejskit.html .. _`oejskit`: oejskit.html
.. _`doctest`: doctest.html .. _`doctest`: doctest.html
.. _`pytest_mark.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_mark.py .. _`pytest_mark.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_mark.py
.. _`get in contact`: ../../contact.html .. _`get in contact`: ../../contact.html
.. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_capture.py .. _`pytest_capture.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_capture.py
.. _`figleaf`: figleaf.html .. _`figleaf`: figleaf.html
.. _`customize`: ../customize.html .. _`customize`: ../customize.html
.. _`hooklog`: hooklog.html .. _`hooklog`: hooklog.html
.. _`pytest_terminal.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_terminal.py .. _`pytest_terminal.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_terminal.py
.. _`recwarn`: recwarn.html .. _`recwarn`: recwarn.html
.. _`pytest_pdb.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_pdb.py .. _`pytest_pdb.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_pdb.py
.. _`monkeypatch`: monkeypatch.html .. _`monkeypatch`: monkeypatch.html
.. _`coverage`: coverage.html .. _`coverage`: coverage.html
.. _`resultlog`: resultlog.html .. _`resultlog`: resultlog.html
.. _`cov`: cov.html .. _`cov`: cov.html
.. _`pytest_junitxml.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_junitxml.py .. _`pytest_junitxml.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_junitxml.py
.. _`django`: django.html .. _`django`: django.html
.. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_unittest.py .. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_unittest.py
.. _`nose`: nose.html .. _`nose`: nose.html
.. _`pytest_resultlog.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.2/py/_plugin/pytest_resultlog.py .. _`pytest_resultlog.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_resultlog.py
.. _`pdb`: pdb.html .. _`pdb`: pdb.html