add Meme's pytest-cov plugin to the plugin index page
--HG-- branch : trunk
This commit is contained in:
holger krekel
parent
6cc89c9fcf
commit
73d9900844
|
|
@ -5,7 +5,7 @@ WIDTH = 75
|
||||||
plugins = [
|
plugins = [
|
||||||
('advanced python testing',
|
('advanced python testing',
|
||||||
'skipping mark pdb figleaf '
|
'skipping mark pdb figleaf '
|
||||||
'monkeypatch coverage capture capturelog recwarn tmpdir',),
|
'monkeypatch coverage cov capture capturelog recwarn tmpdir',),
|
||||||
('distributed testing, CI and deployment',
|
('distributed testing, CI and deployment',
|
||||||
'xdist pastebin junitxml resultlog genscript',),
|
'xdist pastebin junitxml resultlog genscript',),
|
||||||
('testing domains and conventions codecheckers',
|
('testing domains and conventions codecheckers',
|
||||||
|
|
@ -24,6 +24,7 @@ externals = {
|
||||||
'figleaf': None,
|
'figleaf': None,
|
||||||
'capturelog': None,
|
'capturelog': None,
|
||||||
'coverage': None,
|
'coverage': None,
|
||||||
|
'cov': None,
|
||||||
'codecheckers': None,
|
'codecheckers': None,
|
||||||
'django': "for testing django applications",
|
'django': "for testing django applications",
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,250 @@
|
||||||
|
|
||||||
|
produce code coverage reports using the 'coverage' package, including support for distributed testing.
|
||||||
|
======================================================================================================
|
||||||
|
|
||||||
|
|
||||||
|
.. contents::
|
||||||
|
:local:
|
||||||
|
|
||||||
|
This plugin produces coverage reports using the coverage package. It
|
||||||
|
supports centralised testing and distributed testing in both load and
|
||||||
|
each modes.
|
||||||
|
|
||||||
|
All features offered by the coverage package should be available,
|
||||||
|
either through this plugin or through coverage's own config file.
|
||||||
|
|
||||||
|
|
||||||
|
Installation
|
||||||
|
------------
|
||||||
|
|
||||||
|
The `pytest-cov pypi`_ package may be installed / uninstalled with pip::
|
||||||
|
|
||||||
|
pip install pytest-cov
|
||||||
|
pip uninstall pytest-cov
|
||||||
|
|
||||||
|
Alternatively easy_install can be used::
|
||||||
|
|
||||||
|
easy_install pytest-cov
|
||||||
|
|
||||||
|
.. _`pytest-cov pypi`: http://pypi.python.org/pypi/pytest-cov/
|
||||||
|
|
||||||
|
|
||||||
|
Usage
|
||||||
|
-----
|
||||||
|
|
||||||
|
Centralised Testing
|
||||||
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Running centralised testing::
|
||||||
|
|
||||||
|
py.test --cov myproj tests/
|
||||||
|
|
||||||
|
Shows a terminal report::
|
||||||
|
|
||||||
|
-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
|
||||||
|
Name Stmts Exec Cover Missing
|
||||||
|
--------------------------------------------------
|
||||||
|
myproj/__init__ 2 2 100%
|
||||||
|
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
|
||||||
|
myproj/feature4286 94 87 92% 183-188, 197
|
||||||
|
--------------------------------------------------
|
||||||
|
TOTAL 353 333 94%
|
||||||
|
|
||||||
|
|
||||||
|
Distributed Testing
|
||||||
|
~~~~~~~~~~~~~~~~~~~
|
||||||
|
|
||||||
|
Distributed testing with dist mode set to load::
|
||||||
|
|
||||||
|
py.test --cov myproj -n 2 tests/
|
||||||
|
|
||||||
|
The results from the slaves will be combined like so::
|
||||||
|
|
||||||
|
-------------------- coverage: platform linux2, python 2.6.4-final-0 ---------------------
|
||||||
|
Name Stmts Exec Cover Missing
|
||||||
|
--------------------------------------------------
|
||||||
|
myproj/__init__ 2 2 100%
|
||||||
|
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
|
||||||
|
myproj/feature4286 94 87 92% 183-188, 197
|
||||||
|
--------------------------------------------------
|
||||||
|
TOTAL 353 333 94%
|
||||||
|
|
||||||
|
|
||||||
|
Distributed testing in each mode::
|
||||||
|
|
||||||
|
py.test --cov myproj --dist=each
|
||||||
|
--tx=popen//python=/usr/local/python265/bin/python
|
||||||
|
--tx=popen//python=/usr/local/python27b1/bin/python
|
||||||
|
tests/
|
||||||
|
|
||||||
|
Will produce a report for each slave::
|
||||||
|
|
||||||
|
-------------------- coverage: platform linux2, python 2.6.5-final-0 ---------------------
|
||||||
|
Name Stmts Exec Cover Missing
|
||||||
|
--------------------------------------------------
|
||||||
|
myproj/__init__ 2 2 100%
|
||||||
|
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
|
||||||
|
myproj/feature4286 94 87 92% 183-188, 197
|
||||||
|
--------------------------------------------------
|
||||||
|
TOTAL 353 333 94%
|
||||||
|
--------------------- coverage: platform linux2, python 2.7.0-beta-1 ---------------------
|
||||||
|
Name Stmts Exec Cover Missing
|
||||||
|
--------------------------------------------------
|
||||||
|
myproj/__init__ 2 2 100%
|
||||||
|
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
|
||||||
|
myproj/feature4286 94 87 92% 183-188, 197
|
||||||
|
--------------------------------------------------
|
||||||
|
TOTAL 353 333 94%
|
||||||
|
|
||||||
|
|
||||||
|
Distributed testing in each mode can also produce a single combined
|
||||||
|
report. This is useful to get coverage information spanning things
|
||||||
|
such as all python versions::
|
||||||
|
|
||||||
|
py.test --cov myproj --cov-combine-each --dist=each
|
||||||
|
--tx=popen//python=/usr/local/python265/bin/python
|
||||||
|
--tx=popen//python=/usr/local/python27b1/bin/python
|
||||||
|
tests/
|
||||||
|
|
||||||
|
Which looks like::
|
||||||
|
|
||||||
|
---------------------------------------- coverage ----------------------------------------
|
||||||
|
platform linux2, python 2.6.5-final-0
|
||||||
|
platform linux2, python 2.7.0-beta-1
|
||||||
|
Name Stmts Exec Cover Missing
|
||||||
|
--------------------------------------------------
|
||||||
|
myproj/__init__ 2 2 100%
|
||||||
|
myproj/myproj 257 244 94% 24-26, 99, 149, 233-236, 297-298, 369-370
|
||||||
|
myproj/feature4286 94 87 92% 183-188, 197
|
||||||
|
--------------------------------------------------
|
||||||
|
TOTAL 353 333 94%
|
||||||
|
|
||||||
|
|
||||||
|
Reporting
|
||||||
|
---------
|
||||||
|
|
||||||
|
By default a terminal report is output. This report can be disabled
|
||||||
|
if desired, such as when results are going to a continuous integration
|
||||||
|
system and the terminal output won't be seen.
|
||||||
|
|
||||||
|
In addition and without rerunning tests it is possible to generate
|
||||||
|
annotated source code, a html report and an xml report.
|
||||||
|
|
||||||
|
The directories for annotated source code and html reports can be
|
||||||
|
specified as can the file name for the xml report.
|
||||||
|
|
||||||
|
Since testing often takes a non trivial amount of time at the end of
|
||||||
|
testing any / all of the reports may be generated.
|
||||||
|
|
||||||
|
|
||||||
|
Coverage Data File
|
||||||
|
------------------
|
||||||
|
|
||||||
|
During testing there may be many data files with coverage data. These
|
||||||
|
will have unique suffixes and will be combined at the end of testing.
|
||||||
|
|
||||||
|
Upon completion, for --dist=load (and also for --dist=each when the
|
||||||
|
--cov-combine-each option is used) there will only be one data file.
|
||||||
|
|
||||||
|
For --dist=each there may be many data files where each one will have
|
||||||
|
the platform / python version info appended to the name.
|
||||||
|
|
||||||
|
These data files are left at the end of testing so that it is possible
|
||||||
|
to use normal coverage tools to examine them.
|
||||||
|
|
||||||
|
At the beginning of testing any data files that are about to be used
|
||||||
|
will first be erased so ensure the data is clean for each test run.
|
||||||
|
|
||||||
|
It is possible to set the name of the data file. If needed the
|
||||||
|
platform / python version will be appended automatically to this name.
|
||||||
|
|
||||||
|
|
||||||
|
Coverage Config File
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
Coverage by default will read its own config file. An alternative
|
||||||
|
file name may be specified or reading config can be disabled entirely.
|
||||||
|
|
||||||
|
Care has been taken to ensure that the coverage env vars and config
|
||||||
|
file options work the same under this plugin as they do under coverage
|
||||||
|
itself.
|
||||||
|
|
||||||
|
Since options may be specified in different ways the order of
|
||||||
|
precedence between pytest-cov and coverage from highest to lowest is:
|
||||||
|
|
||||||
|
1. pytest command line
|
||||||
|
2. pytest env var
|
||||||
|
3. pytest conftest
|
||||||
|
4. coverage env var
|
||||||
|
5. coverage config file
|
||||||
|
6. coverage default
|
||||||
|
|
||||||
|
|
||||||
|
Limitations
|
||||||
|
-----------
|
||||||
|
|
||||||
|
For distributed testing the slaves must have the pytest-cov package
|
||||||
|
installed. This is needed since the plugin must be registered through
|
||||||
|
setuptools / distribute for pytest to start the plugin on the slave.
|
||||||
|
|
||||||
|
|
||||||
|
Acknowledgements
|
||||||
|
----------------
|
||||||
|
|
||||||
|
Holger Krekel for pytest with its distributed testing support.
|
||||||
|
|
||||||
|
Ned Batchelder for coverage and its ability to combine the coverage
|
||||||
|
results of parallel runs.
|
||||||
|
|
||||||
|
Whilst this plugin has been built fresh from the ground up to support
|
||||||
|
distributed testing it has been influenced by the work done on
|
||||||
|
pytest-coverage (Ross Lawley, James Mills, Holger Krekel) and
|
||||||
|
nose-cover (Jason Pellerin) which are other coverage plugins for
|
||||||
|
pytest and nose respectively.
|
||||||
|
|
||||||
|
No doubt others have contributed to these tools as well.
|
||||||
|
|
||||||
|
command line options
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
|
||||||
|
``--cov-on``
|
||||||
|
enable coverage, only needed if not specifying any --cov options
|
||||||
|
``--cov=package``
|
||||||
|
collect coverage for the specified package (multi-allowed)
|
||||||
|
``--cov-no-terminal``
|
||||||
|
disable printing a report on the terminal
|
||||||
|
``--cov-annotate``
|
||||||
|
generate an annotated source code report
|
||||||
|
``--cov-html``
|
||||||
|
generate a html report
|
||||||
|
``--cov-xml``
|
||||||
|
generate an xml report
|
||||||
|
``--cov-annotate-dir=dir``
|
||||||
|
directory for the annotate report, default: %default
|
||||||
|
``--cov-html-dir=dir``
|
||||||
|
directory for the html report, default: coverage_html
|
||||||
|
``--cov-xml-file=path``
|
||||||
|
file for the xml report, default: coverage.xml
|
||||||
|
``--cov-data-file=path``
|
||||||
|
file containing coverage data, default: .coverage
|
||||||
|
``--cov-combine-each``
|
||||||
|
for dist=each mode produce a single combined report
|
||||||
|
``--cov-branch``
|
||||||
|
enable branch coverage
|
||||||
|
``--cov-pylib``
|
||||||
|
enable python library coverage
|
||||||
|
``--cov-timid``
|
||||||
|
enable slower and simpler tracing
|
||||||
|
``--cov-no-missing-lines``
|
||||||
|
disable showing missing lines, only relevant to the terminal report
|
||||||
|
``--cov-no-missing-files``
|
||||||
|
disable showing message about missing source files
|
||||||
|
``--cov-omit=prefix1,prefix2,...``
|
||||||
|
ignore files with these prefixes
|
||||||
|
``--cov-no-config``
|
||||||
|
disable coverage reading its config file
|
||||||
|
``--cov-config-file=path``
|
||||||
|
config file for coverage, default: %default
|
||||||
|
|
||||||
|
.. include:: links.txt
|
||||||
|
|
@ -14,6 +14,8 @@ monkeypatch_ safely patch object attributes, dicts and environment variables.
|
||||||
|
|
||||||
coverage_ (external) Write and report coverage data with the 'coverage' package.
|
coverage_ (external) Write and report coverage data with the 'coverage' package.
|
||||||
|
|
||||||
|
cov_ (external) produce code coverage reports using the 'coverage' package, including support for distributed testing.
|
||||||
|
|
||||||
capture_ configurable per-test stdout/stderr capturing mechanisms.
|
capture_ configurable per-test stdout/stderr capturing mechanisms.
|
||||||
|
|
||||||
capturelog_ (external) capture output of logging module.
|
capturelog_ (external) capture output of logging module.
|
||||||
|
|
|
||||||
|
|
@ -38,6 +38,7 @@
|
||||||
.. _`monkeypatch`: monkeypatch.html
|
.. _`monkeypatch`: monkeypatch.html
|
||||||
.. _`coverage`: coverage.html
|
.. _`coverage`: coverage.html
|
||||||
.. _`resultlog`: resultlog.html
|
.. _`resultlog`: resultlog.html
|
||||||
|
.. _`cov`: cov.html
|
||||||
.. _`pytest_junitxml.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.1/py/_plugin/pytest_junitxml.py
|
.. _`pytest_junitxml.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.1/py/_plugin/pytest_junitxml.py
|
||||||
.. _`django`: django.html
|
.. _`django`: django.html
|
||||||
.. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.1/py/_plugin/pytest_unittest.py
|
.. _`pytest_unittest.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.1/py/_plugin/pytest_unittest.py
|
||||||
|
|
|
||||||
Loading…
Reference in New Issue