p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merged branch features into features

This commit is contained in:
Steffen Allner 2016-07-03 21:07:02 +02:00
parent 32ca5cdb09 7927dff8a1
commit f7b5bb2f97
35 changed files with 536 additions and 535 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
AUTHORS
View File

@ -100,6 +100,7 @@ Samuele Pedroni
Steffen Allner
Stephan Obermann
Tareq Alayan
Ted Xiao
Simon Gomizelj
Stefano Taschini
Stefan Farmbauer
@ -109,3 +110,4 @@ Trevor Bekolay
Vasily Kuznetsov
Wouter van Ackooy
Bernard Pratz
Stefan Zimmermann

64
CHANGELOG.rst
View File

@ -1,3 +1,40 @@
3.0.0.dev1
==========
**Changes**
*
*
*
* Fix (`#607`_): pytest.skip() is no longer allowed at module level to
prevent misleading use as test function decorator. When used at a module
level an error will be raised during collection.
Thanks `@omarkohl`_ for the complete PR (`#1519`_).
*
**Incompatible changes**
* Removing the following deprecated commandline options
* ``--genscript``
* ``--no-assert``
* ``--nomagic``
* ``--report``
Thanks to `@RedBeardCode`_ for the PR(`#1664`_)
* removed support code for python 3 < 3.3 addressing (`#1627`_)
.. _#607: https://github.com/pytest-dev/pytest/issues/607
.. _#1519: https://github.com/pytest-dev/pytest/pull/1519
.. _#1664: https://github.com/pytest-dev/pytest/pull/1664
.. _#1627: https://github.com/pytest-dev/pytest/pull/1627
2.10.0.dev1
===========
@ -58,6 +95,19 @@
finalizer and has access to the fixture's result cache.
Thanks `@d6e`_, `@sallner`_
* Issue a warning for asserts whose test is a tuple literal. Such asserts will
never fail because tuples are always truthy and are usually a mistake
(see `#1562`_). Thanks `@kvas-it`_, for the PR.
* New cli flag ``--override-ini`` or ``-o`` that overrides values from the ini file.
Example '-o xfail_strict=True'. A complete ini-options can be viewed
by py.test --help. Thanks `@blueyed`_ and `@fengxx`_ for the PR
* Remove all py.test-X* entry points. The versioned, suffixed entry points
were never documented and a leftover from a pre-virtualenv era. These entry
points also created broken entry points in wheels, so removing them also
removes a source of confusion for users (`#1632`_).
Thanks `@obestwalter`_ for the PR.
**Changes**
@ -118,8 +168,11 @@
Thanks `@Vogtinator`_ for reporting. Thanks to `@RedBeardCode`_ and
`@tomviner`_ for PR.
*
* Add proposal to docs for a new feature that enables users to combine multiple
fixtures into one. Thanks to `@hpk42`_ and `@hackebrot`_.
.. _#1632: https://github.com/pytest-dev/pytest/issues/1632
.. _#1580: https://github.com/pytest-dev/pytest/pull/1580
.. _#1605: https://github.com/pytest-dev/pytest/issues/1605
.. _#1597: https://github.com/pytest-dev/pytest/pull/1597
@ -131,6 +184,8 @@
.. _@nikratio: https://github.com/nikratio
.. _@RedBeardCode: https://github.com/RedBeardCode
.. _@Vogtinator: https://github.com/Vogtinator
.. _@blueyed: https://github.com/blueyed
.. _@fengxx: https://github.com/fengxx
* Fix `#1421`_: Exit tests if a collection error occurs and add
``--continue-on-collection-errors`` option to restore previous behaviour.
@ -139,6 +194,11 @@
*
* ``OptionGroup.addoption()`` now checks if option names were already
added before, to make it easier to track down issues like `#1618`_.
Before, you only got exceptions later from ``argparse`` library,
giving no clue about the actual reason for double-added options.
.. _@milliams: https://github.com/milliams
.. _@csaftoiu: https://github.com/csaftoiu
.. _@flub: https://github.com/flub
@ -172,10 +232,12 @@
.. _#1619: https://github.com/pytest-dev/pytest/issues/1619
.. _#372: https://github.com/pytest-dev/pytest/issues/372
.. _#1544: https://github.com/pytest-dev/pytest/issues/1544
.. _#1562: https://github.com/pytest-dev/pytest/issues/1562
.. _#1616: https://github.com/pytest-dev/pytest/pull/1616
.. _#1628: https://github.com/pytest-dev/pytest/pull/1628
.. _#1629: https://github.com/pytest-dev/pytest/issues/1629
.. _#1633: https://github.com/pytest-dev/pytest/pull/1633
.. _#1618: https://github.com/pytest-dev/pytest/issues/1618
**Bug Fixes**

8
README.rst
View File

@ -17,7 +17,7 @@
:target: https://ci.appveyor.com/project/pytestbot/pytest
The ``pytest`` framework makes it easy to write small tests, yet
scales to support complex functional testing for applications and libraries.
scales to support complex functional testing for applications and libraries.
An example of a simple test:
@ -35,7 +35,7 @@ To execute it::
$ pytest
======= test session starts ========
platform linux -- Python 3.4.3, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
platform linux -- Python 3.4.3, pytest-2.8.5, py-1.4.31, pluggy-0.3.1
collected 1 items
test_sample.py F
@ -52,7 +52,7 @@ To execute it::
======= 1 failed in 0.12 seconds ========
Due to ``pytest``'s detailed assertion introspection, only plain ``assert`` statements are used. See `getting-started <http://pytest.org/latest/getting-started.html#our-first-test-run>`_ for more examples.
Features
--------
@ -69,7 +69,7 @@ Features
- Can run `unittest <http://pytest.org/latest/unittest.html>`_ (or trial),
`nose <http://pytest.org/latest/nose.html>`_ test suites out of the box;
- Python2.6+, Python3.2+, PyPy-2.3, Jython-2.5 (untested);
- Python2.6+, Python3.3+, PyPy-2.3, Jython-2.5 (untested);
- Rich plugin architecture, with over 150+ `external plugins <http://pytest.org/latest/plugins.html#installing-external-plugins-searching>`_ and thriving community;

13
_pytest/assertion/__init__.py
View File

@ -25,15 +25,6 @@ def pytest_addoption(parser):
'rewrite' (the default) rewrites assert
statements in test modules on import to
provide assert expression information. """)
group.addoption('--no-assert',
action="store_true",
default=False,
dest="noassert",
help="DEPRECATED equivalent to --assert=plain")
group.addoption('--nomagic', '--no-magic',
action="store_true",
default=False,
help="DEPRECATED equivalent to --assert=plain")
class AssertionState:
@ -48,10 +39,6 @@ class AssertionState:
def pytest_load_initial_conftests(early_config, parser, args):
ns, ns_unknown_args = parser.parse_known_and_unknown_args(args)
mode = ns.assertmode
no_assert = ns.noassert
no_magic = ns.nomagic
if no_assert or no_magic:
mode = "plain"
if mode == "rewrite":
try:
import ast # noqa

20
_pytest/assertion/rewrite.py
View File

@ -125,7 +125,7 @@ class AssertionRewritingHook(object):
co = _read_pyc(fn_pypath, pyc, state.trace)
if co is None:
state.trace("rewriting %r" % (fn,))
source_stat, co = _rewrite_test(state, fn_pypath)
source_stat, co = _rewrite_test(self.config, fn_pypath)
if co is None:
# Probably a SyntaxError in the test.
return None
@ -252,8 +252,9 @@ N = "\n".encode("utf-8")
cookie_re = re.compile(r"^[ \t\f]*#.*coding[:=][ \t]*[-\w.]+")
BOM_UTF8 = '\xef\xbb\xbf'
def _rewrite_test(state, fn):
def _rewrite_test(config, fn):
"""Try to read and rewrite *fn* and return the code object."""
state = config._assertstate
try:
stat = fn.stat()
source = fn.read("rb")
@ -298,7 +299,7 @@ def _rewrite_test(state, fn):
# Let this pop up again in the real import.
state.trace("failed to parse: %r" % (fn,))
return None, None
rewrite_asserts(tree)
rewrite_asserts(tree, fn, config)
try:
co = compile(tree, fn.strpath, "exec")
except SyntaxError:
@ -354,9 +355,9 @@ def _read_pyc(source, pyc, trace=lambda x: None):
return co
def rewrite_asserts(mod):
def rewrite_asserts(mod, module_path=None, config=None):
"""Rewrite the assert statements in mod."""
AssertionRewriter().run(mod)
AssertionRewriter(module_path, config).run(mod)
def _saferepr(obj):
@ -543,6 +544,11 @@ class AssertionRewriter(ast.NodeVisitor):
"""
def __init__(self, module_path, config):
super(AssertionRewriter, self).__init__()
self.module_path = module_path
self.config = config
def run(self, mod):
"""Find all assert statements in *mod* and rewrite them."""
if not mod.body:
@ -683,6 +689,10 @@ class AssertionRewriter(ast.NodeVisitor):
the expression is false.
"""
if isinstance(assert_.test, ast.Tuple) and self.config is not None:
fslocation = (self.module_path, assert_.lineno)
self.config.warn('R1', 'assertion is always true, perhaps '
'remove parentheses?', fslocation=fslocation)
self.statements = []
self.variables = []
self.variable_counter = itertools.count()

41
_pytest/config.py
View File

@ -64,8 +64,9 @@ _preinit = []
default_plugins = (
"mark main terminal runner python pdb unittest capture skipping "
"tmpdir monkeypatch recwarn pastebin helpconfig nose assertion genscript "
"junitxml resultlog doctest cacheprovider setuponly setupplan").split()
"tmpdir monkeypatch recwarn pastebin helpconfig nose assertion "
"junitxml resultlog doctest cacheprovider freeze_support "
"setuponly setupplan").split()
builtin_plugins = set(default_plugins)
builtin_plugins.add("pytester")
@ -686,6 +687,10 @@ class OptionGroup:
results in help showing '--two-words' only, but --twowords gets
accepted **and** the automatic destination is in args.twowords
"""
conflict = set(optnames).intersection(
name for opt in self.options for name in opt.names())
if conflict:
raise ValueError("option names %s already added" % conflict)
option = Argument(*optnames, **attrs)
self._addoption_instance(option, shortupper=False)
@ -999,14 +1004,16 @@ class Config(object):
description, type, default = self._parser._inidict[name]
except KeyError:
raise ValueError("unknown configuration value: %r" %(name,))
try:
value = self.inicfg[name]
except KeyError:
if default is not None:
return default
if type is None:
return ''
return []
value = self._get_override_ini_value(name)
if value is None:
try:
value = self.inicfg[name]
except KeyError:
if default is not None:
return default
if type is None:
return ''
return []
if type == "pathlist":
dp = py.path.local(self.inicfg.config.path).dirpath()
l = []
@ -1037,6 +1044,20 @@ class Config(object):
l.append(relroot)
return l
def _get_override_ini_value(self, name):
value = None
# override_ini is a list of list, to support both -o foo1=bar1 foo2=bar2 and
# and -o foo1=bar1 -o foo2=bar2 options
# always use the last item if multiple value set for same ini-name,
# e.g. -o foo=bar1 -o foo=bar2 will set foo to bar2
if self.getoption("override_ini", None):
for ini_config_list in self.option.override_ini:
for ini_config in ini_config_list:
(key, user_ini_value) = ini_config.split("=", 1)
if key == name:
value = user_ini_value
return value
def getoption(self, name, default=notset, skip=False):
""" return command line option value.

45
_pytest/freeze_support.py Normal file
View File

@ -0,0 +1,45 @@
"""
Provides a function to report all internal modules for using freezing tools
pytest
"""
def pytest_namespace():
return {'freeze_includes': freeze_includes}
def freeze_includes():
"""
Returns a list of module names used by py.test that should be
included by cx_freeze.
"""
import py
import _pytest
result = list(_iter_all_modules(py))
result += list(_iter_all_modules(_pytest))
return result
def _iter_all_modules(package, prefix=''):
"""
Iterates over the names of all modules that can be found in the given
package, recursively.
Example:
_iter_all_modules(_pytest) ->
['_pytest.assertion.newinterpret',
'_pytest.capture',
'_pytest.core',
...
]
"""
import os
import pkgutil
if type(package) is not str:
path, prefix = package.__path__[0], package.__name__ + '.'
else:
path = package
for _, name, is_package in pkgutil.iter_modules([path]):
if is_package:
for m in _iter_all_modules(os.path.join(path, name), prefix=name + '.'):
yield prefix + m
else:
yield prefix + name

132
_pytest/genscript.py
View File

@ -1,132 +0,0 @@
""" (deprecated) generate a single-file self-contained version of pytest """
import os
import sys
import pkgutil
import py
import _pytest
def find_toplevel(name):
for syspath in sys.path:
base = py.path.local(syspath)
lib = base/name
if lib.check(dir=1):
return lib
mod = base.join("%s.py" % name)
if mod.check(file=1):
return mod
raise LookupError(name)
def pkgname(toplevel, rootpath, path):
parts = path.parts()[len(rootpath.parts()):]
return '.'.join([toplevel] + [x.purebasename for x in parts])
def pkg_to_mapping(name):
toplevel = find_toplevel(name)
name2src = {}
if toplevel.check(file=1): # module
name2src[toplevel.purebasename] = toplevel.read()
else: # package
for pyfile in toplevel.visit('*.py'):
pkg = pkgname(name, toplevel, pyfile)
name2src[pkg] = pyfile.read()
# with wheels py source code might be not be installed
# and the resulting genscript is useless, just bail out.
assert name2src, "no source code found for %r at %r" %(name, toplevel)
return name2src
def compress_mapping(mapping):
import base64, pickle, zlib
data = pickle.dumps(mapping, 2)
data = zlib.compress(data, 9)
data = base64.encodestring(data)
data = data.decode('ascii')
return data
def compress_packages(names):
mapping = {}
for name in names:
mapping.update(pkg_to_mapping(name))
return compress_mapping(mapping)
def generate_script(entry, packages):
data = compress_packages(packages)
tmpl = py.path.local(__file__).dirpath().join('standalonetemplate.py')
exe = tmpl.read()
exe = exe.replace('@SOURCES@', data)
exe = exe.replace('@ENTRY@', entry)
return exe
def pytest_addoption(parser):
group = parser.getgroup("debugconfig")
group.addoption("--genscript", action="store", default=None,
dest="genscript", metavar="path",
help="create standalone pytest script at given target path.")
def pytest_cmdline_main(config):
import _pytest.config
genscript = config.getvalue("genscript")
if genscript:
tw = _pytest.config.create_terminal_writer(config)
tw.line("WARNING: usage of genscript is deprecated.",
red=True)
deps = ['py', '_pytest', 'pytest'] # pluggy is vendored
if sys.version_info < (2,7):
deps.append("argparse")
tw.line("generated script will run on python2.6-python3.3++")
else:
tw.line("WARNING: generated script will not run on python2.6 "
"due to 'argparse' dependency. Use python2.6 "
"to generate a python2.6 compatible script", red=True)
script = generate_script(
'import pytest; raise SystemExit(pytest.cmdline.main())',
deps,
)
genscript = py.path.local(genscript)
genscript.write(script)
tw.line("generated pytest standalone script: %s" % genscript,
bold=True)
return 0
def pytest_namespace():
return {'freeze_includes': freeze_includes}
def freeze_includes():
"""
Returns a list of module names used by pytest that should be
included by cx_freeze.
"""
result = list(_iter_all_modules(py))
result += list(_iter_all_modules(_pytest))
return result
def _iter_all_modules(package, prefix=''):
"""
Iterates over the names of all modules that can be found in the given
package, recursively.
Example:
_iter_all_modules(_pytest) ->
['_pytest.assertion.newinterpret',
'_pytest.capture',
'_pytest.core',
...
]
"""
if type(package) is not str:
path, prefix = package.__path__[0], package.__name__ + '.'
else:
path = package
for _, name, is_package in pkgutil.iter_modules([path]):
if is_package:
for m in _iter_all_modules(os.path.join(path, name), prefix=name + '.'):
yield prefix + m
else:
yield prefix + name

4
_pytest/helpconfig.py
View File

@ -20,6 +20,10 @@ def pytest_addoption(parser):
group.addoption('--debug',
action="store_true", dest="debug", default=False,
help="store internal tracing debug information in 'pytestdebug.log'.")
# support for "--overwrite-ini ININAME=INIVALUE" to override values from the ini file
# Example '-o xfail_strict=True'.
group._addoption('-o', '--override-ini', nargs='*', dest="override_ini", action="append",
help="overrides ini values which do not have a separate command-line flag")
@pytest.hookimpl(hookwrapper=True)

6
_pytest/python.py
View File

@ -656,6 +656,12 @@ class Module(pytest.File, PyCollector):
"Make sure your test modules/packages have valid Python names."
% (self.fspath, exc or exc_class)
)
except _pytest.runner.Skipped:
raise self.CollectError(
"Using @pytest.skip outside a test (e.g. as a test function "
"decorator) is not allowed. Use @pytest.mark.skip or "
"@pytest.mark.skipif instead."
)
#print "imported test module", mod
self.config.pluginmanager.consider_module(mod)
return mod

89
_pytest/standalonetemplate.py
View File

@ -1,89 +0,0 @@
#! /usr/bin/env python
# Hi There!
# You may be wondering what this giant blob of binary data here is, you might
# even be worried that we're up to something nefarious (good for you for being
# paranoid!). This is a base64 encoding of a zip file, this zip file contains
# a fully functional basic pytest script.
#
# Pytest is a thing that tests packages, pytest itself is a package that some-
# one might want to install, especially if they're looking to run tests inside
# some package they want to install. Pytest has a lot of code to collect and
# execute tests, and other such sort of "tribal knowledge" that has been en-
# coded in its code base. Because of this we basically include a basic copy
# of pytest inside this blob. We do this because it let's you as a maintainer
# or application developer who wants people who don't deal with python much to
# easily run tests without installing the complete pytest package.
#
# If you're wondering how this is created: you can create it yourself if you
# have a complete pytest installation by using this command on the command-
# line: ``pytest --genscript=runtests.py``.
sources = """
@SOURCES@"""
import sys
import base64
import zlib
class DictImporter(object):
def __init__(self, sources):
self.sources = sources
def find_module(self, fullname, path=None):
if fullname == "argparse" and sys.version_info >= (2,7):
# we were generated with <python2.7 (which pulls in argparse)
# but we are running now on a stdlib which has it, so use that.
return None
if fullname in self.sources:
return self
if fullname + '.__init__' in self.sources:
return self
return None
def load_module(self, fullname):
# print "load_module:", fullname
from types import ModuleType
try:
s = self.sources[fullname]
is_pkg = False
except KeyError:
s = self.sources[fullname + '.__init__']
is_pkg = True
co = compile(s, fullname, 'exec')
module = sys.modules.setdefault(fullname, ModuleType(fullname))
module.__file__ = "%s/%s" % (__file__, fullname)
module.__loader__ = self
if is_pkg:
module.__path__ = [fullname]
do_exec(co, module.__dict__) # noqa
return sys.modules[fullname]
def get_source(self, name):
res = self.sources.get(name)
if res is None:
res = self.sources.get(name + '.__init__')
return res
if __name__ == "__main__":
try:
import pkg_resources # noqa
except ImportError:
sys.stderr.write("ERROR: setuptools not installed\n")
sys.exit(2)
if sys.version_info >= (3, 0):
exec("def do_exec(co, loc): exec(co, loc)\n")
import pickle
sources = sources.encode("ascii") # ensure bytes
sources = pickle.loads(zlib.decompress(base64.decodebytes(sources)))
else:
import cPickle as pickle
exec("def do_exec(co, loc): exec co in loc\n")
sources = pickle.loads(zlib.decompress(base64.decodestring(sources)))
importer = DictImporter(sources)
sys.meta_path.insert(0, importer)
entry = "@ENTRY@"
do_exec(entry, locals()) # noqa

14
_pytest/terminal.py
View File

@ -27,9 +27,6 @@ def pytest_addoption(parser):
group._addoption('-l', '--showlocals',
action="store_true", dest="showlocals", default=False,
help="show locals in tracebacks (disabled by default).")
group._addoption('--report',
action="store", dest="report", default=None, metavar="opts",
help="(deprecated, use -r)")
group._addoption('--tb', metavar="style",
action="store", dest="tbstyle", default='auto',
choices=['auto', 'long', 'short', 'no', 'line', 'native'],
@ -54,17 +51,6 @@ def pytest_configure(config):
def getreportopt(config):
reportopts = ""
optvalue = config.option.report
if optvalue:
py.builtin.print_("DEPRECATED: use -r instead of --report option.",
file=sys.stderr)
if optvalue:
for setting in optvalue.split(","):
setting = setting.strip()
if setting == "skipped":
reportopts += "s"
elif setting == "xfailed":
reportopts += "x"
reportchars = config.option.reportchars
if reportchars:
for char in reportchars:

3
doc/en/assert.rst
View File

@ -314,3 +314,6 @@ For further information, Benjamin Peterson wrote up `Behind the scenes of pytest
.. versionchanged:: 2.1
Introduce the ``--assert`` option. Deprecate ``--no-assert`` and
``--nomagic``.
.. versionchanged:: 3.0
Removes the ``--no-assert`` and``--nomagic`` options.

38
doc/en/getting-started.rst
View File

@ -193,45 +193,9 @@ Where to go next
Here are a few suggestions where to go next:
* :ref:`cmdline` for command line invocation examples
* :ref:`good practices <goodpractices>` for virtualenv, test layout, genscript support
* :ref:`good practices <goodpractices>` for virtualenv, test layout
* :ref:`fixtures` for providing a functional baseline to your tests
* :ref:`apiref` for documentation and examples on using ``pytest``
* :ref:`plugins` managing and writing plugins
.. _`installation issues`:
Known Installation issues
------------------------------
easy_install or pip not found?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.. _`install pip`: http://www.pip-installer.org/en/latest/index.html
`Install pip`_ for a state of the art python package installer.
Install `setuptools`_ to get ``easy_install`` which allows to install
``.egg`` binary format packages in addition to source-based ones.
pytest not found on Windows despite installation?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
.. _`Python for Windows`: http://www.imladris.com/Scripts/PythonForWindows.html
- **Windows**: If "easy_install" or "pytest" are not found
you need to add the Python script path to your ``PATH``, see here:
`Python for Windows`_. You may alternatively use an `ActivePython install`_
which does this for you automatically.
.. _`ActivePython install`: http://www.activestate.com/activepython/downloads
.. _`Jython does not create command line launchers`: http://bugs.jython.org/issue1491
- **Jython2.5.1 on Windows XP**: `Jython does not create command line launchers`_
so ``pytest`` will not work correctly. You may install pytest on
CPython and type ``pytest --genscript=mytest`` and then use
``jython mytest`` to run your tests with Jython using ``pytest``.
:ref:`examples` for more complex examples
.. include:: links.inc

34
doc/en/goodpractices.rst
View File

@ -243,38 +243,4 @@ using the ``--pytest-args`` or ``-a`` command-line option. For example::
is equivalent to running ``pytest --durations=5``.
.. _standalone:
.. _`genscript method`:
(deprecated) Create a pytest standalone script
-----------------------------------------------
.. deprecated:: 2.8
.. note::
``genscript`` has been deprecated because:
* It cannot support plugins, rendering its usefulness extremely limited;
* Tooling has become much better since ``genscript`` was introduced;
* It is possible to build a zipped ``pytest`` application without the
shortcomings above.
There's no planned version in which this command will be removed
at the moment of this writing, but its use is discouraged for new
applications.
If you are a maintainer or application developer and want people
who don't deal with python much to easily run tests you may generate
a standalone ``pytest`` script::
pytest --genscript=runtests.py
This generates a ``runtests.py`` script which is a fully functional basic
``pytest`` script, running unchanged under Python2 and Python3.
You can tell people to download the script and then e.g. run it like this::
python runtests.py
.. include:: links.inc

4
doc/en/index.rst
View File

@ -6,7 +6,7 @@ pytest: helps you write better programs
**a mature full-featured Python testing tool**
- runs on Posix/Windows, Python 2.6-3.5, PyPy and (possibly still) Jython-2.5.1
- runs on Posix/Windows, Python 2.6, 2.7 and 3.3-3.5, PyPy and (possibly still) Jython-2.5.1
- free and open source software, distributed under the terms of the :ref:`MIT license <license>`
- **well tested** with more than a thousand tests against itself
- **strict backward compatibility policy** for safe pytest upgrades
@ -57,5 +57,3 @@ pytest: helps you write better programs
.. _`easy`: http://bruynooghe.blogspot.com/2009/12/skipping-slow-test-by-default-in-pytest.html

1
doc/en/plugins.rst
View File

@ -138,7 +138,6 @@ in the `pytest repository <https://github.com/pytest-dev/pytest>`_.
_pytest.capture
_pytest.config
_pytest.doctest
_pytest.genscript
_pytest.helpconfig
_pytest.junitxml
_pytest.mark

148
doc/en/proposals/parametrize_with_fixtures.rst Normal file
View File

@ -0,0 +1,148 @@
=========================
Parametrize with fixtures
=========================
Problem
-------
As a user I have functional tests that I would like to run against various
scenarios.
In this particular example we want to generate a new project based on a
cookiecutter template. We want to test default values but also data that
emulates user input.
- use default values
- emulate user input
- specify 'author'
- specify 'project_slug'
- specify 'author' and 'project_slug'
This is how a functional test could look like:
.. code-block:: python
import pytest
@pytest.fixture
def default_context():
return {'extra_context': {}}
@pytest.fixture(params=[
{'author': 'alice'},
{'project_slug': 'helloworld'},
{'author': 'bob', 'project_slug': 'foobar'},
])
def extra_context(request):
return {'extra_context': request.param}
@pytest.fixture(params=['default', 'extra'])
def context(request):
if request.param == 'default':
return request.getfuncargvalue('default_context')
else:
return request.getfuncargvalue('extra_context')
def test_generate_project(cookies, context):
"""Call the cookiecutter API to generate a new project from a
template.
"""
result = cookies.bake(extra_context=context)
assert result.exit_code == 0
assert result.exception is None
assert result.project.isdir()
Issues
------
* By using ``request.getfuncargvalue()`` we rely on actual fixture function
execution to know what fixtures are involved, due to it's dynamic nature
* More importantly, ``request.getfuncargvalue()`` cannot be combined with
parametrized fixtures, such as ``extra_context``
* This is very inconvenient if you wish to extend an existing test suite by
certain parameters for fixtures that are already used by tests
pytest version 3.0 reports an error if you try to run above code::
Failed: The requested fixture has no parameter defined for the current
test.
Requested fixture 'extra_context'
Proposed solution
-----------------
A new function that can be used in modules can be used to dynamically define
fixtures from existing ones.
.. code-block:: python
pytest.define_combined_fixture(
name='context',
fixtures=['default_context', 'extra_context'],
)
The new fixture ``context`` inherits the scope from the used fixtures and yield
the following values.
- ``{}``
- ``{'author': 'alice'}``
- ``{'project_slug': 'helloworld'}``
- ``{'author': 'bob', 'project_slug': 'foobar'}``
Alternative approach
--------------------
A new helper function named ``fixture_request`` tells pytest to yield all
parameters of a fixture.
.. code-block:: python
@pytest.fixture(params=[
pytest.fixture_request('default_context'),
pytest.fixture_request('extra_context'),
])
def context(request):
"""Returns all values for ``default_context``, one-by-one before it
does the same for ``extra_context``.
request.param:
- {}
- {'author': 'alice'}
- {'project_slug': 'helloworld'}
- {'author': 'bob', 'project_slug': 'foobar'}
"""
return request.param
The same helper can be used in combination with ``pytest.mark.parametrize``.
.. code-block:: python
@pytest.mark.parametrize(
'context, expected_response_code',
[
(pytest.fixture_request('default_context'), 0),
(pytest.fixture_request('extra_context'), 0),
],
)
def test_generate_project(cookies, context, exit_code):
"""Call the cookiecutter API to generate a new project from a
template.
"""
result = cookies.bake(extra_context=context)
assert result.exit_code == exit_code

28
doc/en/test/plugin/genscript.rst
View File

@ -1,28 +0,0 @@
(deprecated) generate standalone test script to be distributed along with an application.
============================================================================
.. contents::
:local:
command line options
--------------------
``--genscript=path``
create standalone ``pytest`` script at given target path.
Start improving this plugin in 30 seconds
=========================================
1. Download `pytest_genscript.py`_ plugin source code
2. put it somewhere as ``pytest_genscript.py`` into your import path
3. a subsequent ``pytest`` run will use your local version
Checkout customize_, other plugins_ or `get in contact`_.
.. include:: links.txt

2
doc/en/test/plugin/helpconfig.rst
View File

@ -18,8 +18,6 @@ command line options
early-load given plugin (multi-allowed).
``--trace-config``
trace considerations of conftest.py files.
``--nomagic``
don't reinterpret asserts, no traceback cutting.
``--debug``
generate and show internal debugging information.
``--help-config``

2
doc/en/test/plugin/links.rst
View File

@ -2,10 +2,8 @@
.. _`pytest_recwarn.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_recwarn.py
.. _`unittest`: unittest.html
.. _`pytest_monkeypatch.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_monkeypatch.py
.. _`pytest_genscript.py`: http://bitbucket.org/hpk42/py-trunk/raw/1.3.4/py/_plugin/pytest_genscript.py
.. _`pastebin`: pastebin.html
.. _`skipping`: skipping.html
.. _`genscript`: genscript.html
.. _`plugins`: index.html
.. _`mark`: mark.html
.. _`tmpdir`: tmpdir.html

2
doc/en/test/plugin/terminal.rst
View File

@ -18,8 +18,6 @@ command line options
show extra test summary info as specified by chars (f)ailed, (s)skipped, (x)failed, (X)passed.
``-l, --showlocals``
show locals in tracebacks (disabled by default).
``--report=opts``
(deprecated, use -r)
``--tb=style``
traceback print mode (long/short/line/no).
``--full-trace``

32
setup.py
View File

@ -13,7 +13,7 @@ classifiers = ['Development Status :: 6 - Mature',
'Topic :: Software Development :: Libraries',
'Topic :: Utilities'] + [
('Programming Language :: Python :: %s' % x) for x in
'2 2.6 2.7 3 3.2 3.3 3.4 3.5'.split()]
'2 2.6 2.7 3 3.3 3.4 3.5'.split()]
with open('README.rst') as fd:
long_description = fd.read()
@ -51,10 +51,10 @@ def main():
install_requires = ['py>=1.4.29'] # pluggy is vendored in _pytest.vendored_packages
extras_require = {}
if has_environment_marker_support():
extras_require[':python_version=="2.6" or python_version=="3.0" or python_version=="3.1"'] = ['argparse']
extras_require[':python_version=="2.6"'] = ['argparse']
extras_require[':sys_platform=="win32"'] = ['colorama']
else:
if sys.version_info < (2, 7) or (3,) <= sys.version_info < (3, 2):
if sys.version_info < (2, 7):
install_requires.append('argparse')
if sys.platform == 'win32':
install_requires.append('colorama')
@ -69,7 +69,8 @@ def main():
platforms=['unix', 'linux', 'osx', 'cygwin', 'win32'],
author='Holger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and others',
author_email='holger at merlinux.eu',
entry_points=make_entry_points(),
entry_points={'console_scripts':
['pytest=pytest:main', 'py.test=pytest:main']},
classifiers=classifiers,
cmdclass={'test': PyTest},
# the following should be enabled for release
@ -81,29 +82,6 @@ def main():
)
def cmdline_entrypoints(versioninfo, platform, basename):
target = 'pytest:main'
if platform.startswith('java'):
points = {'py.test-jython': target}
else:
if basename.startswith('pypy'):
points = {'py.test-%s' % basename: target}
else: # cpython
points = {'py.test-%s.%s' % versioninfo[:2] : target}
points['py.test'] = target
points['pytest'] = target
return points
def make_entry_points():
basename = os.path.basename(sys.executable)
points = cmdline_entrypoints(sys.version_info, sys.platform, basename)
keys = list(points.keys())
keys.sort()
l = ['%s = %s' % (x, points[x]) for x in keys]
return {'console_scripts': l}
class PyTest(Command):
user_options = []
def initialize_options(self):

4
testing/code/test_source.py
View File

@ -385,8 +385,7 @@ def test_deindent():
lines = deindent(source.splitlines())
assert lines == ['', 'def f():', ' def g():', ' pass', ' ']
@pytest.mark.xfail("sys.version_info[:3] < (2,7,0) or "
"((3,0) <= sys.version_info[:2] < (3,2))")
@pytest.mark.xfail("sys.version_info[:3] < (2,7,0)")
def test_source_of_class_at_eof_without_newline(tmpdir):
# this test fails because the implicit inspect.getsource(A) below
# does not return the "x = 1" last line.
@ -656,4 +655,3 @@ something
'''"""
result = getstatement(1, source)
assert str(result) == "'''\n'''"

32
testing/test_assertion.py
View File

@ -474,16 +474,8 @@ def test_assertion_options(testdir):
""")
result = testdir.runpytest()
assert "3 == 4" in result.stdout.str()
off_options = (("--no-assert",),
("--nomagic",),
("--no-assert", "--nomagic"),
("--assert=plain",),
("--assert=plain", "--no-assert"),
("--assert=plain", "--nomagic"),
("--assert=plain", "--no-assert", "--nomagic"))
for opt in off_options:
result = testdir.runpytest_subprocess(*opt)
assert "3 == 4" not in result.stdout.str()
result = testdir.runpytest_subprocess("--assert=plain")
assert "3 == 4" not in result.stdout.str()
def test_old_assert_mode(testdir):
testdir.makepyfile("""
@ -559,7 +551,7 @@ def test_warn_missing(testdir):
result.stderr.fnmatch_lines([
"*WARNING*assert statements are not executed*",
])
result = testdir.run(sys.executable, "-OO", "-m", "pytest", "--no-assert")
result = testdir.run(sys.executable, "-OO", "-m", "pytest")
result.stderr.fnmatch_lines([
"*WARNING*assert statements are not executed*",
])
@ -640,3 +632,21 @@ def test_diff_newline_at_end(monkeypatch, testdir):
* + asdf
* ? +
""")
def test_assert_tuple_warning(testdir):
testdir.makepyfile("""
def test_tuple():
assert(False, 'you shall not pass')
""")
result = testdir.runpytest('-rw')
result.stdout.fnmatch_lines('WR1*:2 assertion is always true*')
def test_assert_indirect_tuple_no_warning(testdir):
testdir.makepyfile("""
def test_tuple():
tpl = ('foo', 'bar')
assert tpl
""")
result = testdir.runpytest('-rw')
output = '\n'.join(result.stdout.lines)
assert 'WR1' not in output

91
testing/test_config.py
View File

@ -365,7 +365,7 @@ def test_options_on_small_file_do_not_blow_up(testdir):
""")
for opts in ([], ['-l'], ['-s'], ['--tb=no'], ['--tb=short'],
['--tb=long'], ['--fulltrace'], ['--nomagic'],
['--tb=long'], ['--fulltrace'],
['--traceconfig'], ['-v'], ['-v', '-v']):
runfiletest(opts + [path])
@ -583,3 +583,92 @@ class TestRootdir:
inifile = tmpdir.ensure("pytest.ini")
rootdir, inifile, inicfg = determine_setup(inifile, [tmpdir])
assert rootdir == tmpdir
class TestOverrideIniArgs:
""" test --override-ini """
@pytest.mark.parametrize("name", "setup.cfg tox.ini pytest.ini".split())
def test_override_ini_names(self, testdir, name):
testdir.tmpdir.join(name).write(py.std.textwrap.dedent("""
[pytest]
custom = 1.0
"""))
testdir.makeconftest("""
def pytest_addoption(parser):
parser.addini("custom", "")
""")
testdir.makepyfile("""
def test_pass(pytestconfig):
ini_val = pytestconfig.getini("custom")
print('\\ncustom_option:%s\\n' % ini_val)
""")
result = testdir.runpytest("--override-ini", "custom=2.0", "-s")
assert result.ret == 0
result.stdout.fnmatch_lines([
"custom_option:2.0"
])
result = testdir.runpytest("--override-ini", "custom=2.0",
"--override-ini=custom=3.0", "-s")
assert result.ret == 0
result.stdout.fnmatch_lines([
"custom_option:3.0"
])
def test_override_ini_pathlist(self, testdir):
testdir.makeconftest("""
def pytest_addoption(parser):
parser.addini("paths", "my new ini value", type="pathlist")
""")
testdir.makeini("""
[pytest]
paths=blah.py
""")
testdir.makepyfile("""
import py.path
def test_pathlist(pytestconfig):
config_paths = pytestconfig.getini("paths")
print(config_paths)
for cpf in config_paths:
print('\\nuser_path:%s' % cpf.basename)
""")
result = testdir.runpytest("--override-ini", 'paths=foo/bar1.py foo/bar2.py', "-s")
result.stdout.fnmatch_lines([
"user_path:bar1.py",
"user_path:bar2.py"
])
def test_override_multiple_and_default(self, testdir):
testdir.makeconftest("""
def pytest_addoption(parser):
parser.addini("custom_option_1", "", default="o1")
parser.addini("custom_option_2", "", default="o2")
parser.addini("custom_option_3", "", default=False, type="bool")
parser.addini("custom_option_4", "", default=True, type="bool")
""")
testdir.makeini("""
[pytest]
custom_option_1=custom_option_1
custom_option_2=custom_option_2
""")
testdir.makepyfile("""
def test_multiple_options(pytestconfig):
prefix="custom_option"
for x in range(1,5):
ini_value=pytestconfig.getini("%s_%d" % (prefix, x))
print('\\nini%d:%s' % (x, ini_value))
""")
result = testdir.runpytest("--override-ini",
'custom_option_1=fulldir=/tmp/user1',
'custom_option_2=url=/tmp/user2?a=b&d=e',
"-o", 'custom_option_3=True',
"-o", 'custom_option_4=no',
"-s")
result.stdout.fnmatch_lines([
"ini1:fulldir=/tmp/user1",
"ini2:url=/tmp/user2?a=b&d=e",
"ini3:True",
"ini4:False"
])

51
testing/test_genscript.py
View File

@ -1,51 +0,0 @@
import pytest
import sys
@pytest.fixture(scope="module")
def standalone(request):
return Standalone(request)
class Standalone:
def __init__(self, request):
self.testdir = request.getfuncargvalue("testdir")
script = "mypytest"
result = self.testdir.runpytest("--genscript=%s" % script)
assert result.ret == 0
self.script = self.testdir.tmpdir.join(script)
assert self.script.check()
def run(self, anypython, testdir, *args):
return testdir._run(anypython, self.script, *args)
def test_gen(testdir, anypython, standalone):
if sys.version_info >= (2,7):
result = testdir._run(anypython, "-c",
"import sys;print (sys.version_info >=(2,7))")
assert result.ret == 0
if result.stdout.str() == "False":
pytest.skip("genscript called from python2.7 cannot work "
"earlier python versions")
result = standalone.run(anypython, testdir, '--version')
if result.ret == 2:
result.stderr.fnmatch_lines(["*ERROR: setuptools not installed*"])
elif result.ret == 0:
result.stderr.fnmatch_lines([
"*imported from*mypytest*"
])
p = testdir.makepyfile("def test_func(): assert 0")
result = standalone.run(anypython, testdir, p)
assert result.ret != 0
else:
pytest.fail("Unexpected return code")
def test_freeze_includes():
"""
Smoke test for freeze_includes(), to ensure that it works across all
supported python versions.
"""
includes = pytest.freeze_includes()
assert len(includes) > 1
assert '_pytest.genscript' in includes

59
testing/test_junitxml.py
View File

@ -1,7 +1,6 @@
# -*- coding: utf-8 -*-
from xml.dom import minidom
from _pytest.main import EXIT_NOTESTSCOLLECTED
import py
import sys
import os
@ -158,6 +157,47 @@ class TestPython:
snode = tnode.find_first_by_tag("skipped")
snode.assert_attr(type="pytest.skip", message="hello23", )
def test_mark_skip_contains_name_reason(self, testdir):
testdir.makepyfile("""
import pytest
@pytest.mark.skip(reason="hello24")
def test_skip():
assert True
""")
result, dom = runandparse(testdir)
assert result.ret == 0
node = dom.find_first_by_tag("testsuite")
node.assert_attr(skips=1)
tnode = node.find_first_by_tag("testcase")
tnode.assert_attr(
file="test_mark_skip_contains_name_reason.py",
line="1",
classname="test_mark_skip_contains_name_reason",
name="test_skip")
snode = tnode.find_first_by_tag("skipped")
snode.assert_attr(type="pytest.skip", message="hello24", )
def test_mark_skipif_contains_name_reason(self, testdir):
testdir.makepyfile("""
import pytest
GLOBAL_CONDITION = True
@pytest.mark.skipif(GLOBAL_CONDITION, reason="hello25")
def test_skip():
assert True
""")
result, dom = runandparse(testdir)
assert result.ret == 0
node = dom.find_first_by_tag("testsuite")
node.assert_attr(skips=1)
tnode = node.find_first_by_tag("testcase")
tnode.assert_attr(
file="test_mark_skipif_contains_name_reason.py",
line="2",
classname="test_mark_skipif_contains_name_reason",
name="test_skip")
snode = tnode.find_first_by_tag("skipped")
snode.assert_attr(type="pytest.skip", message="hello25", )
def test_classname_instance(self, testdir):
testdir.makepyfile("""
class TestClass:
@ -351,23 +391,6 @@ class TestPython:
fnode.assert_attr(message="collection failure")
assert "SyntaxError" in fnode.toxml()
def test_collect_skipped(self, testdir):
testdir.makepyfile("import pytest; pytest.skip('xyz')")
result, dom = runandparse(testdir)
assert result.ret == EXIT_NOTESTSCOLLECTED
node = dom.find_first_by_tag("testsuite")
node.assert_attr(skips=1, tests=1)
tnode = node.find_first_by_tag("testcase")
tnode.assert_attr(
file="test_collect_skipped.py",
name="test_collect_skipped")
# pytest doesn't give us a line here.
assert tnode["line"] is None
fnode = tnode.find_first_by_tag("skipped")
fnode.assert_attr(message="collection skipped")
def test_unicode(self, testdir):
value = 'hx\xc4\x85\xc4\x87\n'
testdir.makepyfile("""

26
testing/test_mark.py
View File

@ -418,6 +418,32 @@ class TestFunctional:
items, rec = testdir.inline_genitems(p)
self.assert_markers(items, test_foo=('a', 'b'), test_bar=('a',))
@pytest.mark.issue568
@pytest.mark.xfail(reason="markers smear on methods of base classes")
def test_mark_should_not_pass_to_siebling_class(self, testdir):
p = testdir.makepyfile("""
import pytest
class TestBase:
def test_foo(self):
pass
@pytest.mark.b
class TestSub(TestBase):
pass
class TestOtherSub(TestBase):
pass
""")
items, rec = testdir.inline_genitems(p)
base_item, sub_item, sub_item_other = items
assert not hasattr(base_item.obj, 'b')
assert not hasattr(sub_item_other.obj, 'b')
def test_mark_decorator_baseclasses_merged(self, testdir):
p = testdir.makepyfile("""
import pytest

7
testing/test_parseopt.py
View File

@ -77,6 +77,13 @@ class TestParser:
assert len(group.options) == 1
assert isinstance(group.options[0], parseopt.Argument)
def test_group_addoption_conflict(self):
group = parseopt.OptionGroup("hello again")
group.addoption("--option1", "--option-1", action="store_true")
with pytest.raises(ValueError) as err:
group.addoption("--option1", "--option-one", action="store_true")
assert str(set(["--option1"])) in str(err.value)
def test_group_shortopt_lowercase(self, parser):
group = parser.getgroup("hello")
pytest.raises(ValueError, """

9
testing/test_resultlog.py
View File

@ -83,19 +83,10 @@ class TestWithFunctionIntegration:
def test_collection_report(self, testdir):
ok = testdir.makepyfile(test_collection_ok="")
skip = testdir.makepyfile(test_collection_skip=
"import pytest ; pytest.skip('hello')")
fail = testdir.makepyfile(test_collection_fail="XXX")
lines = self.getresultlog(testdir, ok)
assert not lines
lines = self.getresultlog(testdir, skip)
assert len(lines) == 2
assert lines[0].startswith("S ")
assert lines[0].endswith("test_collection_skip.py")
assert lines[1].startswith(" ")
assert lines[1].endswith("test_collection_skip.py:1: Skipped: hello")
lines = self.getresultlog(testdir, fail)
assert lines
assert lines[0].startswith("F ")

12
testing/test_runner.py
View File

@ -365,18 +365,6 @@ class TestSessionReports:
assert res[0].name == "test_func1"
assert res[1].name == "TestClass"
def test_skip_at_module_scope(self, testdir):
col = testdir.getmodulecol("""
import pytest
pytest.skip("hello")
def test_func():
pass
""")
rep = main.collect_one_node(col)
assert not rep.failed
assert not rep.passed
assert rep.skipped
reporttypes = [
runner.BaseReport,

8
testing/test_session.py
View File

@ -174,10 +174,6 @@ class TestNewSession(SessionTests):
class TestY(TestX):
pass
""",
test_two="""
import pytest
pytest.skip('xxx')
""",
test_three="xxxdsadsadsadsa",
__init__=""
)
@ -189,11 +185,9 @@ class TestNewSession(SessionTests):
started = reprec.getcalls("pytest_collectstart")
finished = reprec.getreports("pytest_collectreport")
assert len(started) == len(finished)
assert len(started) == 8 # XXX extra TopCollector
assert len(started) == 7 # XXX extra TopCollector
colfail = [x for x in finished if x.failed]
colskipped = [x for x in finished if x.skipped]
assert len(colfail) == 1
assert len(colskipped) == 1
def test_minus_x_import_error(self, testdir):
testdir.makepyfile(__init__="")

28
testing/test_skipping.py
View File

@ -209,7 +209,7 @@ class TestXFail:
def test_this_false():
assert 1
""")
result = testdir.runpytest(p, '--report=xfailed', )
result = testdir.runpytest(p, '-rx', )
result.stdout.fnmatch_lines([
"*test_one*test_this*",
"*NOTRUN*noway",
@ -227,7 +227,7 @@ class TestXFail:
def setup_module(mod):
raise ValueError(42)
""")
result = testdir.runpytest(p, '--report=xfailed', )
result = testdir.runpytest(p, '-rx', )
result.stdout.fnmatch_lines([
"*test_one*test_this*",
"*NOTRUN*hello",
@ -682,19 +682,15 @@ def test_skipped_reasons_functional(testdir):
def test_method(self):
doskip()
""",
test_two = """
from conftest import doskip
doskip()
""",
conftest = """
import pytest
def doskip():
pytest.skip('test')
"""
)
result = testdir.runpytest('--report=skipped')
result = testdir.runpytest('-rs')
result.stdout.fnmatch_lines([
"*SKIP*3*conftest.py:3: test",
"*SKIP*2*conftest.py:3: test",
])
assert result.ret == 0
@ -941,3 +937,19 @@ def test_xfail_item(testdir):
assert not failed
xfailed = [r for r in skipped if hasattr(r, 'wasxfail')]
assert xfailed
def test_module_level_skip_error(testdir):
"""
Verify that using pytest.skip at module level causes a collection error
"""
testdir.makepyfile("""
import pytest
@pytest.skip
def test_func():
assert True
""")
result = testdir.runpytest()
result.stdout.fnmatch_lines(
"*Using @pytest.skip outside a test * is not allowed*"
)

12
testing/test_terminal.py
View File

@ -223,8 +223,7 @@ class TestCollectonly:
""")
result = testdir.runpytest("--collect-only", "-rs")
result.stdout.fnmatch_lines([
"SKIP*hello*",
"*1 skip*",
"*ERROR collecting*",
])
def test_collectonly_failed_module(self, testdir):
@ -591,16 +590,7 @@ def test_getreportopt():
class config:
class option:
reportchars = ""
config.option.report = "xfailed"
assert getreportopt(config) == "x"
config.option.report = "xfailed,skipped"
assert getreportopt(config) == "xs"
config.option.report = "skipped,xfailed"
assert getreportopt(config) == "sx"
config.option.report = "skipped"
config.option.reportchars = "sf"
assert getreportopt(config) == "sf"