add new "parametrize refinement" issue critical for 2.2 series
also cleanup issues a bit and make a note on xUnit/funcargs collaboration
This commit is contained in:
holger krekel
parent
d15ee2fb87
commit
d965101f6a
94
ISSUES.txt
94
ISSUES.txt
|
|
@ -1,3 +1,24 @@
|
|||
refine parametrize API in 2.2 series
|
||||
-------------------------------------------------------------
|
||||
tags: critical feature 2.2
|
||||
|
||||
extend metafunc.parametrize to better support indirection
|
||||
by specifying a setupfunc(request, val) which will _substitute_
|
||||
the funcarg factory. Here is an example:
|
||||
|
||||
def setupdb(request, val):
|
||||
# setup "resource" based on test request and the values passed
|
||||
# in to parametrize. setupfunc is called for each such value.
|
||||
# you may use request.addfinalizer() or request.cached_setup ...
|
||||
return db
|
||||
|
||||
@pytest.mark.parametrize("db", ["pg", "mysql"], setupfunc=setupdb)
|
||||
def test_heavy_functional_test(db):
|
||||
...
|
||||
|
||||
There would be no need to write funcarg factories for this example, only
|
||||
to explain the attributes and functionality of "request".
|
||||
|
||||
checks / deprecations for next release
|
||||
---------------------------------------------------------------
|
||||
tags: bug 2.4 core xdist
|
||||
|
|
@ -27,7 +48,7 @@ teardown.
|
|||
|
||||
do early-teardown of test modules
|
||||
-----------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
currently teardowns are called when the next tests is setup
|
||||
except for the function/method level where interally
|
||||
|
|
@ -39,7 +60,7 @@ prints of teardown-code appear in the setup of the next test.
|
|||
|
||||
consider and document __init__ file usage in test directories
|
||||
---------------------------------------------------------------
|
||||
tags: bug 2.2 core
|
||||
tags: bug 2.3 core
|
||||
|
||||
Currently, a test module is imported with its fully qualified
|
||||
package path, determined by checking __init__ files upwards.
|
||||
|
|
@ -54,7 +75,7 @@ certain scenarios makes sense.
|
|||
|
||||
relax requirement to have tests/testing contain an __init__
|
||||
----------------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
bb: http://bitbucket.org/hpk42/py-trunk/issue/64
|
||||
|
||||
A local test run of a "tests" directory may work
|
||||
|
|
@ -65,7 +86,7 @@ i.e. port the nose-logic of unloading a test module.
|
|||
|
||||
customize test function collection
|
||||
-------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
- introduce py.test.mark.nocollect for not considering a function for
|
||||
test collection at all. maybe also introduce a py.test.mark.test to
|
||||
|
|
@ -74,7 +95,7 @@ tags: feature 2.2
|
|||
|
||||
introduce pytest.mark.importorskip
|
||||
-------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
in addition to the imperative pytest.importorskip also introduce
|
||||
a pytest.mark.importorskip so that the test count is more correct.
|
||||
|
|
@ -82,7 +103,7 @@ a pytest.mark.importorskip so that the test count is more correct.
|
|||
|
||||
introduce py.test.mark.platform
|
||||
-------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
Introduce nice-to-spell platform-skipping, examples:
|
||||
|
||||
|
|
@ -99,7 +120,7 @@ interpreter versions.
|
|||
|
||||
pytest.mark.xfail signature change
|
||||
-------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
change to pytest.mark.xfail(reason, (optional)condition)
|
||||
to better implement the word meaning. It also signals
|
||||
|
|
@ -109,7 +130,7 @@ Compatibility? how to introduce a new name/keep compat?
|
|||
|
||||
introduce py.test.mark registration
|
||||
-----------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
introduce a hook that allows to register a named mark decorator
|
||||
with documentation and add "py.test --marks" to get
|
||||
|
|
@ -118,7 +139,7 @@ definitions.
|
|||
|
||||
allow to non-intrusively apply skipfs/xfail/marks
|
||||
---------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
use case: mark a module or directory structures
|
||||
to be skipped on certain platforms (i.e. no import
|
||||
|
|
@ -129,14 +150,14 @@ from conftests or plugins.
|
|||
|
||||
explicit referencing of conftest.py files
|
||||
-----------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
allow to name conftest.py files (in sub directories) that should
|
||||
be imported early, as to include command line options.
|
||||
|
||||
improve central py.test ini file
|
||||
----------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
introduce more declarative configuration options:
|
||||
- (to-be-collected test directories)
|
||||
|
|
@ -147,33 +168,16 @@ introduce more declarative configuration options:
|
|||
|
||||
new documentation
|
||||
----------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
- logo py.test
|
||||
- examples for unittest or functional testing
|
||||
- resource management for functional testing
|
||||
- patterns: page object
|
||||
- parametrized testing
|
||||
- better / more integrated plugin docs
|
||||
|
||||
generalize parametrized testing to generate combinations
|
||||
-------------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
|
||||
think about extending metafunc.addcall or add a new method to allow to
|
||||
generate tests with combinations of all generated versions - what to do
|
||||
about "id" and "param" in such combinations though?
|
||||
|
||||
introduce py.test.mark.multi
|
||||
-----------------------------------------
|
||||
tags: feature 1.3
|
||||
|
||||
introduce py.test.mark.multi to specify a number
|
||||
of values for a given function argument.
|
||||
|
||||
have imported module mismatch honour relative paths
|
||||
--------------------------------------------------------
|
||||
tags: bug 2.2
|
||||
tags: bug 2.3
|
||||
|
||||
With 1.1.1 py.test fails at least on windows if an import
|
||||
is relative and compared against an absolute conftest.py
|
||||
|
|
@ -181,7 +185,7 @@ path. Normalize.
|
|||
|
||||
call termination with small timeout
|
||||
-------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
test: testing/pytest/dist/test_dsession.py - test_terminate_on_hanging_node
|
||||
|
||||
Call gateway group termination with a small timeout if available.
|
||||
|
|
@ -189,7 +193,7 @@ Should make dist-testing less likely to leave lost processes.
|
|||
|
||||
consider globals: py.test.ensuretemp and config
|
||||
--------------------------------------------------------------
|
||||
tags: experimental-wish 2.2
|
||||
tags: experimental-wish 2.3
|
||||
|
||||
consider deprecating py.test.ensuretemp and py.test.config
|
||||
to further reduce py.test globality. Also consider
|
||||
|
|
@ -198,7 +202,7 @@ a plugin rather than being there from the start.
|
|||
|
||||
consider allowing funcargs for setup methods
|
||||
--------------------------------------------------------------
|
||||
tags: experimental-wish 2.2
|
||||
tags: experimental-wish 2.3
|
||||
|
||||
Users have expressed the wish to have funcargs available to setup
|
||||
functions. Experiment with allowing funcargs there - it might
|
||||
|
|
@ -208,13 +212,20 @@ factories with a request object that not have a cls/function
|
|||
attributes. However, how to handle parametrized test functions
|
||||
and funcargs?
|
||||
|
||||
setup_function -> request can be like it is now
|
||||
setup_class -> request has no request.function
|
||||
setup_module -> request has no request.cls
|
||||
maybe introduce a setup method like:
|
||||
|
||||
setup_invocation(self, request)
|
||||
|
||||
which has full access to the test invocation through "request"
|
||||
through which you can get funcargvalues, use cached_setup etc.
|
||||
Therefore, the access to funcargs would be indirect but it
|
||||
could be consistently implemented. setup_invocation() would
|
||||
be a "glue" function for bringing together the xUnit and funcargs
|
||||
world.
|
||||
|
||||
consider pytest_addsyspath hook
|
||||
-----------------------------------------
|
||||
tags: 2.2
|
||||
tags: 2.3
|
||||
|
||||
py.test could call a new pytest_addsyspath() in order to systematically
|
||||
allow manipulation of sys.path and to inhibit it via --no-addsyspath
|
||||
|
|
@ -226,7 +237,7 @@ and pytest_configure.
|
|||
|
||||
show plugin information in test header
|
||||
----------------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
Now that external plugins are becoming more numerous
|
||||
it would be useful to have external plugins along with
|
||||
|
|
@ -234,7 +245,7 @@ their versions displayed as a header line.
|
|||
|
||||
deprecate global py.test.config usage
|
||||
----------------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
py.test.ensuretemp and py.test.config are probably the last
|
||||
objects containing global state. Often using them is not
|
||||
|
|
@ -244,7 +255,7 @@ as others.
|
|||
|
||||
remove deprecated bits in collect.py
|
||||
-------------------------------------------------------------------
|
||||
tags: feature 2.2
|
||||
tags: feature 2.3
|
||||
|
||||
In an effort to further simplify code, review and remove deprecated bits
|
||||
in collect.py. Probably good:
|
||||
|
|
@ -253,7 +264,7 @@ in collect.py. Probably good:
|
|||
|
||||
implement fslayout decorator
|
||||
---------------------------------
|
||||
tags: feature
|
||||
tags: feature 2.3
|
||||
|
||||
Improve the way how tests can work with pre-made examples,
|
||||
keeping the layout close to the test function:
|
||||
|
|
@ -272,3 +283,4 @@ def test_run(pytester, fslayout):
|
|||
assert result.ret == 0
|
||||
assert result.passed == 1
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue