test_ok1/ISSUES.txt

196 lines
7.2 KiB
Plaintext
Raw Normal View History

refine session initialization / fix custom collect crash
---------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: bug 1.4 core xdist
When calling "py.test path/X" py.test can crash if the collection
of that directory is skipped. Calling "py.test path" will give
proper output. The reason is that for the very first colitems
getinitialnodes() and a collection is done before the fully
controlled session and pytest_make_collect_report protocol takes over.
Try to remove the redundant getinitialnodes related logic and amend
the session collect logic to care for this "initial" case as well.
Apart from simplification a side effect the dsession's session
and the core session probably converge some more.
2009-12-21 05:11:36 +08:00
introduce py.test.mark.nocollect
-------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: feature 1.4
for not considering a function for test collection at all.
maybe also introduce a py.test.mark.test to explicitely
mark a function to become a tested one. Lookup JUnit
ways of tagging tests.
2009-12-21 05:11:36 +08:00
introduce py.test.mark.platform
-------------------------------------------------------
tags: feature 1.4
Introduce nice-to-spell platform-skipping, examples:
@py.test.mark.platform("python3")
@py.test.mark.platform("not python3")
@py.test.mark.platform("win32 and not python3")
@py.test.mark.platform("darwin")
@py.test.mark.platform("not (jython and win32)")
@py.test.mark.platform("not (jython and win32)", xfail=True)
etc. Idea is to allow Python expressions which can operate
on common spellings for operating systems and python
interpreter versions.
2010-02-07 09:55:50 +08:00
introduce py.test.mark registration
-----------------------------------------
tags: feature 1.3
introduce a hook that allows to register a named mark decorator
with documentation and add "py.test --marks" to get
a list of available marks. Deprecate "dynamic" mark
definitions.
do early-teardown of test modules
-----------------------------------------
tags: feature 1.3
currently teardowns are called when the next tests is setup
except for the function/method level where interally
"teardown_exact" tears down immediately. Generalize
this to perform the "neccessary" teardown compared to
the "next" test item during teardown - this should
get rid of some irritations because otherwise e.g.
prints of teardown-code appear in the setup of the next test.
do recursive walk of conftest.py files?
-----------------------------------------
tags: feature 1.3
it maybe makes sense to generally do a recursive search of conftest.py
files before command line parsing - this would help to offer the
full list of options as applicable to a given test project.
2010-02-07 09:55:50 +08:00
consider introducing py.test.mark.skip_[not]win32/jython/pyXY
-------------------------------------------------------------
tags: feature 1.3
conveniently introduce markers for platforms to
have a shorter form for skipping.
generalize parametrized testing to generate combinations
-------------------------------------------------------------
tags: feature 1.3
think about extending metafunc.addcall or add a new method to allow to
generate tests with combinations of all generated versions - what to do
about "id" and "param" in such combinations though?
introduce py.test.mark.multi
-----------------------------------------
tags: feature 1.3
introduce py.test.mark.multi to specify a number
of values for a given function argument.
2009-12-21 05:11:36 +08:00
have imported module mismatch honour relative paths
--------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: bug 1.4
2009-12-21 05:11:36 +08:00
With 1.1.1 py.test fails at least on windows if an import
is relative and compared against an absolute conftest.py
path. Normalize.
make node._checkcollectable more robust
-------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: bug 1.4
currently node._checkcollectable() can raise
exceptions for all kinds of reasons ('conftest.py' loading
problems, missing rsync-dirs, platform-skip-at-import-level
issues, ...). It should just return True/False and cause
a good error message.
call termination with small timeout
-------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: feature 1.4
test: testing/pytest/dist/test_dsession.py - test_terminate_on_hanging_node
Call gateway group termination with a small timeout if available.
Should make dist-testing less likely to leave lost processes.
2009-12-07 20:27:24 +08:00
consider globals: py.test.ensuretemp and config
--------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: experimental-wish 1.4
consider deprecating py.test.ensuretemp and py.test.config
to further reduce py.test globality. Also consider
having py.test.config and ensuretemp coming from
a plugin rather than being there from the start.
consider allowing funcargs to setup methods
--------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: experimental-wish 1.4
Users have expressed the wish to have funcargs available to setup
functions. Experiment with allowing funcargs there - it might
also help to make the py.test.ensuretemp and config deprecation.
For filling funcargs for setup methods, we could call funcarg
factories with a request object that not have a cls/function
attributes. However, how to handle parametrized test functions
and funcargs?
consider pytest_addsyspath hook
-----------------------------------------
2010-05-09 14:43:28 +08:00
tags: 1.4
py.test could call a new pytest_addsyspath() in order to systematically
allow manipulation of sys.path and to inhibit it via --no-addsyspath
in order to more easily run against installed packages.
Alternatively it could also be done via the config object
and pytest_configure.
relax requirement to have tests/testing contain an __init__
----------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: feature 1.4
bb: http://bitbucket.org/hpk42/py-trunk/issue/64
A local test run of a "tests" directory may work
but a remote one fail because the tests directory
does not contain an "__init__.py". Either give
an error or make it work without the __init__.py
show plugin information in test header
----------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: feature 1.4
Now that external plugins are becoming more numerous
it would be useful to have external plugins along with
their versions displayed as a header line.
generate/refine plugin doc generation
----------------------------------------------------------------
2010-05-09 14:43:28 +08:00
tags: feature 1.4
2010-05-09 14:43:28 +08:00
review and prepare docs for 1.4.0 release. Probably
have docs living with the plugin and require them to
be available on doc generation time, at least when
the target is the website? Or rather go for interactive help?
deprecate global py.test.config usage
----------------------------------------------------------------
tags: feature 1.4
py.test.ensuretemp and py.test.config are probably the last
objects containing global state. Often using them is not
neccessary. This is about trying to get rid of them, i.e.
deprecating them and checking with PyPy's usages as well
as others.
remove deprecated bits in collect.py
-------------------------------------------------------------------
tags: feature 1.4
In an effort to further simplify code, review and remove deprecated bits
in collect.py. Probably good:
- inline consider_file/dir methods, no need to have them
subclass-overridable because of hooks