update todo to make some sense again
shift samuele in license file --HG-- branch : trunk
This commit is contained in:
parent
a58292b473
commit
d4a28b1b2d
222
TODO.txt
222
TODO.txt
|
@ -4,111 +4,39 @@ Things to do for 1.0.0
|
||||||
py.test
|
py.test
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
- compatilibity: honour/warn item.run() method of test items (so far
|
- clarify setup/run events and runner.py versus pytest_runner.py,
|
||||||
probably only execute() is warned about or the other way round)
|
introduce a general pytest_item_setup(item, setupstate)
|
||||||
|
and always isolate py._com.comregistry when py lib's own tests are run
|
||||||
|
|
||||||
- introduce plugin arch, port existing things to plugins:
|
- hook review and hook docs
|
||||||
- importorskip
|
|
||||||
- filelog
|
|
||||||
- chtmpdir per method
|
|
||||||
- apigen
|
|
||||||
- xfail
|
|
||||||
- acceptance/assertlines
|
|
||||||
- dist-testing?
|
|
||||||
|
|
||||||
- introduce setuptools-style version checking, at least
|
|
||||||
for py lib itself, maybe also for other packages:
|
|
||||||
|
|
||||||
py.checkversion("py>=1.0")
|
|
||||||
|
|
||||||
- generative tests: it is somewhat misleading for
|
|
||||||
the classical htmlviews that generated tests are identified
|
|
||||||
by numbers rather than by its parameters. unclear how
|
|
||||||
to fix this because we may not always be able to assume
|
|
||||||
that we can identify a generated tests by using its parameters
|
|
||||||
(it might not be hashable, doesn't have a sensical repr ...?)
|
|
||||||
|
|
||||||
- turn deprecation / apiwarnings into events, report them at the end?
|
- turn deprecation / apiwarnings into events, report them at the end?
|
||||||
|
|
||||||
- get APIGEN back to work
|
|
||||||
|
|
||||||
- get web reporter back to work
|
|
||||||
|
|
||||||
- introduce decorator "shouldfail" or "xfail"
|
|
||||||
as to mark a test as "expected to fail",
|
|
||||||
report specially if it surprisingly passes
|
|
||||||
|
|
||||||
- nightly test runs on multiple platforms
|
- nightly test runs on multiple platforms
|
||||||
|
|
||||||
- review and refactor architecture of py.test with particular
|
|
||||||
respect to:
|
|
||||||
- writing (stacked) extensions / plugins (compared to Nose)
|
|
||||||
- porting existing extensions (htmlconftest / buildbot / PyPy's conftest's ...)
|
|
||||||
- fast and stable distributed testing
|
|
||||||
- reliable cross-platform testing
|
|
||||||
|
|
||||||
- improve py.test documentation to reflect new
|
|
||||||
event architecture
|
|
||||||
|
|
||||||
- review and optimize skip-handling (it can be quite slow in
|
|
||||||
certain situations because e.g. setup/teardown is fully performed
|
|
||||||
although we have "skip by keyword" and could detect this early)
|
|
||||||
|
|
||||||
py.execnet
|
py.execnet
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
- cross-python version (2.2/2.3-2.5/6) and cross-platform testing of
|
- cross-python version (2.2/2.3-2.5/6) and cross-platform testing of
|
||||||
setup/teardown semantics
|
setup/teardown semantics
|
||||||
|
|
||||||
- optimize general setup and rsync timing?
|
py.test apigen plugin
|
||||||
|
---------------------------
|
||||||
|
|
||||||
py.apigen
|
- make it work again with the new plugin arch
|
||||||
----------------
|
|
||||||
|
|
||||||
- make it work again
|
packaging / svn-mercurial interaction
|
||||||
|
--------------------------------------------
|
||||||
|
|
||||||
see apigen_refactorings.txt
|
- decide if to go with or without setuptools, check windows 2.6
|
||||||
|
availability
|
||||||
|
|
||||||
- check out CodeInvestigator
|
- open a mercurial branch for releases?
|
||||||
http://codeinvestigator.googlepages.com/main
|
|
||||||
|
|
||||||
or other code that collects data from running a program
|
- write a script to dumb-bridge the mercurial repo to svn
|
||||||
(in our case running the tests)
|
(i.e. forget about svn history)
|
||||||
|
|
||||||
Criticism and solutions
|
1.1 and beyond
|
||||||
--------------------------------
|
|
||||||
"too big":
|
|
||||||
- too much code, you need entire py lib, hard to include into app
|
|
||||||
+ have a small pytest boostrap that loads pylib.zip from net
|
|
||||||
+ provide smaller script ala simpy
|
|
||||||
- lots of cmdline options, possibilities, documentation
|
|
||||||
rather unsorted
|
|
||||||
|
|
||||||
"needless differences between py.test and nosetests":
|
|
||||||
- py.test.skip
|
|
||||||
- raises
|
|
||||||
|
|
||||||
"tutorial structure missing", e.g.:
|
|
||||||
- "how to get started" in a minimal way, also how to use
|
|
||||||
existing conftests/plugins
|
|
||||||
- how to configure py.test
|
|
||||||
- how to write plugins/extensions
|
|
||||||
|
|
||||||
"too much magic"
|
|
||||||
- re-execution of assert expressions
|
|
||||||
+ rename "--nomagic" to something that turns off "superassertions"
|
|
||||||
+ hint at --tb=...
|
|
||||||
- get rid of py/magic directory
|
|
||||||
|
|
||||||
has a good ui but could be better
|
|
||||||
- support developer communication, e.g. py.test
|
|
||||||
--sendfailures=freenode-pypy
|
|
||||||
--sendfailures=pocoo # prints out paste.pocoo.url with traceback
|
|
||||||
- graphical interface, probably QT
|
|
||||||
- generally store test results and use them for subsequent calls
|
|
||||||
|
|
||||||
|
|
||||||
ld (review and shift to above)
|
|
||||||
=================================
|
=================================
|
||||||
|
|
||||||
refactorings
|
refactorings
|
||||||
|
@ -119,16 +47,6 @@ refactorings
|
||||||
- check if it works on win32
|
- check if it works on win32
|
||||||
- refine error reporting (don't show python tracebacks)
|
- refine error reporting (don't show python tracebacks)
|
||||||
|
|
||||||
- generalization of "host specifications" for execnet and
|
|
||||||
py.test --dist usages in particular (see also revision 37500 which
|
|
||||||
contained a draft for that). The goal is to have cross-platform
|
|
||||||
testing and dist-testing and other usages of py.execnet all
|
|
||||||
use a common syntax for specifiying connection methods and
|
|
||||||
be able to instantiate gateways/connections through it.
|
|
||||||
|
|
||||||
- unification of "gateway"/host setup and teardown, including
|
|
||||||
rsyncing, i.e. cross-platform and dist-testing.
|
|
||||||
|
|
||||||
- py.log: unify API, possibly deprecate duplicate ones,
|
- py.log: unify API, possibly deprecate duplicate ones,
|
||||||
base things on a Config object (hte latter almost a feature though)
|
base things on a Config object (hte latter almost a feature though)
|
||||||
(M988)
|
(M988)
|
||||||
|
@ -139,7 +57,9 @@ features
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
- (Harald Armin Massa): make py2exe work with py lib
|
- (Harald Armin Massa): make py2exe work with py lib
|
||||||
|
|
||||||
- optimize file checking with --looponfailing (harald has code for win32)
|
- optimize file checking with --looponfailing (harald has code for win32)
|
||||||
|
|
||||||
- have a py.test scan/run database for results and test names
|
- have a py.test scan/run database for results and test names
|
||||||
etc. (to allow quicker selection of tests and post-run
|
etc. (to allow quicker selection of tests and post-run
|
||||||
information on failures etc.) (M760)
|
information on failures etc.) (M760)
|
||||||
|
@ -168,116 +88,8 @@ features
|
||||||
- references from ReST docs to modules, functions and classes
|
- references from ReST docs to modules, functions and classes
|
||||||
of apigen generated html docs (M960)
|
of apigen generated html docs (M960)
|
||||||
|
|
||||||
- review svn-testing (and escape characters), consider
|
|
||||||
svn-bindings (M634)
|
|
||||||
|
|
||||||
- py.test.pdb - there is my hack for a while now, which integrates
|
- py.test.pdb - there is my hack for a while now, which integrates
|
||||||
rlcompleter2 with pdb. First of all it requires some strange changes
|
rlcompleter2 with pdb. First of all it requires some strange changes
|
||||||
to rlcompleter itself, which has no tests. Long-term plan would be
|
to rlcompleter itself, which has no tests. Long-term plan would be
|
||||||
to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and
|
to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and
|
||||||
have it tested. This requires work though.
|
have it tested. This requires work though.
|
||||||
|
|
||||||
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
--- below neeeds more review ---
|
|
||||||
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
|
|
||||||
|
|
||||||
More random notes, goals
|
|
||||||
--------------------------------
|
|
||||||
- REDUCE "MAGICALNESS", from an IRC discussion with ronny:
|
|
||||||
- integration into IDEs
|
|
||||||
- python2.6/python3 compat?
|
|
||||||
- initpkg exports
|
|
||||||
- assert reinterpretation
|
|
||||||
- greenlet compiles at runtime only in "dev-mode"
|
|
||||||
- conftest's are "scary", hum, what about
|
|
||||||
|
|
||||||
nicefications:
|
|
||||||
looponfailing shoudl nicely signal tests that failed but now PASS tests
|
|
||||||
rename Node to itemtestloop
|
|
||||||
refactor config and session tests to go into test_distsession.py
|
|
||||||
raises DID NOT RAISE: report the return value
|
|
||||||
have node.shutdown perform out-of-band so that shutdowns happens more quickly
|
|
||||||
tracebacks of importerrors of test modules should start with the test module file
|
|
||||||
|
|
||||||
- fix hostmanage to care for setting PYTHONPATH properly
|
|
||||||
|
|
||||||
- BRANCH:
|
|
||||||
adding of options
|
|
||||||
|
|
||||||
- BRANCH:
|
|
||||||
TEMPDIR handling
|
|
||||||
syspath handling (notify on changes, restore for each test?)
|
|
||||||
|
|
||||||
- COMPLETE REPORTING FOR MERGE!
|
|
||||||
- remove ItemStart and CollectionStart
|
|
||||||
which are only needed for collectonly.
|
|
||||||
implement it some other way.
|
|
||||||
|
|
||||||
- pre-counting of test items
|
|
||||||
|
|
||||||
- move assert reinterp back to session?
|
|
||||||
|
|
||||||
- merge "--tb" and "--fulltrace" option, --tb=full
|
|
||||||
- implement --showouterr, don't show outerr by default
|
|
||||||
|
|
||||||
- domainpath?
|
|
||||||
|
|
||||||
- translate remote filenames to local filenames so that,
|
|
||||||
probably based on option
|
|
||||||
|
|
||||||
- review safe_repr
|
|
||||||
|
|
||||||
- move OutcomeRepr.where/exconly attr to ReprExceptionInfo
|
|
||||||
or substitute as LocationRepr?
|
|
||||||
|
|
||||||
- reprcrash rename message to exconly
|
|
||||||
|
|
||||||
- test terminal reporter
|
|
||||||
ACCEPTANCE test for eventlog writing
|
|
||||||
ACCEPTANCE test for acceptance custom reporting :)
|
|
||||||
|
|
||||||
ACCEPTANCE showing nice Collection Errors
|
|
||||||
|
|
||||||
ACCEPTANCE test for "py.test2" exit signals
|
|
||||||
ACCEPTANCE test for "py.test2 --traceconfig"
|
|
||||||
ACCEPTANCE test for nice reprsentation of failures during Collection
|
|
||||||
ACCEPTANCE test for nice reprsentation of failures during Generator Collection
|
|
||||||
ACCEPTANCE test for "py.test2" conftest containing syntax errors
|
|
||||||
|
|
||||||
ACCEPTANCE test for "py.test2" honouring conftest specifying "extrainfo"
|
|
||||||
ACCEPTANCE test for "py.test2" on a simple example project
|
|
||||||
|
|
||||||
- merge CollectionFinish and ItemTestReport
|
|
||||||
maybe with base class: BaseReport and some common attrs/methods
|
|
||||||
|
|
||||||
- merge terminal/remote and dist-testing
|
|
||||||
and make allocation of tests to hosts more dynamic
|
|
||||||
|
|
||||||
- time setup/teardown and the actual test runs separately
|
|
||||||
ACCEPTANCE:
|
|
||||||
py.test2 --
|
|
||||||
py.test2 --repeat=10
|
|
||||||
|
|
||||||
- expect failing tests
|
|
||||||
py.test2.expectfail(feature=138)
|
|
||||||
|
|
||||||
|
|
||||||
while killing a process:
|
|
||||||
Exception in thread receiver:
|
|
||||||
Traceback (most recent call last):
|
|
||||||
File "threading.py", line 460, in __bootstrap
|
|
||||||
self.run()
|
|
||||||
File "threading.py", line 440, in run
|
|
||||||
self.__target(*self.__args, **self.__kwargs)
|
|
||||||
File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 140, in _thread_receiver
|
|
||||||
self._stopsend()
|
|
||||||
File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 329, in _stopsend
|
|
||||||
self._send(None)
|
|
||||||
File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 147, in _send
|
|
||||||
self._io.close_write()
|
|
||||||
File "/home/hpk/py/branch/event/py/execnet/inputoutput.py", line 106, in close_write
|
|
||||||
self.outfile.close()
|
|
||||||
IOError: [Errno 32] Broken pipe
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
@ -6,6 +6,7 @@ of the following people and organizations:
|
||||||
|
|
||||||
Holger Krekel, holger at merlinux eu
|
Holger Krekel, holger at merlinux eu
|
||||||
Guido Wesdorp, johnny at johnnydebris net
|
Guido Wesdorp, johnny at johnnydebris net
|
||||||
|
Samuele Pedroni, pedronis at openend se
|
||||||
Carl Friedrich Bolz, cfbolz at gmx de
|
Carl Friedrich Bolz, cfbolz at gmx de
|
||||||
Armin Rigo, arigo at tunes org
|
Armin Rigo, arigo at tunes org
|
||||||
Maciek Fijalkowski, fijal at genesilico.pl
|
Maciek Fijalkowski, fijal at genesilico.pl
|
||||||
|
@ -14,7 +15,6 @@ of the following people and organizations:
|
||||||
|
|
||||||
Contributors include::
|
Contributors include::
|
||||||
|
|
||||||
Samuele Pedroni
|
|
||||||
Chris Lamb
|
Chris Lamb
|
||||||
Harald Armin Massa
|
Harald Armin Massa
|
||||||
Ralf Schmitt
|
Ralf Schmitt
|
||||||
|
|
Loading…
Reference in New Issue