test_ok1/TODO.txt

Things to do for 1.0.0 
========================= 

py.test 
--------------

- get APIGEN back to work 

- get web reporter back to work 

- introduce decorator "shouldfail" or "xfail" 
  as to mark a test as "expected to fail", 
  report specially if it surprisingly passes 

- introduce setuptools-style version checking, at least
  for py lib itself, maybe also for other packages: 

     py.checkversion("py>=1.0")

- nightly test runs on multiple platforms 

- review and refactor architecture of py.test with particular
  respect to: 
  - writing (stacked) extensions / plugins (compared to Nose) 
  - porting existing extensions (htmlconftest / buildbot / PyPy's conftest's ...) 
  - fast and stable distributed testing 
  - reliable cross-platform testing 

- improve py.test documentation to reflect new
  event architecture 

- review and optimize skip-handling (it can be quite slow in
  certain situations because e.g. setup/teardown is fully performed 
  although we have "skip by keyword" and could detect this early)

py.execnet
--------------

- cross-python version (2.2/2.3-2.5/6) and cross-platform testing of 
  setup/teardown semantics

- optimize general setup and rsync timing? 

py.apigen 
----------------

- make it work again 

see apigen_refactorings.txt

- check out CodeInvestigator 
  http://codeinvestigator.googlepages.com/main

  or other code that collects data from running a program 
  (in our case running the tests)

ld (review and shift to above)
=================================

refactorings
------------------

- refine doctests usage (particularly skips of doctests if
  some imports/conditions are not satisfied) 

- generalization of "host specifications" for execnet and
  py.test --dist usages in particular (see also revision 37500 which 
  contained a draft for that).  The goal is to have cross-platform 
  testing and dist-testing and other usages of py.execnet all 
  use a common syntax for specifiying connection methods and 
  be able to instantiate gateways/connections through it. 

- unification of "gateway"/host setup and teardown, including
  rsyncing, i.e. cross-platform and dist-testing. 

- py.log: unify API, possibly deprecate duplicate ones, 
  base things on a Config object  (hte latter almost a feature though) 
  (M988)

- see to teardown more eagerly

features 
--------------

- (Harald Armin Massa): make py2exe work with py lib 
- optimize file checking with --looponfailing (harald has code for win32)
- have a py.test scan/run database for results and test names
  etc. (to allow quicker selection of tests and post-run 
  information on failures etc.)  (M760) 

- have config options from environment, command line or conftest's

- consider features of py.apigen (recheck closed "M1016")

- integrate rlcompleter2 (make it remotely workable) 
  and maybe integrate with "pdb" / pdbplus (M975) 

- integrate native collecting of unittest.py tests from py.test 
  (along the PyPy lib-python tests)  (M987) 

- provide an automated conversion script helper for converting
  unittest.py based tests to py.test ones. (M987)

- references from ReST docs to modules, functions and classes
  of apigen generated html docs (M960)

- review svn-testing (and escape characters), consider
  svn-bindings (M634)

- py.test.pdb - there is my hack for a while now, which integrates
  rlcompleter2 with pdb. First of all it requires some strange changes
  to rlcompleter itself, which has no tests. Long-term plan would be
  to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and
  have it tested. This requires work though.

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
--- below neeeds more review ---
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

More random notes, goals
--------------------------------
- REDUCE "MAGICALNESS", from an IRC discussion with ronny:
  - integration into IDEs
  - python2.6/python3 compat? 
  - initpkg exports
  - assert reinterpretation 
  - greenlet compiles at runtime only in "dev-mode"
  - conftest's are "scary", hum, what about 

nicefications: 
looponfailing shoudl nicely signal tests that failed but now PASS tests
rename Node to itemtestloop
refactor config and session tests to go into test_distsession.py 
raises DID NOT RAISE: report the return value
have node.shutdown perform out-of-band so that shutdowns happens more quickly
tracebacks of importerrors of test modules should start with the test module file

- fix hostmanage to care for setting PYTHONPATH properly

- BRANCH: 
  adding of options 

- BRANCH: 
    TEMPDIR handling 
    syspath handling (notify on changes, restore for each test?) 

- COMPLETE REPORTING FOR MERGE!
  - remove ItemStart and CollectionStart 
    which are only needed for collectonly. 
    implement it some other way. 

  - pre-counting of test items 

  - move assert reinterp back to session? 

  - merge "--tb" and "--fulltrace" option, --tb=full 
  - implement --showouterr, don't show outerr by default 

  - domainpath? 

  - translate remote filenames to local filenames so that,
    probably based on option

  - review safe_repr

  - move OutcomeRepr.where/exconly attr to ReprExceptionInfo 
    or substitute as LocationRepr? 

- reprcrash rename message to exconly 

- test terminal reporter 
  ACCEPTANCE test for eventlog writing
  ACCEPTANCE test for acceptance custom reporting :) 

  ACCEPTANCE showing nice Collection Errors  
    
  ACCEPTANCE test for "py.test2" exit signals 
  ACCEPTANCE test for "py.test2 --traceconfig" 
  ACCEPTANCE test for nice reprsentation of failures during Collection
  ACCEPTANCE test for nice reprsentation of failures during Generator Collection 
  ACCEPTANCE test for "py.test2" conftest containing syntax errors
  
  ACCEPTANCE test for "py.test2" honouring conftest specifying "extrainfo"
  ACCEPTANCE test for "py.test2" on a simple example project 
 
- merge CollectionFinish and ItemTestReport
  maybe with base class: BaseReport and some common attrs/methods
 
- merge terminal/remote and dist-testing
  and make allocation of tests to hosts more dynamic

- time setup/teardown and the actual test runs separately 
  ACCEPTANCE:
  py.test2 --
  py.test2 --repeat=10

- expect failing tests 
  py.test2.expectfail(feature=138)


while killing a process:
Exception in thread receiver:
Traceback (most recent call last):
  File "threading.py", line 460, in __bootstrap
    self.run()
  File "threading.py", line 440, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 140, in _thread_receiver
    self._stopsend()
  File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 329, in _stopsend
    self._send(None)
  File "/home/hpk/py/branch/event/py/execnet/gateway.py", line 147, in _send
    self._io.close_write()
  File "/home/hpk/py/branch/event/py/execnet/inputoutput.py", line 106, in close_write
    self.outfile.close()
IOError: [Errno 32] Broken pipe