diff --git a/py/doc/TODO.txt b/py/doc/TODO.txt index 4873776d4..9007ce8c5 100644 --- a/py/doc/TODO.txt +++ b/py/doc/TODO.txt @@ -4,26 +4,27 @@ Things to do for 1.0.0 py.test -------------- -- BUG: write test/fix --showlocals (not showing anything) +- simplify collect API + +- get APIGEN back to work + +- get web reporter back to work + +- introduce decorator "shouldfail" or "xfail" + as to mark a test as "expected to fail", + report specially if it surprisingly passes + +- nightly test runs on multiple platforms - review and refactor architecture of py.test with particular respect to: - - allow custom reporting - writing (stacked) extensions / plugins (compared to Nose) - - event naming and processing - porting existing extensions (htmlconftest / buildbot / PyPy's conftest's ...) - fast and stable distributed testing - reliable cross-platform testing -- fix reporting/usage degradation after reporter-merge merge: - - collapse skips with same reason and lineno into one line - -- fix and investigate win32 failures - -- (needs review) adjust py.test documentation to reflect new - collector/session architecture - -- document py.test's conftest.py approach +- improve py.test documentation to reflect new + event architecture - review and optimize skip-handling (it can be quite slow in certain situations because e.g. setup/teardown is fully performed @@ -40,8 +41,11 @@ py.execnet py.apigen ---------------- -- refactor to produce intermediate data/files capturing - info of test runs +- make it work again + +- py.apigen tool -> separate runtime-data collection and + web page generation. (see M750), provide "py.apigen" tool + - refactor html renderer to work on intermediate data/files rather than on the live data @@ -55,12 +59,6 @@ ld (review and shift to above) refactorings ------------------ -- session / collection unification (particularly tryiter and buildname2items) - -- reporting unification, i.e. use dist-testing Reporter class - also for "normal" session, consider introduction of tkinter - session (M978) - - refine doctests usage (particularly skips of doctests if some imports/conditions are not satisfied) @@ -74,22 +72,17 @@ refactorings - unification of "gateway"/host setup and teardown, including rsyncing, i.e. cross-platform and dist-testing. -- py.apigen tool -> separate runtime-data collection and - web page generation. (see M750), provide "py.apigen" tool - for generating API documentation - - py.log: unify API, possibly deprecate duplicate ones, base things on a Config object (hte latter almost a feature though) (M988) -- consider setup/teardown for generative tests (M826) - -- fix teardown problems regarding when teardown is done (should be done - after test run, not before the next one) +- see to teardown more eagerly features -------------- +- (Harald Armin Massa): make py.exe work with py lib +- optimize file checking with --looponfailing (harald has code for win32) - have a py.test scan/run database for results and test names etc. (to allow quicker selection of tests and post-run information on failures etc.) (M760) @@ -111,467 +104,10 @@ features - review svn-testing (and escape characters), consider svn-bindings (M634) - -packaging -------------------------------------- - -* debian and TAR/zip packages for py lib, - particularly look into C module issues (greenlet most importantly) - -* do something about c-extensions both on unix-ish - versus win32 systems - -* ensure compatibility with Python 2.3 - 2.5, - see what is missing for 2.2 - -* optional: support setuptools (eggs?) installs, and instally - from pypi (and register pylib there) - -* (DONE/c-modules don't) see if things work on Win32 (partially done) - -* (partly DONE) refine and implement releasescheme/download +- py.test.pdb - there is my hack for a while now, which integrates + rlcompleter2 with pdb. First of all it requires some strange changes + to rlcompleter itself, which has no tests. Long-term plan would be + to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and + have it tested. This requires work though. -APIGEN / source viewer -------------------------------------- - -* (DONE, XXX functions/methods?) integrate rest directive into - py/documentation/conftest.py - with help code from py.__.rest.directive.... - make sure that the txt files in py/documentation/ use it - -testing ------------ - -* these should all work on 1.0 and on the py lib and pypy: - - running "py.test -s" - - running "py.test --pdb" - - running "py.test --looponfailing" - - running "py.test" distributed on some hosts - - (guido tested all on win32, everything works except --dist (requires - os.fork to work)) - -code quality ------------------ - -* no function implementation longer than 30 lines - -* no lines longer than 80 characters - -* review the pylib issue tracker - (cfbolz: done: what has a 1.0.0 tag (or lower) should be looked at again) - -py.test -------- - -* (postponed, likely) py.test fails to parse strangely formatted code after assertion failure - -Missing docstrings ------------------- - -:: - - code.Traceback.recursionindex misses a docstring - code.Traceback.filter misses a docstring - code.Traceback.cut misses a docstring - code.Traceback.__getitem__ misses a docstring - code.Traceback.Entry misses a docstring - code.Traceback.Entry.ishidden misses a docstring - code.Traceback.Entry.getfirstlinesource misses a docstring - code.Traceback.Entry.__str__ misses a docstring - code.Traceback.Entry.__repr__ misses a docstring - code.Traceback.Entry.__init__ misses a docstring - code.Source.getblockend misses a docstring - code.Source.__str__ misses a docstring - code.Source.__len__ misses a docstring - code.Source.__init__ misses a docstring - code.Source.__getslice__ misses a docstring - code.Source.__getitem__ misses a docstring - code.Source.__eq__ misses a docstring - code.Frame.repr misses a docstring - code.Frame.is_true misses a docstring - code.Frame.getargs misses a docstring - code.Frame.exec_ misses a docstring - code.Frame.eval misses a docstring - code.Frame.__init__ misses a docstring - code.ExceptionInfo.exconly misses a docstring - code.ExceptionInfo.errisinstance misses a docstring - code.ExceptionInfo.__str__ misses a docstring - code.ExceptionInfo.__init__ misses a docstring - code.Code misses a docstring - code.Code.source misses a docstring - code.Code.getargs misses a docstring - code.Code.__init__ misses a docstring - code.Code.__eq__ misses a docstring - execnet.SshGateway misses a docstring - execnet.SshGateway.join misses a docstring - execnet.SshGateway.exit misses a docstring - execnet.SshGateway.__repr__ misses a docstring - execnet.SshGateway.__init__ misses a docstring - execnet.SshGateway.ThreadOut.write misses a docstring - execnet.SshGateway.ThreadOut.setwritefunc misses a docstring - execnet.SshGateway.ThreadOut.setdefaultwriter misses a docstring - execnet.SshGateway.ThreadOut.resetdefault misses a docstring - execnet.SshGateway.ThreadOut.isatty misses a docstring - execnet.SshGateway.ThreadOut.flush misses a docstring - execnet.SshGateway.ThreadOut.delwritefunc misses a docstring - execnet.SshGateway.ThreadOut.deinstall misses a docstring - execnet.SocketGateway misses a docstring - execnet.SocketGateway.join misses a docstring - execnet.SocketGateway.exit misses a docstring - execnet.SocketGateway.__repr__ misses a docstring - execnet.SocketGateway.__init__ misses a docstring - execnet.PopenGateway misses a docstring - execnet.PopenGateway.remote_bootstrap_gateway misses a docstring - execnet.PopenGateway.join misses a docstring - execnet.PopenGateway.exit misses a docstring - execnet.PopenGateway.__repr__ misses a docstring - execnet.PopenGateway.__init__ misses a docstring - initpkg misses a docstring - log.setconsumer misses a docstring - log.get misses a docstring - log.Syslog misses a docstring - log.STDOUT misses a docstring - log.STDERR misses a docstring - log.Producer.set_consumer misses a docstring - log.Producer.get_consumer misses a docstring - log.Producer.__repr__ misses a docstring - log.Producer.__init__ misses a docstring - log.Producer.__getattr__ misses a docstring - log.Producer.__call__ misses a docstring - log.Producer.Message misses a docstring - log.Producer.Message.prefix misses a docstring - log.Producer.Message.content misses a docstring - log.Producer.Message.__str__ misses a docstring - log.Producer.Message.__init__ misses a docstring - log.Path misses a docstring - log.Path.__init__ misses a docstring - log.Path.__call__ misses a docstring - magic.View.__viewkey__ misses a docstring - magic.View.__repr__ misses a docstring - magic.View.__new__ misses a docstring - magic.View.__matchkey__ misses a docstring - magic.View.__getattr__ misses a docstring - magic.AssertionError misses a docstring - path.svnwc.visit misses a docstring - path.svnwc.mkdir misses a docstring - path.svnwc.dump misses a docstring - path.svnwc.check misses a docstring - path.svnwc.add misses a docstring - path.svnwc.__str__ misses a docstring - path.svnwc.__repr__ misses a docstring - path.svnwc.__new__ misses a docstring - path.svnwc.__iter__ misses a docstring - path.svnwc.__hash__ misses a docstring - path.svnwc.__eq__ misses a docstring - path.svnwc.__div__ misses a docstring - path.svnwc.__contains__ misses a docstring - path.svnwc.__cmp__ misses a docstring - path.svnwc.__add__ misses a docstring - path.svnwc.Checkers misses a docstring - path.svnurl.visit misses a docstring - path.svnurl.check misses a docstring - path.svnurl.__repr__ misses a docstring - path.svnurl.__new__ misses a docstring - path.svnurl.__ne__ misses a docstring - path.svnurl.__iter__ misses a docstring - path.svnurl.__hash__ misses a docstring - path.svnurl.__div__ misses a docstring - path.svnurl.__contains__ misses a docstring - path.svnurl.__cmp__ misses a docstring - path.svnurl.__add__ misses a docstring - path.svnurl.Checkers misses a docstring - path.local.visit misses a docstring - path.local.sysexec has an 'XXX' in its docstring - path.local.check misses a docstring - path.local.__repr__ misses a docstring - path.local.__iter__ misses a docstring - path.local.__hash__ misses a docstring - path.local.__eq__ misses a docstring - path.local.__div__ misses a docstring - path.local.__contains__ misses a docstring - path.local.__cmp__ misses a docstring - path.local.__add__ misses a docstring - path.local.Checkers misses a docstring - test.rest.RestReporter misses a docstring - test.rest.RestReporter.summary misses a docstring - test.rest.RestReporter.skips misses a docstring - test.rest.RestReporter.repr_traceback misses a docstring - test.rest.RestReporter.repr_source misses a docstring - test.rest.RestReporter.repr_signal misses a docstring - test.rest.RestReporter.repr_failure misses a docstring - test.rest.RestReporter.report_unknown misses a docstring - test.rest.RestReporter.report_TestStarted misses a docstring - test.rest.RestReporter.report_TestFinished misses a docstring - test.rest.RestReporter.report_SkippedTryiter misses a docstring - test.rest.RestReporter.report_SendItem misses a docstring - test.rest.RestReporter.report_RsyncFinished misses a docstring - test.rest.RestReporter.report_ReceivedItemOutcome misses a docstring - test.rest.RestReporter.report_Nodes misses a docstring - test.rest.RestReporter.report_ItemStart misses a docstring - test.rest.RestReporter.report_ImmediateFailure misses a docstring - test.rest.RestReporter.report_HostReady misses a docstring - test.rest.RestReporter.report_HostRSyncing misses a docstring - test.rest.RestReporter.report_FailedTryiter misses a docstring - test.rest.RestReporter.report misses a docstring - test.rest.RestReporter.print_summary misses a docstring - test.rest.RestReporter.prepare_source misses a docstring - test.rest.RestReporter.hangs misses a docstring - test.rest.RestReporter.get_rootpath misses a docstring - test.rest.RestReporter.get_path_from_item misses a docstring - test.rest.RestReporter.get_linkwriter misses a docstring - test.rest.RestReporter.get_item_name misses a docstring - test.rest.RestReporter.get_host misses a docstring - test.rest.RestReporter.failures misses a docstring - test.rest.RestReporter.add_rest misses a docstring - test.rest.RestReporter.__init__ misses a docstring - test.rest.RelLinkWriter misses a docstring - test.rest.RelLinkWriter.get_link misses a docstring - test.rest.NoLinkWriter misses a docstring - test.rest.NoLinkWriter.get_link misses a docstring - test.rest.LinkWriter misses a docstring - test.rest.LinkWriter.get_link misses a docstring - test.rest.LinkWriter.__init__ misses a docstring - test.exit misses a docstring - test.deprecated_call misses a docstring - test.collect.Module misses a docstring - test.collect.Module.tryiter has an 'XXX' in its docstring - test.collect.Module.teardown misses a docstring - test.collect.Module.startcapture misses a docstring - test.collect.Module.skipbykeyword misses a docstring - test.collect.Module.setup misses a docstring - test.collect.Module.run misses a docstring - test.collect.Module.makeitem misses a docstring - test.collect.Module.listnames misses a docstring - test.collect.Module.join misses a docstring - test.collect.Module.haskeyword misses a docstring - test.collect.Module.getsortvalue misses a docstring - test.collect.Module.getpathlineno misses a docstring - test.collect.Module.getouterr misses a docstring - test.collect.Module.getitembynames misses a docstring - test.collect.Module.funcnamefilter misses a docstring - test.collect.Module.finishcapture misses a docstring - test.collect.Module.classnamefilter misses a docstring - test.collect.Module.buildname2items misses a docstring - test.collect.Module.__repr__ misses a docstring - test.collect.Module.__ne__ misses a docstring - test.collect.Module.__init__ misses a docstring - test.collect.Module.__hash__ misses a docstring - test.collect.Module.__eq__ misses a docstring - test.collect.Module.__cmp__ misses a docstring - test.collect.Module.Skipped misses a docstring - test.collect.Module.Passed misses a docstring - test.collect.Module.Outcome misses a docstring - test.collect.Module.Failed misses a docstring - test.collect.Module.ExceptionFailure misses a docstring - test.collect.Instance misses a docstring - test.collect.Instance.tryiter has an 'XXX' in its docstring - test.collect.Instance.teardown misses a docstring - test.collect.Instance.startcapture misses a docstring - test.collect.Instance.skipbykeyword misses a docstring - test.collect.Instance.setup misses a docstring - test.collect.Instance.run misses a docstring - test.collect.Instance.makeitem misses a docstring - test.collect.Instance.listnames misses a docstring - test.collect.Instance.join misses a docstring - test.collect.Instance.haskeyword misses a docstring - test.collect.Instance.getsortvalue misses a docstring - test.collect.Instance.getpathlineno misses a docstring - test.collect.Instance.getouterr misses a docstring - test.collect.Instance.getitembynames misses a docstring - test.collect.Instance.funcnamefilter misses a docstring - test.collect.Instance.finishcapture misses a docstring - test.collect.Instance.classnamefilter misses a docstring - test.collect.Instance.buildname2items misses a docstring - test.collect.Instance.__repr__ misses a docstring - test.collect.Instance.__ne__ misses a docstring - test.collect.Instance.__init__ misses a docstring - test.collect.Instance.__hash__ misses a docstring - test.collect.Instance.__eq__ misses a docstring - test.collect.Instance.__cmp__ misses a docstring - test.collect.Generator misses a docstring - test.collect.Generator.tryiter has an 'XXX' in its docstring - test.collect.Generator.teardown misses a docstring - test.collect.Generator.startcapture misses a docstring - test.collect.Generator.skipbykeyword misses a docstring - test.collect.Generator.setup misses a docstring - test.collect.Generator.run misses a docstring - test.collect.Generator.listnames misses a docstring - test.collect.Generator.join misses a docstring - test.collect.Generator.haskeyword misses a docstring - test.collect.Generator.getsortvalue misses a docstring - test.collect.Generator.getpathlineno misses a docstring - test.collect.Generator.getouterr misses a docstring - test.collect.Generator.getitembynames misses a docstring - test.collect.Generator.getcallargs misses a docstring - test.collect.Generator.finishcapture misses a docstring - test.collect.Generator.buildname2items misses a docstring - test.collect.Generator.__repr__ misses a docstring - test.collect.Generator.__ne__ misses a docstring - test.collect.Generator.__init__ misses a docstring - test.collect.Generator.__hash__ misses a docstring - test.collect.Generator.__eq__ misses a docstring - test.collect.Generator.__cmp__ misses a docstring - test.collect.DoctestFile misses a docstring - test.collect.DoctestFile.tryiter has an 'XXX' in its docstring - test.collect.DoctestFile.teardown misses a docstring - test.collect.DoctestFile.startcapture misses a docstring - test.collect.DoctestFile.skipbykeyword misses a docstring - test.collect.DoctestFile.setup misses a docstring - test.collect.DoctestFile.run misses a docstring - test.collect.DoctestFile.makeitem misses a docstring - test.collect.DoctestFile.listnames misses a docstring - test.collect.DoctestFile.join misses a docstring - test.collect.DoctestFile.haskeyword misses a docstring - test.collect.DoctestFile.getsortvalue misses a docstring - test.collect.DoctestFile.getpathlineno misses a docstring - test.collect.DoctestFile.getouterr misses a docstring - test.collect.DoctestFile.getitembynames misses a docstring - test.collect.DoctestFile.funcnamefilter misses a docstring - test.collect.DoctestFile.finishcapture misses a docstring - test.collect.DoctestFile.classnamefilter misses a docstring - test.collect.DoctestFile.buildname2items misses a docstring - test.collect.DoctestFile.__repr__ misses a docstring - test.collect.DoctestFile.__ne__ misses a docstring - test.collect.DoctestFile.__init__ misses a docstring - test.collect.DoctestFile.__hash__ misses a docstring - test.collect.DoctestFile.__eq__ misses a docstring - test.collect.DoctestFile.__cmp__ misses a docstring - test.collect.Directory misses a docstring - test.collect.Directory.tryiter has an 'XXX' in its docstring - test.collect.Directory.teardown misses a docstring - test.collect.Directory.startcapture misses a docstring - test.collect.Directory.skipbykeyword misses a docstring - test.collect.Directory.setup misses a docstring - test.collect.Directory.run misses a docstring - test.collect.Directory.recfilter misses a docstring - test.collect.Directory.makeitem misses a docstring - test.collect.Directory.listnames misses a docstring - test.collect.Directory.join misses a docstring - test.collect.Directory.haskeyword misses a docstring - test.collect.Directory.getsortvalue misses a docstring - test.collect.Directory.getpathlineno misses a docstring - test.collect.Directory.getouterr misses a docstring - test.collect.Directory.getitembynames misses a docstring - test.collect.Directory.finishcapture misses a docstring - test.collect.Directory.filefilter misses a docstring - test.collect.Directory.buildname2items misses a docstring - test.collect.Directory.__repr__ misses a docstring - test.collect.Directory.__ne__ misses a docstring - test.collect.Directory.__init__ misses a docstring - test.collect.Directory.__hash__ misses a docstring - test.collect.Directory.__eq__ misses a docstring - test.collect.Directory.__cmp__ misses a docstring - test.collect.Collector misses a docstring - test.collect.Collector.tryiter has an 'XXX' in its docstring - test.collect.Collector.teardown misses a docstring - test.collect.Collector.startcapture misses a docstring - test.collect.Collector.skipbykeyword misses a docstring - test.collect.Collector.setup misses a docstring - test.collect.Collector.run misses a docstring - test.collect.Collector.listnames misses a docstring - test.collect.Collector.join misses a docstring - test.collect.Collector.haskeyword misses a docstring - test.collect.Collector.getsortvalue misses a docstring - test.collect.Collector.getpathlineno misses a docstring - test.collect.Collector.getouterr misses a docstring - test.collect.Collector.getitembynames misses a docstring - test.collect.Collector.finishcapture misses a docstring - test.collect.Collector.buildname2items misses a docstring - test.collect.Collector.__repr__ misses a docstring - test.collect.Collector.__ne__ misses a docstring - test.collect.Collector.__init__ misses a docstring - test.collect.Collector.__hash__ misses a docstring - test.collect.Collector.__eq__ misses a docstring - test.collect.Collector.__cmp__ misses a docstring - test.collect.Class misses a docstring - test.collect.Class.tryiter has an 'XXX' in its docstring - test.collect.Class.teardown misses a docstring - test.collect.Class.startcapture misses a docstring - test.collect.Class.skipbykeyword misses a docstring - test.collect.Class.setup misses a docstring - test.collect.Class.run misses a docstring - test.collect.Class.makeitem misses a docstring - test.collect.Class.listnames misses a docstring - test.collect.Class.join misses a docstring - test.collect.Class.haskeyword misses a docstring - test.collect.Class.getsortvalue misses a docstring - test.collect.Class.getpathlineno misses a docstring - test.collect.Class.getouterr misses a docstring - test.collect.Class.getitembynames misses a docstring - test.collect.Class.funcnamefilter misses a docstring - test.collect.Class.finishcapture misses a docstring - test.collect.Class.classnamefilter misses a docstring - test.collect.Class.buildname2items misses a docstring - test.collect.Class.__repr__ misses a docstring - test.collect.Class.__ne__ misses a docstring - test.collect.Class.__init__ misses a docstring - test.collect.Class.__hash__ misses a docstring - test.collect.Class.__eq__ misses a docstring - test.collect.Class.__cmp__ misses a docstring - test.cmdline.main misses a docstring - test.Item misses a docstring - test.Item.tryiter has an 'XXX' in its docstring - test.Item.teardown misses a docstring - test.Item.startcapture misses a docstring - test.Item.skipbykeyword misses a docstring - test.Item.setup misses a docstring - test.Item.run misses a docstring - test.Item.listnames misses a docstring - test.Item.join misses a docstring - test.Item.haskeyword misses a docstring - test.Item.getsortvalue misses a docstring - test.Item.getpathlineno misses a docstring - test.Item.getouterr misses a docstring - test.Item.getitembynames misses a docstring - test.Item.finishcapture misses a docstring - test.Item.buildname2items misses a docstring - test.Item.__repr__ misses a docstring - test.Item.__ne__ misses a docstring - test.Item.__init__ misses a docstring - test.Item.__hash__ misses a docstring - test.Item.__eq__ misses a docstring - test.Item.__cmp__ misses a docstring - test.Function.tryiter has an 'XXX' in its docstring - test.Function.teardown misses a docstring - test.Function.startcapture misses a docstring - test.Function.skipbykeyword misses a docstring - test.Function.setup misses a docstring - test.Function.run misses a docstring - test.Function.listnames misses a docstring - test.Function.join misses a docstring - test.Function.haskeyword misses a docstring - test.Function.getsortvalue misses a docstring - test.Function.getpathlineno misses a docstring - test.Function.getouterr misses a docstring - test.Function.getitembynames misses a docstring - test.Function.finishcapture misses a docstring - test.Function.buildname2items misses a docstring - test.Function.__repr__ misses a docstring - test.Function.__ne__ misses a docstring - test.Function.__init__ misses a docstring - test.Function.__hash__ misses a docstring - test.Function.__eq__ misses a docstring - test.Function.__cmp__ misses a docstring - test.Config.__init__ misses a docstring - xml.raw.__init__ misses a docstring - xml.html misses a docstring - xml.html.__tagclass__ misses a docstring - xml.html.__tagclass__.unicode misses a docstring - xml.html.__tagclass__.__unicode__ misses a docstring - xml.html.__tagclass__.__repr__ misses a docstring - xml.html.__tagclass__.__init__ misses a docstring - xml.html.__tagclass__.Attr misses a docstring - xml.html.__tagclass__.Attr.__init__ misses a docstring - xml.html.__metaclass__ misses a docstring - xml.html.__metaclass__.__getattr__ misses a docstring - xml.html.Style misses a docstring - xml.html.Style.__init__ misses a docstring - xml.escape misses a docstring - xml.Tag misses a docstring - xml.Tag.unicode misses a docstring - xml.Tag.__unicode__ misses a docstring - xml.Tag.__repr__ misses a docstring - xml.Tag.__init__ misses a docstring - xml.Namespace misses a docstring diff --git a/py/doc/future/code_template.txt b/py/doc/future/code_template.txt deleted file mode 100644 index 86b1b1d93..000000000 --- a/py/doc/future/code_template.txt +++ /dev/null @@ -1,640 +0,0 @@ -=============================================================== -py.code_template: Lightweight and flexible code template system -=============================================================== - -.. contents:: -.. sectnum:: - -Motivation -========== - -There are as many python templating systems as there are web frameworks -(a lot). This is partly because it is so darned easy to write a templating -system in Python. What are the distinguishing characteristics of the -py.code_template templating system? - - * Optimized for generating code (Python, C, bash scripts, etc.), - not XML or HTML - - * Designed for use by Python programmers, not by web artists - - + Aesthetic sensibilities are different - - + The templates should be an organic part of a module -- just more code - - + Templates do not need to be incredibly full-featured, because - programmers are perfectly capable of escaping to Python for - advanced features. - - - No requirement to support inheritance - - No requirement to support exec - - * Designed so that templates can be coded in the most natural way - for the task at hand - - + Generation of code and scripts often does not follow MVC paradigm! - - + Small template fragments are typically coded *inside* Python modules - - + Sometimes it is natural to put strings inside code; sometimes it is - natural to put code inside strings. Both should be supported as - reasonably and naturally as possible. - -Imaginary-world examples -======================== - -These would be real-world examples, but, not only is this module not yet -implemented, as of now, PyPy is not incredibly useful to the average -programmer... - -translator/c/genc.py --------------------- - -The original function:: - - def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]): - # - # All declarations - # - structdeflist = database.getstructdeflist() - print >> f - print >> f, '/***********************************************************/' - print >> f, '/*** Structure definitions ***/' - print >> f - for node in structdeflist: - print >> f, 'struct %s;' % node.name - print >> f - for node in structdeflist: - for line in node.definition(): - print >> f, line - print >> f - print >> f, '/***********************************************************/' - print >> f, '/*** Forward declarations ***/' - print >> f - for node in database.globalcontainers(): - for line in node.forward_declaration(): - print >> f, line - - # - # Implementation of functions and global structures and arrays - # - print >> f - print >> f, '/***********************************************************/' - print >> f, '/*** Implementations ***/' - print >> f - for line in preimplementationlines: - print >> f, line - print >> f, '#include "src/g_include.h"' - print >> f - blank = True - for node in database.globalcontainers(): - if blank: - print >> f - blank = False - for line in node.implementation(): - print >> f, line - blank = True - -This could be refactored heavily. An initial starting point -would look something like this, although later, the template -instance could be passed in and reused directly, rather than -passing the file handle around:: - - def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]): - def container_implementation(): - # Helper function designed to introduce blank lines - # between container implementations - - blank = True - for node in database.globalcontainers(): - if blank: - yield '' - blank = False - for line in node.implementation(): - yield line - blank = True - - t = code_template.Template() - # - # All declarations - # - structdeflist = database.getstructdeflist() - t.write(dedent=8, text=''' - - /***********************************************************/ - /*** Structure definitions ***/ - - {for node in structdeflist} - struct {node.name}; - {endfor} - - {for node in structdeflist} - {for line in node.definition} - {line} - {endfor} - {endfor} - - /***********************************************************/ - /*** Forward declarations ***/ - - {for node in database.globalcontainers()} - {for line in node.forward_declaration()} - {line} - {endfor} - {endfor} - - {** - ** Implementation of functions and global structures and arrays - **} - - /***********************************************************/ - /*** Implementations ***/ - - {for line in preimplementationlines} - {line} - {endfor} - - #include "src/g_include.h" - - {for line in container_implementation()} - {line} - {endfor} - """) - t.output(f) - -translator/c/genc.py gen_makefile ---------------------------------- - -The original code:: - - MAKEFILE = ''' - CC = gcc - - $(TARGET): $(OBJECTS) - \t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS) - - %.o: %.c - \t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS) - - clean: - \trm -f $(OBJECTS) - ''' - - def gen_makefile(self, targetdir): - def write_list(lst, prefix): - for i, fn in enumerate(lst): - print >> f, prefix, fn, - if i < len(lst)-1: - print >> f, '\\' - else: - print >> f - prefix = ' ' * len(prefix) - - compiler = self.getccompiler(extra_includes=['.']) - cfiles = [] - ofiles = [] - for fn in compiler.cfilenames: - fn = py.path.local(fn).basename - assert fn.endswith('.c') - cfiles.append(fn) - ofiles.append(fn[:-2] + '.o') - - f = targetdir.join('Makefile').open('w') - print >> f, '# automatically generated Makefile' - print >> f - print >> f, 'TARGET =', py.path.local(compiler.outputfilename).basename - print >> f - write_list(cfiles, 'SOURCES =') - print >> f - write_list(ofiles, 'OBJECTS =') - print >> f - args = ['-l'+libname for libname in compiler.libraries] - print >> f, 'LIBS =', ' '.join(args) - args = ['-L'+path for path in compiler.library_dirs] - print >> f, 'LIBDIRS =', ' '.join(args) - args = ['-I'+path for path in compiler.include_dirs] - write_list(args, 'INCLUDEDIRS =') - print >> f - print >> f, 'CFLAGS =', ' '.join(compiler.compile_extra) - print >> f, 'LDFLAGS =', ' '.join(compiler.link_extra) - print >> f, MAKEFILE.strip() - f.close() - - -Could look something like this:: - - MAKEFILE = ''' - # automatically generated Makefile - - TARGET = {py.path.local(compiler.outputfilename).basename} - - {for line in write_list(cfiles, 'SOURCES =')} - {line} - {endfor} - - {for line in write_list(ofiles, 'OBJECTS =')} - {line} - {endfor} - - LIBS ={for libname in compiler.libraries} -l{libname}{endfor} - LIBDIRS ={for path in compiler.library_dirs} -L{path}{endfor} - INCLUDEDIRS ={for path in compiler.include_dirs} -I{path}{endfor} - - CFLAGS ={for extra in compiler.compile_extra} {extra}{endfor} - LDFLAGS ={for extra in compiler.link_extra} {extra}{endfor} - - CC = gcc - - $(TARGET): $(OBJECTS) - \t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS) - - %.o: %.c - \t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS) - - clean: - \trm -f $(OBJECTS) - ''' - - def gen_makefile(self, targetdir): - def write_list(lst, prefix): - for i, fn in enumerate(lst): - yield '%s %s %s' % (prefix, fn, i < len(list)-1 and '\\' or '') - prefix = ' ' * len(prefix) - - compiler = self.getccompiler(extra_includes=['.']) - cfiles = [] - ofiles = [] - for fn in compiler.cfilenames: - fn = py.path.local(fn).basename - assert fn.endswith('.c') - cfiles.append(fn) - ofiles.append(fn[:-2] + '.o') - - code_template.Template(MAKEFILE).output(targetdir.join('Makefile')) - - -translator/llvm/module/excsupport.py ------------------------------------- - -The original string:: - - invokeunwind_code = ''' - ccc %(returntype)s%%__entrypoint__%(entrypointname)s { - %%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception - - no_exception: - store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - ret %(returntype)s %%result - - exception: - ret %(noresult)s - } - - ccc int %%__entrypoint__raised_LLVMException() { - %%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type - %%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int - ret int %%result - } - - internal fastcc void %%unwind() { - unwind - } - ''' - -Could look something like this if it was used in conjunction with a template:: - - invokeunwind_code = ''' - ccc {returntype}%__entrypoint__{entrypointname} { - %result = invoke {cconv} {returntype}%{entrypointname} to label %no_exception except label %exception - - no_exception: - store %RPYTHON_EXCEPTION_VTABLE* null, %RPYTHON_EXCEPTION_VTABLE** %last_exception_type - ret {returntype} %result - - exception: - ret {noresult} - } - - ccc int %__entrypoint__raised_LLVMException() { - %tmp = load %RPYTHON_EXCEPTION_VTABLE** %last_exception_type - %result = cast %RPYTHON_EXCEPTION_VTABLE* %tmp to int - ret int %result - } - - internal fastcc void %unwind() { - unwind - } - ''' - - -Template syntax -=============== - -Design decision ---------------- - -As all programmers must know by now, all the special symbols on the keyboard -are quite heavily overloaded. Often, template systems work around this fact -by having special notation like `<*` ... `*>` or {% ... %}. Some template systems -even have multiple special notations -- one for comments, one for statements, -one for expressions, etc. - -I find these hard to type and ugly. Other markups are either too lightweight, -or use characters which occur so frequently in the target languages that it -becomes hard to distinguish marked-up content from content which should be -rendered as-is. - -The compromise taken by *code_template* is to use braces (**{}**) for markup. - -This immediately raises the question: what about when the marked-up language -is C or C++? The answer is that if the leading brace is immediately followed -by whitespace, it is normal text; if not it is the start of markup. - -To support normal text which has a leading brace immediately followed by -an identifier, if the first whitespace character after the brace is a space -character (e.g. not a newline or tab), it will be removed from the output. - -Examples:: - - { This is normal text and the space between { and This will be removed} - {'this must be a valid Python expression' + ' because it is treated as markup'} - { - This is normal text, but nothing is altered (the newline is kept intact) - } - - {{1:'Any valid Python expression is allowed as markup'}[1].ljust(30)} - -.. _`Code element`: - -Elements --------- - -Templates consist of normal text and code elements. -(Comments are considered to be code elements.) - -All code elements start with a `left brace`_ which is not followed by -whitespace. - -Keyword element -~~~~~~~~~~~~~~~ - -A keyword element is a `code element`_ which starts with a keyword_. - -For example, *{if foo}* is a keyword element, but *{foo}* is a `substituted expression`_. - -Keyword -~~~~~~~ - -A keyword is a word used in `conditional text`_ or in `repeated text`_, e.g. -one of *if*, *elif*, *else*, *endif*, *for*, or *endfor*. - -Keywords are designed to match their Python equivalents. However, since -templates cannot use spacing to indicate expression nesting, the additional -keywords *endif* and *endfor* are required. - -Left brace -~~~~~~~~~~ - -All elements other than normal text start with a left brace -- the symbol '{', -sometimes known as a 'curly bracket'. A left brace is itself considered -to be normal text if it is followed by whitespace. If the whitespace starts -with a space character, that space character will be stripped from the output. -If the whitespace starts with a tab or linefeed character, the whitespace will -be left in the output. - -Normal Text -~~~~~~~~~~~ - -Normal text remains unsubstituted. Transition from text to the other elements -is effected by use of a `left brace`_ which is not followed by whitespace. - -Comment -~~~~~~~ - -A comment starts with a left brace followed by an asterisk ('{`*`'), and -ends with an asterisk followed by a right brace ('`*`}'):: - - This is a template -- this text will be copied to the output. - {* This is a comment and this text will not be copied to the output *} - - {* - Comments can span lines, - but cannot be nested - *} - -Substituted expression -~~~~~~~~~~~~~~~~~~~~~~ - -Any python expression may be used:: - - Dear {record.name}, - we are sorry to inform you that you did not win {record.contest}. - -The expression must be surrounded by braces, and there must not be any -whitespace between the leftmost brace and the start of the expression. - -The expression will automatically be converted to a string with str(). - -Conditional text -~~~~~~~~~~~~~~~~ - -The following template has text which is included conditionally:: - - This text will always be included in the output - {if foo} - This text will be included if foo is true - {elif bar} - This text will be included if foo is not true but bar is true - {else} - This text will be included if neither foo nor bar is true - {endif} - -The {elif} and {else} elements are optional. - -Repeated text -~~~~~~~~~~~~~ - -The following template shows how to pull multiple items out of a list:: - - {for student, score in sorted(scorelist)} - {student.ljust(20)} {score} - {endfor} - -Whitespace removal or modification ----------------------------------- - -In general, whitespace in `Normal Text`_ is transferred unchanged to the -output. There are three exceptions to this rule: - -Line separators -~~~~~~~~~~~~~~~ - -Each newline is converted to the final output using os.linesep. - -Beginning or end of string -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -py.code_template is designed to allow easy use of templates inside of python -modules. The canonical way to write a template is inside a triple-quoted -string, e.g.:: - - my_template = ''' - This is my template. It can have any text at all in it except - another triple-single-quote. - ''' - -To support this usage, if the first character is a newline, it will be -removed, and if the last line consists solely of whitespace with no -trailing newline, it will also be removed. - -A comment or single keyword element on a line -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Whenever a `keyword element`_ or comment_ is on a line -*by itself*, that line will not be copied to the output. - -This happens when: - - There is nothing on the line before the keyword element - or comment except whitespace (spaces and/or tabs). - - - There is nothing on the line after the keyword element - or comment except a newline. - -Note that even a multi-line comment or keyword element can -have the preceding whitespace and subsequent newline stripped -by this rule. - -The primary purpose of this rule is to allow the Python -programmer to use indentation, **even inside a template**:: - - This is a template - - {if mylist} - List items: - {for item in mylist} - - {item} - {endfor} - {endif} - -Template usage -============== - -Templates are used by importing the Template class from py.code_template, -constructing a template, and then sending data with the write() method. - -In general, there are four methods for getting the formatted data back out -of the template object: - - - read() reads all the data currently in the object - - - output(fobj) outputs the data to a file - - fobj can either be an open file object, or a string. If it is - a string, the file will be opened, written, and closed. - - - open(fobj) (or calling the object constructor with a file object) - - If the open() method is used, or if a file object is passed to - the constructor, each write() will automatically flush the data - out to the file. If the fobj is a string, it is considered to - be *owned*, otherwise it is considered to be *borrowed*. *Owned* - file objects are closed when the class is deleted. - - - write() can be explicitly called with a file object, in which case - it will invoke output() on that object after it generates the data. - -Template instantiation and methods -================================== - -template = code_template.Template(outf=None, cache=None) - -If outf is given, it will be passed to the open() method - -cache may be given as a mapping. If not given, the template will use -the shared default cache. This is not thread safe. - -template.open -------------- - -template.open(outf, borrowed = None) - -The open method closes the internal file object if it was already open, -and then re-opens it on the given file. It is an error to call open() -if there is data in the object left over from previous writes. (Call -output() instead.) - -borrowed defaults to 0 if outf is a string, and 1 if it is a file object. - -borrowed can also be set explicitly if required. - -template.close --------------- - -close() disassociates the file from the template, and closes the file if -it was not borrowed. close() is automatically called by the destructor. - -template.write --------------- - -template.write(text='', outf=None, dedent=0, localvars=None, globalvars=None, -framelevel=1) - -The write method has the following parameters: - - - text is the template itself - - - if outf is not None, the output method will be invoked on the object - after the current template is processed. If no outf is given, data - will be accumulated internal to the instance until a write() with outf - is processed, or read() or output() is called, whichever comes first, if - there is no file object. If there is a file object, data will be flushed - to the file after every write. - - - dedent, if given is applied to each line in the template, to "de-indent" - - - localvars and globalvars default to the dictionaries of the caller. A copy - of localvars is made so that the __TrueSpace__ identifier can be added. - - - cache may be given as a mapping. If not given, the template will use - the shared default cache. This is not thread safe. - - - framelevel is used to determine which stackframe to access for globals - and locals if localvars and/or globalvars are not specified. The default - is to use the caller's frame. - -The write method supports the print >> file protocol by deleting the softspace -attribute on every invocation. This allows code like:: - - t = code_template.Template() - print >> t, "Hello, world" - - -template.read --------------- - -This method reads and flushes all accumulated data in the object. Note that -if a file has been associated with the object, there will never be any data -to read. - -template.output ---------------- - -This method takes one parameter, outf. template.output() first -invokes template.read() to read and flush all accumulated data, -and then outputs the data to the file specified by outf. - -If outf has a write() method, that will be invoked with the -data. If outf has no write() method, it will be treated as -a filename, and that file will be replaced. - -Caching and thread safety -========================= - -The compiled version of every template is cached internal to the -code_template module (unless a separate cache object is specified). - -This allows efficient template reuse, but is not currently thread-safe. -Alternatively, each invocation of a template object can specify a -cache object. This is thread-safe, but not very efficient. A shared -model could be implemented later. - diff --git a/py/doc/future/planning.txt b/py/doc/future/planning.txt deleted file mode 100644 index 991e0187d..000000000 --- a/py/doc/future/planning.txt +++ /dev/null @@ -1,64 +0,0 @@ -Release -======= - -currently working configurations --------------------------------- - -2.3 - 2.4.2 work - -with setuptools: 2.3 - 2.4.2 as 'develop' - -regular installation: works mostly, strange test-failures - -to be tested: 2.2, windows - -absolutely necessary steps: ----------------------------- - - * documentation - - * improving getting started, describe install methods - * describe the rest stuff? - * py.log - * py.path is mostly undocumented, API documentation - - * basic windows testing, maybe disabling execnet?, what about the scripts in windows? - - * are all c extensions compiled when installing globally? - - * refactoring py.log - - * write/read methods on py.path should be renamed/deprecated: setcontent, getcontent instead? - - * what about _subprocess.c? - - * warning for docutils - - * don't expose _extpy - - * py/bin should be nicefied, get optparse interface - - * _findpy.py - * py.cleanup: - * py.lookup: add -i option - * pytest.cmd - * rst2pdf.py: merge with py.rest, add warnings when missing tex - * _makepyrelease.py: move somewhere - * py.countloc - * py.test - * py.rest - * win32 - - * skip tests if dependencies are not installed - -nice to have ------------- - - * sets.py, subprocess.py in compat - * fix -k option to py.test - * add --report=(text|terminal|session|rest|tkinter|rest) to py.test - * put Armin's collect class into py.__builtin__ (not done) - * try get rid of Collect.tryiter() in favour of (not done) - using Armin's collect class - - diff --git a/py/doc/future/planning2.txt b/py/doc/future/planning2.txt deleted file mode 100644 index 8f554a742..000000000 --- a/py/doc/future/planning2.txt +++ /dev/null @@ -1,39 +0,0 @@ -Missing features/bugs in pylib: -==================================== - -* new skip method, so web interface would show skips which - are broken (say py.test.skip("stuff", reason=py.test.BORKEN)), - proposed by me and xoraxax - -* integration of session classes - needed for developement - -* more robust failure recovery from execnet - not sure how to perform - it, but select() approach sounds like a plan (instead of threads) - (unsure what than) - -* provide a bit more patchy approach to green stuff, ie you import it and - all (known) operations on sockets are performed via the green interface, - this should allow using arbitrary applications (well, not using C-level - I/O) to mix with green threads. - -* integrate green execnet a bit more (?) - -* once session integration is done, it would be cool to have nightly - testing done in a systematic manner (instead of bunch of hacks, which - is how it looks like right now), so for example codespeak would be able - to store information (ie via svn) and when one woke up he can type py.test - show and see the information of all nightly test runs which he likes. - -* py.test.pdb - there is my hack for a while now, which integrates - rlcompleter2 with pdb. First of all it requires some strange changes - to rlcompleter itself, which has no tests. Long-term plan would be - to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and - have it tested. This requires work though. - -* add a link to pylib in pypy/lib? Since pylib features mostly work on top - of pypy-c, it would be nice to have it (as we do have it in svn anyway) - -* fix generative tests. - - they should be distributed atomically (for various reasons) - - fix setup/teardown logic (ie setup_generator/teardown_generator) - - XXX there was sth else diff --git a/py/doc/future/pylib_pypy.txt b/py/doc/future/pylib_pypy.txt deleted file mode 100644 index 33bd135a2..000000000 --- a/py/doc/future/pylib_pypy.txt +++ /dev/null @@ -1,37 +0,0 @@ -Here I'm trying to list all problems regarding pypy-c <-> pylib interaction -=========================================================================== - -* in test/terminal/terminal.py lines around 141:: - rev = py.__package__.getrev() - self.out.line("using py lib: %s " % ( - py.path.local(py.__file__).dirpath(), rev)) - -* py.code issues:: - def __init__(self, rawcode): - rawcode = getattr(rawcode, 'im_func', rawcode) - rawcode = getattr(rawcode, 'func_code', rawcode) - self.raw = rawcode - self.filename = rawcode.co_filename - AttributeError: 'internal-code' object has no attribute 'co_filename' - -* types.BuiltinFunctionType == types.MethodType which confuses apigen - -* compiler module problems - some bogus IndentationError - communicates by inspect.getsource() - -* execnet just hangs - -* lack of tmpfile - -* assertion error magic is not working - -* sha counting hangs (misc/testing/test\_initpkg) - -* extpy does not work, because it does not support loops in modules - (while pypy __builtins__ module has a loop), funny :-) - -* py.compat.subprocess hangs for obscure reasons - (possibly the same stuff as execnet - some threading issues and - select.select) - - Armin says: "haha, select.select probably does not release the GIL" \ No newline at end of file diff --git a/py/doc/future/rsession_todo.txt b/py/doc/future/rsession_todo.txt deleted file mode 100644 index 6e8b99020..000000000 --- a/py/doc/future/rsession_todo.txt +++ /dev/null @@ -1,15 +0,0 @@ -Various tasks which needs to be done at some point -================================================== - -* Write down pinging interface, so we'll know if hosts are responding or - are mostly down (detecting hanging nodes) - -* Write down support for rsync progress - -* Discovery of nodes which are available for accepting distributed testing - -* Test the tests rescheduling, so high-latency nodes would not take part - in that. - -* make sure that C-c semantics are ok (nodes are killed properly). - There was an attempt to do so, but it's not tested and not always work. \ No newline at end of file