[svn r57707] review todo.txt and future/* items, merge into one TODO.txt file.

--HG--
branch : trunk
This commit is contained in:
hpk 2008-09-01 10:37:17 +02:00
parent d9f3c868e8
commit ee94d1d2bd
6 changed files with 26 additions and 1285 deletions

View File

@ -4,26 +4,27 @@ Things to do for 1.0.0
py.test py.test
-------------- --------------
- BUG: write test/fix --showlocals (not showing anything) - simplify collect API
- get APIGEN back to work
- get web reporter back to work
- introduce decorator "shouldfail" or "xfail"
as to mark a test as "expected to fail",
report specially if it surprisingly passes
- nightly test runs on multiple platforms
- review and refactor architecture of py.test with particular - review and refactor architecture of py.test with particular
respect to: respect to:
- allow custom reporting
- writing (stacked) extensions / plugins (compared to Nose) - writing (stacked) extensions / plugins (compared to Nose)
- event naming and processing
- porting existing extensions (htmlconftest / buildbot / PyPy's conftest's ...) - porting existing extensions (htmlconftest / buildbot / PyPy's conftest's ...)
- fast and stable distributed testing - fast and stable distributed testing
- reliable cross-platform testing - reliable cross-platform testing
- fix reporting/usage degradation after reporter-merge merge: - improve py.test documentation to reflect new
- collapse skips with same reason and lineno into one line event architecture
- fix and investigate win32 failures
- (needs review) adjust py.test documentation to reflect new
collector/session architecture
- document py.test's conftest.py approach
- review and optimize skip-handling (it can be quite slow in - review and optimize skip-handling (it can be quite slow in
certain situations because e.g. setup/teardown is fully performed certain situations because e.g. setup/teardown is fully performed
@ -40,8 +41,11 @@ py.execnet
py.apigen py.apigen
---------------- ----------------
- refactor to produce intermediate data/files capturing - make it work again
info of test runs
- py.apigen tool -> separate runtime-data collection and
web page generation. (see M750), provide "py.apigen" tool
- refactor html renderer to work on intermediate - refactor html renderer to work on intermediate
data/files rather than on the live data data/files rather than on the live data
@ -55,12 +59,6 @@ ld (review and shift to above)
refactorings refactorings
------------------ ------------------
- session / collection unification (particularly tryiter and buildname2items)
- reporting unification, i.e. use dist-testing Reporter class
also for "normal" session, consider introduction of tkinter
session (M978)
- refine doctests usage (particularly skips of doctests if - refine doctests usage (particularly skips of doctests if
some imports/conditions are not satisfied) some imports/conditions are not satisfied)
@ -74,22 +72,17 @@ refactorings
- unification of "gateway"/host setup and teardown, including - unification of "gateway"/host setup and teardown, including
rsyncing, i.e. cross-platform and dist-testing. rsyncing, i.e. cross-platform and dist-testing.
- py.apigen tool -> separate runtime-data collection and
web page generation. (see M750), provide "py.apigen" tool
for generating API documentation
- py.log: unify API, possibly deprecate duplicate ones, - py.log: unify API, possibly deprecate duplicate ones,
base things on a Config object (hte latter almost a feature though) base things on a Config object (hte latter almost a feature though)
(M988) (M988)
- consider setup/teardown for generative tests (M826) - see to teardown more eagerly
- fix teardown problems regarding when teardown is done (should be done
after test run, not before the next one)
features features
-------------- --------------
- (Harald Armin Massa): make py.exe work with py lib
- optimize file checking with --looponfailing (harald has code for win32)
- have a py.test scan/run database for results and test names - have a py.test scan/run database for results and test names
etc. (to allow quicker selection of tests and post-run etc. (to allow quicker selection of tests and post-run
information on failures etc.) (M760) information on failures etc.) (M760)
@ -111,467 +104,10 @@ features
- review svn-testing (and escape characters), consider - review svn-testing (and escape characters), consider
svn-bindings (M634) svn-bindings (M634)
- py.test.pdb - there is my hack for a while now, which integrates
packaging rlcompleter2 with pdb. First of all it requires some strange changes
------------------------------------- to rlcompleter itself, which has no tests. Long-term plan would be
to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and
* debian and TAR/zip packages for py lib, have it tested. This requires work though.
particularly look into C module issues (greenlet most importantly)
* do something about c-extensions both on unix-ish
versus win32 systems
* ensure compatibility with Python 2.3 - 2.5,
see what is missing for 2.2
* optional: support setuptools (eggs?) installs, and instally
from pypi (and register pylib there)
* (DONE/c-modules don't) see if things work on Win32 (partially done)
* (partly DONE) refine and implement releasescheme/download
APIGEN / source viewer
-------------------------------------
* (DONE, XXX functions/methods?) integrate rest directive into
py/documentation/conftest.py
with help code from py.__.rest.directive....
make sure that the txt files in py/documentation/ use it
testing
-----------
* these should all work on 1.0 and on the py lib and pypy:
- running "py.test -s"
- running "py.test --pdb"
- running "py.test --looponfailing"
- running "py.test" distributed on some hosts
(guido tested all on win32, everything works except --dist (requires
os.fork to work))
code quality
-----------------
* no function implementation longer than 30 lines
* no lines longer than 80 characters
* review the pylib issue tracker
(cfbolz: done: what has a 1.0.0 tag (or lower) should be looked at again)
py.test
-------
* (postponed, likely) py.test fails to parse strangely formatted code after assertion failure
Missing docstrings
------------------
::
code.Traceback.recursionindex misses a docstring
code.Traceback.filter misses a docstring
code.Traceback.cut misses a docstring
code.Traceback.__getitem__ misses a docstring
code.Traceback.Entry misses a docstring
code.Traceback.Entry.ishidden misses a docstring
code.Traceback.Entry.getfirstlinesource misses a docstring
code.Traceback.Entry.__str__ misses a docstring
code.Traceback.Entry.__repr__ misses a docstring
code.Traceback.Entry.__init__ misses a docstring
code.Source.getblockend misses a docstring
code.Source.__str__ misses a docstring
code.Source.__len__ misses a docstring
code.Source.__init__ misses a docstring
code.Source.__getslice__ misses a docstring
code.Source.__getitem__ misses a docstring
code.Source.__eq__ misses a docstring
code.Frame.repr misses a docstring
code.Frame.is_true misses a docstring
code.Frame.getargs misses a docstring
code.Frame.exec_ misses a docstring
code.Frame.eval misses a docstring
code.Frame.__init__ misses a docstring
code.ExceptionInfo.exconly misses a docstring
code.ExceptionInfo.errisinstance misses a docstring
code.ExceptionInfo.__str__ misses a docstring
code.ExceptionInfo.__init__ misses a docstring
code.Code misses a docstring
code.Code.source misses a docstring
code.Code.getargs misses a docstring
code.Code.__init__ misses a docstring
code.Code.__eq__ misses a docstring
execnet.SshGateway misses a docstring
execnet.SshGateway.join misses a docstring
execnet.SshGateway.exit misses a docstring
execnet.SshGateway.__repr__ misses a docstring
execnet.SshGateway.__init__ misses a docstring
execnet.SshGateway.ThreadOut.write misses a docstring
execnet.SshGateway.ThreadOut.setwritefunc misses a docstring
execnet.SshGateway.ThreadOut.setdefaultwriter misses a docstring
execnet.SshGateway.ThreadOut.resetdefault misses a docstring
execnet.SshGateway.ThreadOut.isatty misses a docstring
execnet.SshGateway.ThreadOut.flush misses a docstring
execnet.SshGateway.ThreadOut.delwritefunc misses a docstring
execnet.SshGateway.ThreadOut.deinstall misses a docstring
execnet.SocketGateway misses a docstring
execnet.SocketGateway.join misses a docstring
execnet.SocketGateway.exit misses a docstring
execnet.SocketGateway.__repr__ misses a docstring
execnet.SocketGateway.__init__ misses a docstring
execnet.PopenGateway misses a docstring
execnet.PopenGateway.remote_bootstrap_gateway misses a docstring
execnet.PopenGateway.join misses a docstring
execnet.PopenGateway.exit misses a docstring
execnet.PopenGateway.__repr__ misses a docstring
execnet.PopenGateway.__init__ misses a docstring
initpkg misses a docstring
log.setconsumer misses a docstring
log.get misses a docstring
log.Syslog misses a docstring
log.STDOUT misses a docstring
log.STDERR misses a docstring
log.Producer.set_consumer misses a docstring
log.Producer.get_consumer misses a docstring
log.Producer.__repr__ misses a docstring
log.Producer.__init__ misses a docstring
log.Producer.__getattr__ misses a docstring
log.Producer.__call__ misses a docstring
log.Producer.Message misses a docstring
log.Producer.Message.prefix misses a docstring
log.Producer.Message.content misses a docstring
log.Producer.Message.__str__ misses a docstring
log.Producer.Message.__init__ misses a docstring
log.Path misses a docstring
log.Path.__init__ misses a docstring
log.Path.__call__ misses a docstring
magic.View.__viewkey__ misses a docstring
magic.View.__repr__ misses a docstring
magic.View.__new__ misses a docstring
magic.View.__matchkey__ misses a docstring
magic.View.__getattr__ misses a docstring
magic.AssertionError misses a docstring
path.svnwc.visit misses a docstring
path.svnwc.mkdir misses a docstring
path.svnwc.dump misses a docstring
path.svnwc.check misses a docstring
path.svnwc.add misses a docstring
path.svnwc.__str__ misses a docstring
path.svnwc.__repr__ misses a docstring
path.svnwc.__new__ misses a docstring
path.svnwc.__iter__ misses a docstring
path.svnwc.__hash__ misses a docstring
path.svnwc.__eq__ misses a docstring
path.svnwc.__div__ misses a docstring
path.svnwc.__contains__ misses a docstring
path.svnwc.__cmp__ misses a docstring
path.svnwc.__add__ misses a docstring
path.svnwc.Checkers misses a docstring
path.svnurl.visit misses a docstring
path.svnurl.check misses a docstring
path.svnurl.__repr__ misses a docstring
path.svnurl.__new__ misses a docstring
path.svnurl.__ne__ misses a docstring
path.svnurl.__iter__ misses a docstring
path.svnurl.__hash__ misses a docstring
path.svnurl.__div__ misses a docstring
path.svnurl.__contains__ misses a docstring
path.svnurl.__cmp__ misses a docstring
path.svnurl.__add__ misses a docstring
path.svnurl.Checkers misses a docstring
path.local.visit misses a docstring
path.local.sysexec has an 'XXX' in its docstring
path.local.check misses a docstring
path.local.__repr__ misses a docstring
path.local.__iter__ misses a docstring
path.local.__hash__ misses a docstring
path.local.__eq__ misses a docstring
path.local.__div__ misses a docstring
path.local.__contains__ misses a docstring
path.local.__cmp__ misses a docstring
path.local.__add__ misses a docstring
path.local.Checkers misses a docstring
test.rest.RestReporter misses a docstring
test.rest.RestReporter.summary misses a docstring
test.rest.RestReporter.skips misses a docstring
test.rest.RestReporter.repr_traceback misses a docstring
test.rest.RestReporter.repr_source misses a docstring
test.rest.RestReporter.repr_signal misses a docstring
test.rest.RestReporter.repr_failure misses a docstring
test.rest.RestReporter.report_unknown misses a docstring
test.rest.RestReporter.report_TestStarted misses a docstring
test.rest.RestReporter.report_TestFinished misses a docstring
test.rest.RestReporter.report_SkippedTryiter misses a docstring
test.rest.RestReporter.report_SendItem misses a docstring
test.rest.RestReporter.report_RsyncFinished misses a docstring
test.rest.RestReporter.report_ReceivedItemOutcome misses a docstring
test.rest.RestReporter.report_Nodes misses a docstring
test.rest.RestReporter.report_ItemStart misses a docstring
test.rest.RestReporter.report_ImmediateFailure misses a docstring
test.rest.RestReporter.report_HostReady misses a docstring
test.rest.RestReporter.report_HostRSyncing misses a docstring
test.rest.RestReporter.report_FailedTryiter misses a docstring
test.rest.RestReporter.report misses a docstring
test.rest.RestReporter.print_summary misses a docstring
test.rest.RestReporter.prepare_source misses a docstring
test.rest.RestReporter.hangs misses a docstring
test.rest.RestReporter.get_rootpath misses a docstring
test.rest.RestReporter.get_path_from_item misses a docstring
test.rest.RestReporter.get_linkwriter misses a docstring
test.rest.RestReporter.get_item_name misses a docstring
test.rest.RestReporter.get_host misses a docstring
test.rest.RestReporter.failures misses a docstring
test.rest.RestReporter.add_rest misses a docstring
test.rest.RestReporter.__init__ misses a docstring
test.rest.RelLinkWriter misses a docstring
test.rest.RelLinkWriter.get_link misses a docstring
test.rest.NoLinkWriter misses a docstring
test.rest.NoLinkWriter.get_link misses a docstring
test.rest.LinkWriter misses a docstring
test.rest.LinkWriter.get_link misses a docstring
test.rest.LinkWriter.__init__ misses a docstring
test.exit misses a docstring
test.deprecated_call misses a docstring
test.collect.Module misses a docstring
test.collect.Module.tryiter has an 'XXX' in its docstring
test.collect.Module.teardown misses a docstring
test.collect.Module.startcapture misses a docstring
test.collect.Module.skipbykeyword misses a docstring
test.collect.Module.setup misses a docstring
test.collect.Module.run misses a docstring
test.collect.Module.makeitem misses a docstring
test.collect.Module.listnames misses a docstring
test.collect.Module.join misses a docstring
test.collect.Module.haskeyword misses a docstring
test.collect.Module.getsortvalue misses a docstring
test.collect.Module.getpathlineno misses a docstring
test.collect.Module.getouterr misses a docstring
test.collect.Module.getitembynames misses a docstring
test.collect.Module.funcnamefilter misses a docstring
test.collect.Module.finishcapture misses a docstring
test.collect.Module.classnamefilter misses a docstring
test.collect.Module.buildname2items misses a docstring
test.collect.Module.__repr__ misses a docstring
test.collect.Module.__ne__ misses a docstring
test.collect.Module.__init__ misses a docstring
test.collect.Module.__hash__ misses a docstring
test.collect.Module.__eq__ misses a docstring
test.collect.Module.__cmp__ misses a docstring
test.collect.Module.Skipped misses a docstring
test.collect.Module.Passed misses a docstring
test.collect.Module.Outcome misses a docstring
test.collect.Module.Failed misses a docstring
test.collect.Module.ExceptionFailure misses a docstring
test.collect.Instance misses a docstring
test.collect.Instance.tryiter has an 'XXX' in its docstring
test.collect.Instance.teardown misses a docstring
test.collect.Instance.startcapture misses a docstring
test.collect.Instance.skipbykeyword misses a docstring
test.collect.Instance.setup misses a docstring
test.collect.Instance.run misses a docstring
test.collect.Instance.makeitem misses a docstring
test.collect.Instance.listnames misses a docstring
test.collect.Instance.join misses a docstring
test.collect.Instance.haskeyword misses a docstring
test.collect.Instance.getsortvalue misses a docstring
test.collect.Instance.getpathlineno misses a docstring
test.collect.Instance.getouterr misses a docstring
test.collect.Instance.getitembynames misses a docstring
test.collect.Instance.funcnamefilter misses a docstring
test.collect.Instance.finishcapture misses a docstring
test.collect.Instance.classnamefilter misses a docstring
test.collect.Instance.buildname2items misses a docstring
test.collect.Instance.__repr__ misses a docstring
test.collect.Instance.__ne__ misses a docstring
test.collect.Instance.__init__ misses a docstring
test.collect.Instance.__hash__ misses a docstring
test.collect.Instance.__eq__ misses a docstring
test.collect.Instance.__cmp__ misses a docstring
test.collect.Generator misses a docstring
test.collect.Generator.tryiter has an 'XXX' in its docstring
test.collect.Generator.teardown misses a docstring
test.collect.Generator.startcapture misses a docstring
test.collect.Generator.skipbykeyword misses a docstring
test.collect.Generator.setup misses a docstring
test.collect.Generator.run misses a docstring
test.collect.Generator.listnames misses a docstring
test.collect.Generator.join misses a docstring
test.collect.Generator.haskeyword misses a docstring
test.collect.Generator.getsortvalue misses a docstring
test.collect.Generator.getpathlineno misses a docstring
test.collect.Generator.getouterr misses a docstring
test.collect.Generator.getitembynames misses a docstring
test.collect.Generator.getcallargs misses a docstring
test.collect.Generator.finishcapture misses a docstring
test.collect.Generator.buildname2items misses a docstring
test.collect.Generator.__repr__ misses a docstring
test.collect.Generator.__ne__ misses a docstring
test.collect.Generator.__init__ misses a docstring
test.collect.Generator.__hash__ misses a docstring
test.collect.Generator.__eq__ misses a docstring
test.collect.Generator.__cmp__ misses a docstring
test.collect.DoctestFile misses a docstring
test.collect.DoctestFile.tryiter has an 'XXX' in its docstring
test.collect.DoctestFile.teardown misses a docstring
test.collect.DoctestFile.startcapture misses a docstring
test.collect.DoctestFile.skipbykeyword misses a docstring
test.collect.DoctestFile.setup misses a docstring
test.collect.DoctestFile.run misses a docstring
test.collect.DoctestFile.makeitem misses a docstring
test.collect.DoctestFile.listnames misses a docstring
test.collect.DoctestFile.join misses a docstring
test.collect.DoctestFile.haskeyword misses a docstring
test.collect.DoctestFile.getsortvalue misses a docstring
test.collect.DoctestFile.getpathlineno misses a docstring
test.collect.DoctestFile.getouterr misses a docstring
test.collect.DoctestFile.getitembynames misses a docstring
test.collect.DoctestFile.funcnamefilter misses a docstring
test.collect.DoctestFile.finishcapture misses a docstring
test.collect.DoctestFile.classnamefilter misses a docstring
test.collect.DoctestFile.buildname2items misses a docstring
test.collect.DoctestFile.__repr__ misses a docstring
test.collect.DoctestFile.__ne__ misses a docstring
test.collect.DoctestFile.__init__ misses a docstring
test.collect.DoctestFile.__hash__ misses a docstring
test.collect.DoctestFile.__eq__ misses a docstring
test.collect.DoctestFile.__cmp__ misses a docstring
test.collect.Directory misses a docstring
test.collect.Directory.tryiter has an 'XXX' in its docstring
test.collect.Directory.teardown misses a docstring
test.collect.Directory.startcapture misses a docstring
test.collect.Directory.skipbykeyword misses a docstring
test.collect.Directory.setup misses a docstring
test.collect.Directory.run misses a docstring
test.collect.Directory.recfilter misses a docstring
test.collect.Directory.makeitem misses a docstring
test.collect.Directory.listnames misses a docstring
test.collect.Directory.join misses a docstring
test.collect.Directory.haskeyword misses a docstring
test.collect.Directory.getsortvalue misses a docstring
test.collect.Directory.getpathlineno misses a docstring
test.collect.Directory.getouterr misses a docstring
test.collect.Directory.getitembynames misses a docstring
test.collect.Directory.finishcapture misses a docstring
test.collect.Directory.filefilter misses a docstring
test.collect.Directory.buildname2items misses a docstring
test.collect.Directory.__repr__ misses a docstring
test.collect.Directory.__ne__ misses a docstring
test.collect.Directory.__init__ misses a docstring
test.collect.Directory.__hash__ misses a docstring
test.collect.Directory.__eq__ misses a docstring
test.collect.Directory.__cmp__ misses a docstring
test.collect.Collector misses a docstring
test.collect.Collector.tryiter has an 'XXX' in its docstring
test.collect.Collector.teardown misses a docstring
test.collect.Collector.startcapture misses a docstring
test.collect.Collector.skipbykeyword misses a docstring
test.collect.Collector.setup misses a docstring
test.collect.Collector.run misses a docstring
test.collect.Collector.listnames misses a docstring
test.collect.Collector.join misses a docstring
test.collect.Collector.haskeyword misses a docstring
test.collect.Collector.getsortvalue misses a docstring
test.collect.Collector.getpathlineno misses a docstring
test.collect.Collector.getouterr misses a docstring
test.collect.Collector.getitembynames misses a docstring
test.collect.Collector.finishcapture misses a docstring
test.collect.Collector.buildname2items misses a docstring
test.collect.Collector.__repr__ misses a docstring
test.collect.Collector.__ne__ misses a docstring
test.collect.Collector.__init__ misses a docstring
test.collect.Collector.__hash__ misses a docstring
test.collect.Collector.__eq__ misses a docstring
test.collect.Collector.__cmp__ misses a docstring
test.collect.Class misses a docstring
test.collect.Class.tryiter has an 'XXX' in its docstring
test.collect.Class.teardown misses a docstring
test.collect.Class.startcapture misses a docstring
test.collect.Class.skipbykeyword misses a docstring
test.collect.Class.setup misses a docstring
test.collect.Class.run misses a docstring
test.collect.Class.makeitem misses a docstring
test.collect.Class.listnames misses a docstring
test.collect.Class.join misses a docstring
test.collect.Class.haskeyword misses a docstring
test.collect.Class.getsortvalue misses a docstring
test.collect.Class.getpathlineno misses a docstring
test.collect.Class.getouterr misses a docstring
test.collect.Class.getitembynames misses a docstring
test.collect.Class.funcnamefilter misses a docstring
test.collect.Class.finishcapture misses a docstring
test.collect.Class.classnamefilter misses a docstring
test.collect.Class.buildname2items misses a docstring
test.collect.Class.__repr__ misses a docstring
test.collect.Class.__ne__ misses a docstring
test.collect.Class.__init__ misses a docstring
test.collect.Class.__hash__ misses a docstring
test.collect.Class.__eq__ misses a docstring
test.collect.Class.__cmp__ misses a docstring
test.cmdline.main misses a docstring
test.Item misses a docstring
test.Item.tryiter has an 'XXX' in its docstring
test.Item.teardown misses a docstring
test.Item.startcapture misses a docstring
test.Item.skipbykeyword misses a docstring
test.Item.setup misses a docstring
test.Item.run misses a docstring
test.Item.listnames misses a docstring
test.Item.join misses a docstring
test.Item.haskeyword misses a docstring
test.Item.getsortvalue misses a docstring
test.Item.getpathlineno misses a docstring
test.Item.getouterr misses a docstring
test.Item.getitembynames misses a docstring
test.Item.finishcapture misses a docstring
test.Item.buildname2items misses a docstring
test.Item.__repr__ misses a docstring
test.Item.__ne__ misses a docstring
test.Item.__init__ misses a docstring
test.Item.__hash__ misses a docstring
test.Item.__eq__ misses a docstring
test.Item.__cmp__ misses a docstring
test.Function.tryiter has an 'XXX' in its docstring
test.Function.teardown misses a docstring
test.Function.startcapture misses a docstring
test.Function.skipbykeyword misses a docstring
test.Function.setup misses a docstring
test.Function.run misses a docstring
test.Function.listnames misses a docstring
test.Function.join misses a docstring
test.Function.haskeyword misses a docstring
test.Function.getsortvalue misses a docstring
test.Function.getpathlineno misses a docstring
test.Function.getouterr misses a docstring
test.Function.getitembynames misses a docstring
test.Function.finishcapture misses a docstring
test.Function.buildname2items misses a docstring
test.Function.__repr__ misses a docstring
test.Function.__ne__ misses a docstring
test.Function.__init__ misses a docstring
test.Function.__hash__ misses a docstring
test.Function.__eq__ misses a docstring
test.Function.__cmp__ misses a docstring
test.Config.__init__ misses a docstring
xml.raw.__init__ misses a docstring
xml.html misses a docstring
xml.html.__tagclass__ misses a docstring
xml.html.__tagclass__.unicode misses a docstring
xml.html.__tagclass__.__unicode__ misses a docstring
xml.html.__tagclass__.__repr__ misses a docstring
xml.html.__tagclass__.__init__ misses a docstring
xml.html.__tagclass__.Attr misses a docstring
xml.html.__tagclass__.Attr.__init__ misses a docstring
xml.html.__metaclass__ misses a docstring
xml.html.__metaclass__.__getattr__ misses a docstring
xml.html.Style misses a docstring
xml.html.Style.__init__ misses a docstring
xml.escape misses a docstring
xml.Tag misses a docstring
xml.Tag.unicode misses a docstring
xml.Tag.__unicode__ misses a docstring
xml.Tag.__repr__ misses a docstring
xml.Tag.__init__ misses a docstring
xml.Namespace misses a docstring

View File

@ -1,640 +0,0 @@
===============================================================
py.code_template: Lightweight and flexible code template system
===============================================================
.. contents::
.. sectnum::
Motivation
==========
There are as many python templating systems as there are web frameworks
(a lot). This is partly because it is so darned easy to write a templating
system in Python. What are the distinguishing characteristics of the
py.code_template templating system?
* Optimized for generating code (Python, C, bash scripts, etc.),
not XML or HTML
* Designed for use by Python programmers, not by web artists
+ Aesthetic sensibilities are different
+ The templates should be an organic part of a module -- just more code
+ Templates do not need to be incredibly full-featured, because
programmers are perfectly capable of escaping to Python for
advanced features.
- No requirement to support inheritance
- No requirement to support exec
* Designed so that templates can be coded in the most natural way
for the task at hand
+ Generation of code and scripts often does not follow MVC paradigm!
+ Small template fragments are typically coded *inside* Python modules
+ Sometimes it is natural to put strings inside code; sometimes it is
natural to put code inside strings. Both should be supported as
reasonably and naturally as possible.
Imaginary-world examples
========================
These would be real-world examples, but, not only is this module not yet
implemented, as of now, PyPy is not incredibly useful to the average
programmer...
translator/c/genc.py
--------------------
The original function::
def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]):
#
# All declarations
#
structdeflist = database.getstructdeflist()
print >> f
print >> f, '/***********************************************************/'
print >> f, '/*** Structure definitions ***/'
print >> f
for node in structdeflist:
print >> f, 'struct %s;' % node.name
print >> f
for node in structdeflist:
for line in node.definition():
print >> f, line
print >> f
print >> f, '/***********************************************************/'
print >> f, '/*** Forward declarations ***/'
print >> f
for node in database.globalcontainers():
for line in node.forward_declaration():
print >> f, line
#
# Implementation of functions and global structures and arrays
#
print >> f
print >> f, '/***********************************************************/'
print >> f, '/*** Implementations ***/'
print >> f
for line in preimplementationlines:
print >> f, line
print >> f, '#include "src/g_include.h"'
print >> f
blank = True
for node in database.globalcontainers():
if blank:
print >> f
blank = False
for line in node.implementation():
print >> f, line
blank = True
This could be refactored heavily. An initial starting point
would look something like this, although later, the template
instance could be passed in and reused directly, rather than
passing the file handle around::
def gen_readable_parts_of_main_c_file(f, database, preimplementationlines=[]):
def container_implementation():
# Helper function designed to introduce blank lines
# between container implementations
blank = True
for node in database.globalcontainers():
if blank:
yield ''
blank = False
for line in node.implementation():
yield line
blank = True
t = code_template.Template()
#
# All declarations
#
structdeflist = database.getstructdeflist()
t.write(dedent=8, text='''
/***********************************************************/
/*** Structure definitions ***/
{for node in structdeflist}
struct {node.name};
{endfor}
{for node in structdeflist}
{for line in node.definition}
{line}
{endfor}
{endfor}
/***********************************************************/
/*** Forward declarations ***/
{for node in database.globalcontainers()}
{for line in node.forward_declaration()}
{line}
{endfor}
{endfor}
{**
** Implementation of functions and global structures and arrays
**}
/***********************************************************/
/*** Implementations ***/
{for line in preimplementationlines}
{line}
{endfor}
#include "src/g_include.h"
{for line in container_implementation()}
{line}
{endfor}
""")
t.output(f)
translator/c/genc.py gen_makefile
---------------------------------
The original code::
MAKEFILE = '''
CC = gcc
$(TARGET): $(OBJECTS)
\t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS)
%.o: %.c
\t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS)
clean:
\trm -f $(OBJECTS)
'''
def gen_makefile(self, targetdir):
def write_list(lst, prefix):
for i, fn in enumerate(lst):
print >> f, prefix, fn,
if i < len(lst)-1:
print >> f, '\\'
else:
print >> f
prefix = ' ' * len(prefix)
compiler = self.getccompiler(extra_includes=['.'])
cfiles = []
ofiles = []
for fn in compiler.cfilenames:
fn = py.path.local(fn).basename
assert fn.endswith('.c')
cfiles.append(fn)
ofiles.append(fn[:-2] + '.o')
f = targetdir.join('Makefile').open('w')
print >> f, '# automatically generated Makefile'
print >> f
print >> f, 'TARGET =', py.path.local(compiler.outputfilename).basename
print >> f
write_list(cfiles, 'SOURCES =')
print >> f
write_list(ofiles, 'OBJECTS =')
print >> f
args = ['-l'+libname for libname in compiler.libraries]
print >> f, 'LIBS =', ' '.join(args)
args = ['-L'+path for path in compiler.library_dirs]
print >> f, 'LIBDIRS =', ' '.join(args)
args = ['-I'+path for path in compiler.include_dirs]
write_list(args, 'INCLUDEDIRS =')
print >> f
print >> f, 'CFLAGS =', ' '.join(compiler.compile_extra)
print >> f, 'LDFLAGS =', ' '.join(compiler.link_extra)
print >> f, MAKEFILE.strip()
f.close()
Could look something like this::
MAKEFILE = '''
# automatically generated Makefile
TARGET = {py.path.local(compiler.outputfilename).basename}
{for line in write_list(cfiles, 'SOURCES =')}
{line}
{endfor}
{for line in write_list(ofiles, 'OBJECTS =')}
{line}
{endfor}
LIBS ={for libname in compiler.libraries} -l{libname}{endfor}
LIBDIRS ={for path in compiler.library_dirs} -L{path}{endfor}
INCLUDEDIRS ={for path in compiler.include_dirs} -I{path}{endfor}
CFLAGS ={for extra in compiler.compile_extra} {extra}{endfor}
LDFLAGS ={for extra in compiler.link_extra} {extra}{endfor}
CC = gcc
$(TARGET): $(OBJECTS)
\t$(CC) $(LDFLAGS) -o $@ $(OBJECTS) $(LIBDIRS) $(LIBS)
%.o: %.c
\t$(CC) $(CFLAGS) -o $@ -c $< $(INCLUDEDIRS)
clean:
\trm -f $(OBJECTS)
'''
def gen_makefile(self, targetdir):
def write_list(lst, prefix):
for i, fn in enumerate(lst):
yield '%s %s %s' % (prefix, fn, i < len(list)-1 and '\\' or '')
prefix = ' ' * len(prefix)
compiler = self.getccompiler(extra_includes=['.'])
cfiles = []
ofiles = []
for fn in compiler.cfilenames:
fn = py.path.local(fn).basename
assert fn.endswith('.c')
cfiles.append(fn)
ofiles.append(fn[:-2] + '.o')
code_template.Template(MAKEFILE).output(targetdir.join('Makefile'))
translator/llvm/module/excsupport.py
------------------------------------
The original string::
invokeunwind_code = '''
ccc %(returntype)s%%__entrypoint__%(entrypointname)s {
%%result = invoke %(cconv)s %(returntype)s%%%(entrypointname)s to label %%no_exception except label %%exception
no_exception:
store %%RPYTHON_EXCEPTION_VTABLE* null, %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type
ret %(returntype)s %%result
exception:
ret %(noresult)s
}
ccc int %%__entrypoint__raised_LLVMException() {
%%tmp = load %%RPYTHON_EXCEPTION_VTABLE** %%last_exception_type
%%result = cast %%RPYTHON_EXCEPTION_VTABLE* %%tmp to int
ret int %%result
}
internal fastcc void %%unwind() {
unwind
}
'''
Could look something like this if it was used in conjunction with a template::
invokeunwind_code = '''
ccc {returntype}%__entrypoint__{entrypointname} {
%result = invoke {cconv} {returntype}%{entrypointname} to label %no_exception except label %exception
no_exception:
store %RPYTHON_EXCEPTION_VTABLE* null, %RPYTHON_EXCEPTION_VTABLE** %last_exception_type
ret {returntype} %result
exception:
ret {noresult}
}
ccc int %__entrypoint__raised_LLVMException() {
%tmp = load %RPYTHON_EXCEPTION_VTABLE** %last_exception_type
%result = cast %RPYTHON_EXCEPTION_VTABLE* %tmp to int
ret int %result
}
internal fastcc void %unwind() {
unwind
}
'''
Template syntax
===============
Design decision
---------------
As all programmers must know by now, all the special symbols on the keyboard
are quite heavily overloaded. Often, template systems work around this fact
by having special notation like `<*` ... `*>` or {% ... %}. Some template systems
even have multiple special notations -- one for comments, one for statements,
one for expressions, etc.
I find these hard to type and ugly. Other markups are either too lightweight,
or use characters which occur so frequently in the target languages that it
becomes hard to distinguish marked-up content from content which should be
rendered as-is.
The compromise taken by *code_template* is to use braces (**{}**) for markup.
This immediately raises the question: what about when the marked-up language
is C or C++? The answer is that if the leading brace is immediately followed
by whitespace, it is normal text; if not it is the start of markup.
To support normal text which has a leading brace immediately followed by
an identifier, if the first whitespace character after the brace is a space
character (e.g. not a newline or tab), it will be removed from the output.
Examples::
{ This is normal text and the space between { and This will be removed}
{'this must be a valid Python expression' + ' because it is treated as markup'}
{
This is normal text, but nothing is altered (the newline is kept intact)
}
{{1:'Any valid Python expression is allowed as markup'}[1].ljust(30)}
.. _`Code element`:
Elements
--------
Templates consist of normal text and code elements.
(Comments are considered to be code elements.)
All code elements start with a `left brace`_ which is not followed by
whitespace.
Keyword element
~~~~~~~~~~~~~~~
A keyword element is a `code element`_ which starts with a keyword_.
For example, *{if foo}* is a keyword element, but *{foo}* is a `substituted expression`_.
Keyword
~~~~~~~
A keyword is a word used in `conditional text`_ or in `repeated text`_, e.g.
one of *if*, *elif*, *else*, *endif*, *for*, or *endfor*.
Keywords are designed to match their Python equivalents. However, since
templates cannot use spacing to indicate expression nesting, the additional
keywords *endif* and *endfor* are required.
Left brace
~~~~~~~~~~
All elements other than normal text start with a left brace -- the symbol '{',
sometimes known as a 'curly bracket'. A left brace is itself considered
to be normal text if it is followed by whitespace. If the whitespace starts
with a space character, that space character will be stripped from the output.
If the whitespace starts with a tab or linefeed character, the whitespace will
be left in the output.
Normal Text
~~~~~~~~~~~
Normal text remains unsubstituted. Transition from text to the other elements
is effected by use of a `left brace`_ which is not followed by whitespace.
Comment
~~~~~~~
A comment starts with a left brace followed by an asterisk ('{`*`'), and
ends with an asterisk followed by a right brace ('`*`}')::
This is a template -- this text will be copied to the output.
{* This is a comment and this text will not be copied to the output *}
{*
Comments can span lines,
but cannot be nested
*}
Substituted expression
~~~~~~~~~~~~~~~~~~~~~~
Any python expression may be used::
Dear {record.name},
we are sorry to inform you that you did not win {record.contest}.
The expression must be surrounded by braces, and there must not be any
whitespace between the leftmost brace and the start of the expression.
The expression will automatically be converted to a string with str().
Conditional text
~~~~~~~~~~~~~~~~
The following template has text which is included conditionally::
This text will always be included in the output
{if foo}
This text will be included if foo is true
{elif bar}
This text will be included if foo is not true but bar is true
{else}
This text will be included if neither foo nor bar is true
{endif}
The {elif} and {else} elements are optional.
Repeated text
~~~~~~~~~~~~~
The following template shows how to pull multiple items out of a list::
{for student, score in sorted(scorelist)}
{student.ljust(20)} {score}
{endfor}
Whitespace removal or modification
----------------------------------
In general, whitespace in `Normal Text`_ is transferred unchanged to the
output. There are three exceptions to this rule:
Line separators
~~~~~~~~~~~~~~~
Each newline is converted to the final output using os.linesep.
Beginning or end of string
~~~~~~~~~~~~~~~~~~~~~~~~~~
py.code_template is designed to allow easy use of templates inside of python
modules. The canonical way to write a template is inside a triple-quoted
string, e.g.::
my_template = '''
This is my template. It can have any text at all in it except
another triple-single-quote.
'''
To support this usage, if the first character is a newline, it will be
removed, and if the last line consists solely of whitespace with no
trailing newline, it will also be removed.
A comment or single keyword element on a line
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Whenever a `keyword element`_ or comment_ is on a line
*by itself*, that line will not be copied to the output.
This happens when:
- There is nothing on the line before the keyword element
or comment except whitespace (spaces and/or tabs).
- There is nothing on the line after the keyword element
or comment except a newline.
Note that even a multi-line comment or keyword element can
have the preceding whitespace and subsequent newline stripped
by this rule.
The primary purpose of this rule is to allow the Python
programmer to use indentation, **even inside a template**::
This is a template
{if mylist}
List items:
{for item in mylist}
- {item}
{endfor}
{endif}
Template usage
==============
Templates are used by importing the Template class from py.code_template,
constructing a template, and then sending data with the write() method.
In general, there are four methods for getting the formatted data back out
of the template object:
- read() reads all the data currently in the object
- output(fobj) outputs the data to a file
fobj can either be an open file object, or a string. If it is
a string, the file will be opened, written, and closed.
- open(fobj) (or calling the object constructor with a file object)
If the open() method is used, or if a file object is passed to
the constructor, each write() will automatically flush the data
out to the file. If the fobj is a string, it is considered to
be *owned*, otherwise it is considered to be *borrowed*. *Owned*
file objects are closed when the class is deleted.
- write() can be explicitly called with a file object, in which case
it will invoke output() on that object after it generates the data.
Template instantiation and methods
==================================
template = code_template.Template(outf=None, cache=None)
If outf is given, it will be passed to the open() method
cache may be given as a mapping. If not given, the template will use
the shared default cache. This is not thread safe.
template.open
-------------
template.open(outf, borrowed = None)
The open method closes the internal file object if it was already open,
and then re-opens it on the given file. It is an error to call open()
if there is data in the object left over from previous writes. (Call
output() instead.)
borrowed defaults to 0 if outf is a string, and 1 if it is a file object.
borrowed can also be set explicitly if required.
template.close
--------------
close() disassociates the file from the template, and closes the file if
it was not borrowed. close() is automatically called by the destructor.
template.write
--------------
template.write(text='', outf=None, dedent=0, localvars=None, globalvars=None,
framelevel=1)
The write method has the following parameters:
- text is the template itself
- if outf is not None, the output method will be invoked on the object
after the current template is processed. If no outf is given, data
will be accumulated internal to the instance until a write() with outf
is processed, or read() or output() is called, whichever comes first, if
there is no file object. If there is a file object, data will be flushed
to the file after every write.
- dedent, if given is applied to each line in the template, to "de-indent"
- localvars and globalvars default to the dictionaries of the caller. A copy
of localvars is made so that the __TrueSpace__ identifier can be added.
- cache may be given as a mapping. If not given, the template will use
the shared default cache. This is not thread safe.
- framelevel is used to determine which stackframe to access for globals
and locals if localvars and/or globalvars are not specified. The default
is to use the caller's frame.
The write method supports the print >> file protocol by deleting the softspace
attribute on every invocation. This allows code like::
t = code_template.Template()
print >> t, "Hello, world"
template.read
--------------
This method reads and flushes all accumulated data in the object. Note that
if a file has been associated with the object, there will never be any data
to read.
template.output
---------------
This method takes one parameter, outf. template.output() first
invokes template.read() to read and flush all accumulated data,
and then outputs the data to the file specified by outf.
If outf has a write() method, that will be invoked with the
data. If outf has no write() method, it will be treated as
a filename, and that file will be replaced.
Caching and thread safety
=========================
The compiled version of every template is cached internal to the
code_template module (unless a separate cache object is specified).
This allows efficient template reuse, but is not currently thread-safe.
Alternatively, each invocation of a template object can specify a
cache object. This is thread-safe, but not very efficient. A shared
model could be implemented later.

View File

@ -1,64 +0,0 @@
Release
=======
currently working configurations
--------------------------------
2.3 - 2.4.2 work
with setuptools: 2.3 - 2.4.2 as 'develop'
regular installation: works mostly, strange test-failures
to be tested: 2.2, windows
absolutely necessary steps:
----------------------------
* documentation
* improving getting started, describe install methods
* describe the rest stuff?
* py.log
* py.path is mostly undocumented, API documentation
* basic windows testing, maybe disabling execnet?, what about the scripts in windows?
* are all c extensions compiled when installing globally?
* refactoring py.log
* write/read methods on py.path should be renamed/deprecated: setcontent, getcontent instead?
* what about _subprocess.c?
* warning for docutils
* don't expose _extpy
* py/bin should be nicefied, get optparse interface
* _findpy.py
* py.cleanup:
* py.lookup: add -i option
* pytest.cmd
* rst2pdf.py: merge with py.rest, add warnings when missing tex
* _makepyrelease.py: move somewhere
* py.countloc
* py.test
* py.rest
* win32
* skip tests if dependencies are not installed
nice to have
------------
* sets.py, subprocess.py in compat
* fix -k option to py.test
* add --report=(text|terminal|session|rest|tkinter|rest) to py.test
* put Armin's collect class into py.__builtin__ (not done)
* try get rid of Collect.tryiter() in favour of (not done)
using Armin's collect class

View File

@ -1,39 +0,0 @@
Missing features/bugs in pylib:
====================================
* new skip method, so web interface would show skips which
are broken (say py.test.skip("stuff", reason=py.test.BORKEN)),
proposed by me and xoraxax
* integration of session classes - needed for developement
* more robust failure recovery from execnet - not sure how to perform
it, but select() approach sounds like a plan (instead of threads)
(unsure what than)
* provide a bit more patchy approach to green stuff, ie you import it and
all (known) operations on sockets are performed via the green interface,
this should allow using arbitrary applications (well, not using C-level
I/O) to mix with green threads.
* integrate green execnet a bit more (?)
* once session integration is done, it would be cool to have nightly
testing done in a systematic manner (instead of bunch of hacks, which
is how it looks like right now), so for example codespeak would be able
to store information (ie via svn) and when one woke up he can type py.test
show and see the information of all nightly test runs which he likes.
* py.test.pdb - there is my hack for a while now, which integrates
rlcompleter2 with pdb. First of all it requires some strange changes
to rlcompleter itself, which has no tests. Long-term plan would be
to have pyrepl+rlcompleter2+pdb fixes integrated into pylib and
have it tested. This requires work though.
* add a link to pylib in pypy/lib? Since pylib features mostly work on top
of pypy-c, it would be nice to have it (as we do have it in svn anyway)
* fix generative tests.
- they should be distributed atomically (for various reasons)
- fix setup/teardown logic (ie setup_generator/teardown_generator)
- XXX there was sth else

View File

@ -1,37 +0,0 @@
Here I'm trying to list all problems regarding pypy-c <-> pylib interaction
===========================================================================
* in test/terminal/terminal.py lines around 141::
rev = py.__package__.getrev()
self.out.line("using py lib: %s <rev %s>" % (
py.path.local(py.__file__).dirpath(), rev))
* py.code issues::
def __init__(self, rawcode):
rawcode = getattr(rawcode, 'im_func', rawcode)
rawcode = getattr(rawcode, 'func_code', rawcode)
self.raw = rawcode
self.filename = rawcode.co_filename
AttributeError: 'internal-code' object has no attribute 'co_filename'
* types.BuiltinFunctionType == types.MethodType which confuses apigen
* compiler module problems - some bogus IndentationError
communicates by inspect.getsource()
* execnet just hangs
* lack of tmpfile
* assertion error magic is not working
* sha counting hangs (misc/testing/test\_initpkg)
* extpy does not work, because it does not support loops in modules
(while pypy __builtins__ module has a loop), funny :-)
* py.compat.subprocess hangs for obscure reasons
(possibly the same stuff as execnet - some threading issues and
select.select)
Armin says: "haha, select.select probably does not release the GIL"

View File

@ -1,15 +0,0 @@
Various tasks which needs to be done at some point
==================================================
* Write down pinging interface, so we'll know if hosts are responding or
are mostly down (detecting hanging nodes)
* Write down support for rsync progress
* Discovery of nodes which are available for accepting distributed testing
* Test the tests rescheduling, so high-latency nodes would not take part
in that.
* make sure that C-c semantics are ok (nodes are killed properly).
There was an attempt to do so, but it's not tested and not always work.