p53209761
/
test_ok2
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
test_ok2/py/doc/apigen.txt

289 lines
11 KiB
Plaintext
Raw Blame History

===========================================
apigen - API documentation generation tool
===========================================
What is it?
===========
Apigen is a tool for automatically generating API reference documentation for
Python projects. It works by examining code at runtime rather than at compile
time. This way it is capable of displaying information about the code base
after initialization. A drawback is that you cannot easily document source code
that automatically starts server processes or has some other irreversible
effects upon getting imported.
The apigen functionality is normally triggered from :api:`py.test`, and while
running the tests it gathers information such as code paths, arguments and
return values of callables, and exceptions that can be raised while the code
runs (XXX not yet!) to include in the documentation. It's also possible to
run the tracer (which collects the data) in other code if your project
does not use :api:`py.test` but still wants to collect the runtime information
and build the docs.
Apigen is written for the :api:`py` lib, but can be used to build documentation
for any project: there are hooks in py.test to, by providing a simple script,
build api documentation for the tested project when running py.test. Of course
this does imply :api:`py.test` is actually used: if little or no tests are
actually ran, the additional information (code paths, arguments and return
values and exceptions) can not be gathered and thus there will be less of an
advantage of apigen compared to other solutions.
Features
========
Some features were mentioned above already, but here's a complete list of all
the niceties apigen has to offer:
* source documents
Apigen not only builds the API documentation, but also a tree of
syntax-colored source files, with links from the API docs to the source
files.
* abundance of information
compared to other documentation generation tools, apigen produces an
abundant amount of information: it provides syntax-colored code snippets,
code path traces, etc.
* linking
besides links to the source files, apigen provides links all across the
documentation: callable arguments and return values link to their
definition (if part of the documented code), class definition to their
base classes (again, if they're part of the documented code), and
everywhere are links to the source files (including in traces)
* (hopefully) improves testing
because the documentation is built partially from test results, developers
may (especially if they're using the documentation themselves) be more
aware of untested parts of the code, or parts can use more tests or need
attention
Using apigen
============
To trigger apigen, all you need to do is run the :source:`py/bin/py.test` tool
with an --apigen argument, as such::
$ py.test --apigen=<path>
where <path> is a path to a script containing some special hooks to build
the documents (see below). The script to build the documents for the :api:`py`
lib can be found in :source:`py/apigen/apigen.py`, so building those documents
can be done by cd'ing to the 'py' directory, and executing::
$ py.test --apigen=apigen/apigen.py
The documents will by default be built in the *parent directory* of the
*package dir* (in this case the 'py' directory). Be careful that you don't
overwrite anything!
Other projects
==============
To use apigen from another project, there are three things that you need to do:
Use :api:`py.test` for unit tests
---------------------------------
This is a good idea anyway... ;) The more tests, the more tracing information
and such can be built, so it makes sense to have good test coverage when using
this tool.
Provide :api:`py.test` hooks
----------------------------
To hook into the unit testing framework, you will need to write a script with
two functions. The first should be called 'get_documentable_items', gets a
package dir (the root of the project) as argument, and should return a tuple
with the package name as first element, and a dict as second. The dict should
contain, for all the to-be-documented items, a dotted name as key and a
reference to the item as value.
The second function should be called 'build', and gets also the package dir as
argument, but also a reference to a DocStorageAcessor, which contains
information gathered by the tracer, and a reference to a
:api:`py.io.StdCaptureFD` instance that is used to capture stdout and stderr,
and allows writing to them, when the docs are built.
This 'build' function is responsible for actually building the documentation,
and, depending on your needs, can be used to control each aspect of it. In most
situations you will just copy the code from :source:`py/apigen/apigen.py`'s
build() function, but if you want you can choose to build entirely different
output formats by directly accessing the DocStorageAccessor class.
Provide layout
--------------
For the :api:`py` lib tests, the 'LayoutPage' class found in
:source:`py/apigen/layout.py` is used, which produces HTML specific for that
particular library (with a menubar, etc.). To customize this, you will need to
provide a similar class, most probably using the Page base class from
:source:`py/doc/confrest.py`. Note that this step depends on how heavy the
customization in the previous step is done: if you decide to directly use the
DocStorageAccessor rather than let the code in :source:`py/apigen/htmlgen.py`
build HTML for you, this can be skipped.
Using apigen from code
======================
If you want to avoid using :api:`py.test`, or have an other idea of how to best
collect information while running code, the apigen functionality can be
directly accessed. The most important classes are the Tracer class found in
:source:`py/apigen/tracer/tracer.py`, which holds the information gathered
during the tests, and the DocStorage and DocStorageAccessor classes from
:source:`py/apigen/tracer/docstorage.py`, which (respectively) store the data,
and make it accessible.
Gathering information
---------------------
To gather information about documentation, you will first need to tell the tool
what objects it should investigate. Only information for registered objects
will be stored. An example::
>>> import py
>>> from py.__.apigen.tracer.docstorage import DocStorage, DocStorageAccessor
>>> from py.__.apigen.tracer.tracer import Tracer
>>> toregister = {'py.path.local': py.path.local,
... 'py.path.svnwc': py.path.svnwc}
>>> ds = DocStorage().from_dict(toregister)
>>> t = Tracer(ds)
>>> t.start_tracing()
>>> p = py.path.local('.')
>>> p.check(dir=True)
True
>>> t.end_tracing()
Now the 'ds' variable should contain all kinds of information about both the
:api:`py.path.local` and the :api:`py.path.svnwc` classes, and things like call
stacks, possible argument types, etc. as additional information about
:api:`py.path.local.check()` (since it was called from the traced code).
Using the information
---------------------
To use the information, we need to get a DocStorageAccessor instance to
provide access to the data stored in the DocStorage object::
>>> dsa = DocStorageAccessor(ds)
Currently there is no API reference available for this object, so you'll have
to read the source (:source:`py/apigen/tracer/docstorage.py`) to see what
functionality it offers.
Comparison with other documentation generation tools
====================================================
Apigen is of course not the only documentation generation tool available for
Python. Although we knew in advance that our tool had certain features the
others do not offer, we decided to investigate a bit so that we could do a
proper comparison.
Tools examined
--------------
After some 'googling around', it turned out that the amount of documentation
generation tools available was surprisingly low. There were only 5 packages
I could find, of which 1 (called 'HappyDoc') seems dead (last release 2001),
one (called 'Pudge') not yet born (perhaps DOA even? most of the links on the
website are dead), and one (called 'Endo') specific to the Enthought suite.
The remaining two were Epydoc, which is widely used [1]_, and PyDoctor, which is
used only by (and written for) the Twisted project, but can be used seperately.
Epydoc
~~~~~~
http://epydoc.sourceforge.net/
Epydoc is the best known, and most widely used, documentation generation tool
for Python. It builds a documentation tree by inspecting imported modules and
using Python's introspection features. This way it can display information like
containment, inheritance, and docstrings.
The tool is relatively sophisticated, with support for generating HTML and PDF,
choosing different styles (CSS), generating graphs using Graphviz, etc. Also
it allows using markup (which can be ReST, JavaDoc, or their own 'epytext'
format) inside docstrings for displaying rich text in the result.
Quick overview:
* builds docs from object tree
* displays relatively little information, just inheritance trees, API and
docstrings
* supports some markup (ReST, 'epytext', JavaDoc) in docstrings
PyDoctor
~~~~~~~~
http://codespeak.net/~mwh/pydoctor/
This tool is written by Michael Hudson for the Twisted project. The major
difference between this and Epydoc is that it browses the AST (Abstract Syntax
Tree) instead of using 'live' objects, which means that code that uses special
import mechanisms, or depends on other code that is not available can still be
inspected. On the other hand, code that, for example, puts bound methods into a
module namespace is not documented.
The tool is relatively simple and doesn't support the more advanced features
that Epydoc offers. It was written for Twisted and there are no current plans to
promote its use for unrelated projects.
Quick overview:
* inspects AST rather than object tree
* again not a lot of information, the usual API docstrings, class inheritance
and module structure, but that's it
* rather heavy dependencies (depends on Twisted/Nevow (trunk version))
* written for Twisted, but quite nice output with other applications
Quick overview lists of the other tools
---------------------------------------
HappyDoc
~~~~~~~~
http://happydoc.sourceforge.net/
* dead
* inspects AST
* quite flexible, different output formats (HTML, XML, SGML, PDF)
* pluggable docstring parsers
Pudge
~~~~~
http://pudge.lesscode.org/
* immature, dead?
* builds docs from live object tree (I think?)
* supports ReST
* uses Kid templates
Endo
~~~~
https://svn.enthought.com/enthought/wiki/EndoHowTo
* inspects object tree (I think?)
* 'traits' aware (see https://svn.enthought.com/enthought/wiki/Traits)
* customizable HTML output with custom templating engine
* little documentation, seems like it's written for Enthought's own use
mostly
* heavy dependencies
.. [1] Epydoc doesn't seem to be developed anymore, either, but it's so
widely used it can not be ignored...
Questions, remarks, etc.
========================
For more information, questions, remarks, etc. see http://codespeak.net/py.
This website also contains links to mailing list and IRC channel.
.. _`initpkg`: http://codespeak.net/svn/py/dist/py/initpkg.py
.. _`py.test documentation`: http://codespeak.net/svn/py/dist/py/documentation/test.txt
Reference in New Issue View Git Blame Copy Permalink