[svn r63536] refining docs some more. removing outdated or what-i-consider-unfitting docs

--HG--
branch : trunk
This commit is contained in:
hpk 2009-04-02 15:12:26 +02:00
parent 984201a9e4
commit 5bc2bda8d1
6 changed files with 76 additions and 511 deletions

View File

@ -92,7 +92,7 @@ pygments_style = 'sphinx'
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
html_theme = 'basic'
html_theme = 'default'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
@ -148,7 +148,7 @@ html_static_path = ['_static']
#html_split_index = False
# If true, links to the reST sources are added to the pages.
#html_show_sourcelink = True
html_show_sourcelink = False
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the

View File

@ -1,145 +0,0 @@
=======================================================
Visions and ideas for further development of the py lib
=======================================================
This document tries to describe directions and guiding ideas
for the near-future development of the py lib. *Note that all
statements within this document - even if they sound factual -
mostly just express thoughts and ideas. They not always refer to
real code so read with some caution.*
Distribute tests ad-hoc across multiple platforms
======================================================
After some more refactoring and unification of
the current testing and distribution support code
we'd like to be able to run tests on multiple
platforms simultanously and allow for interaction
and introspection into the (remote) failures.
Make APIGEN useful for more projects
================================================
The new APIGEN tool offers rich information
derived from running tests against an application:
argument types and callsites, i.e. it shows
the places where a particular API is used.
In its first incarnation, there are still
some specialties that likely prevent it
from documenting APIs for other projects.
We'd like to evolve to a `py.apigen` tool
that can make use of information provided
by a py.test run.
Consider APIGEN and pdb integration
===================================
The information provided by APIGEN can be used in many
different ways. An example of this could be to write
an extension to pdb which makes it available.
Imagine you could issue a pdb command
"info <function name>" and get information
regarding incoming, and outgoing types, possible
exceptions, field types and call sites.
Distribute channels/programs across networks
================================================
Apart from stabilizing setup/teardown procedures
for `py.execnet`_, we'd like to generalize its
implementation to allow connecting two programs
across multiple hosts, i.e. we'd like to arbitrarily
send "channels" across the network. Likely this
will be done by using the "pipe" model, i.e.
that each channel is actually a pair of endpoints,
both of which can be independently transported
across the network. The programs who "own"
these endpoints remain connected.
.. _`py.execnet`: execnet.html
Benchmarking and persistent storage
=========================================
For storing test results, but also benchmarking
and other information, we need a solid way
to store all kinds of information from test runs.
We'd like to generate statistics or html-overview
out of it, but also use such information to determine when
a certain test broke, or when its performance
decreased considerably.
.. _`restructured text`: http://docutils.sourceforge.net/docs/user/rst/quickref.html
.. _`python standard library`: http://www.python.org/doc/2.3.4/lib/lib.html
.. _`xpython EuroPython 2004 talk`: http://codespeak.net/svn/user/hpk/talks/xpython-talk.txt
.. _`under the xpy tree`: http://codespeak.net/svn/user/hpk/xpy/xml.py
.. _`future book`: future.html
.. _`PEP-324 subprocess module`: http://www.python.org/peps/pep-0324.html
.. _`subprocess implementation`: http://www.lysator.liu.se/~astrand/popen5/
.. _`py.test`: test.html
.. _`general-path`:
.. _`a more general view on path objects`:
Refactor path implementations to use a Filesystem Abstraction
=============================================================
It seems like a good idea to refactor all `py.path`_ Path implementations to
use an internal Filesystem abstraction. The current code base
would be transformed to have Filesystem implementations for e.g.
local, subversion and subversion "working copy" filesystems. Today
the according code is scattered through path-handling code.
On a related note, Armin Rigo has hacked `pylufs`_ and more recently has
written `pyfuse`_ which allow to
implement kernel-level linux filesystems with pure python. Now
the idea is that the mentioned filesystem implementations would
be directly usable for such linux-filesystem glue code.
In other words, implementing a `memoryfs`_ or a `dictfs`_ would
give you two things for free: a filesystem mountable at kernel level
as well as a uniform "path" object allowing you to access your
filesystem in convenient ways.
Also interesting to check out is Will McGugan's work on
his `fs package`_.
I think the main question is what the very fundamental
filesystem API should look like. Here are some doctests
on how a `draft py.fs`_ could look like. There also
is Matthew Scotts `dictproxy patch`_ which adds
``py.path.dict`` and ``py.path.proxy``.
.. _`dictproxy patch`: http://codespeak.net/pipermail/py-dev/attachments/20050128/d9595512/virtual1-0001.bin
.. _`draft py.fs`: draft_pyfs
.. _`py.path`: http://codespeak.net/py/dist/path.html
.. _`fs package`: http://www.willmcgugan.com/2008/09/21/announcing-fs-010-a-python-file-system/#comment-60276
.. _`memoryfs`: http://codespeak.net/svn/user/arigo/hack/pyfuse/memoryfs.py
.. _`dictfs`: http://codespeak.net/pipermail/py-dev/2005-January/000191.html
.. _`pylufs`: http://codespeak.net/svn/user/arigo/hack/pylufs/
.. _`pyfuse`: http://codespeak.net/svn/user/arigo/hack/pyfuse/
Integrate interactive completion
==================================
It'd be nice to integrate the bash-like
rlcompleter2_ python command line completer
into the py lib, and making it work remotely
and with pdb.
.. _rlcompleter2: http://codespeak.net/rlcompleter2/
Consider more features
==================================
There are many more features and useful classes
that might be nice to integrate. For example, we might put
Armin's `lazy list`_ implementation into the py lib.
.. _`lazy list`: http://codespeak.net/svn/user/arigo/hack/misc/collect.py

View File

@ -1,113 +1,9 @@
===============================================
Implementation and Customization of ``py.test``
ATTIC documentation
===============================================
.. _`basicpicture`:
Collecting and running tests / implementation remarks
======================================================
In order to customize ``py.test`` it's good to understand
its basic architure (WARNING: these are not guaranteed
yet to stay the way they are now!)::
___________________
| |
| Collector |
|___________________|
/ \
| Item.run()
| ^
receive test Items /
| /execute test Item
| /
___________________/
| |
| Session |
|___________________|
.............................
. conftest.py configuration .
. cmdline options .
.............................
The *Session* basically receives test *Items* from a *Collector*,
and executes them via the ``Item.run()`` method. It monitors
the outcome of the test and reports about failures and successes.
.. _`collection process`:
Collectors and the test collection process
------------------------------------------
The collecting process is iterative, i.e. the session
traverses and generates a *collector tree*. Here is an example of such
a tree, generated with the command ``py.test --collectonly py/xmlobj``::
<Directory 'xmlobj'>
<Directory 'testing'>
<Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
<Function 'test_html_name_stickyness'>
<Function 'test_stylenames'>
<Function 'test_class_None'>
<Function 'test_alternating_style'>
<Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
<Function 'test_tag_with_text'>
<Function 'test_class_identity'>
<Function 'test_tag_with_text_and_attributes'>
<Function 'test_tag_with_subclassed_attr_simple'>
<Function 'test_tag_nested'>
<Function 'test_tag_xmlname'>
By default all directories not starting with a dot are traversed,
looking for ``test_*.py`` and ``*_test.py`` files. Those files
are imported under their `package name`_.
The Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed ``test`` by default. Test classes must
start with a capitalized ``Test`` prefix.
.. _`collector API`:
test items are collectors as well
---------------------------------
To make the reporting life simple for the session object
items offer a ``run()`` method as well. In fact the session
distinguishes "collectors" from "items" solely by interpreting
their return value. If it is a list, then we recurse into
it, otherwise we consider the "test" as passed.
.. _`package name`:
constructing the package name for test modules
-------------------------------------------------
Test modules are imported under their fully qualified
name. Given a filesystem ``fspath`` it is constructed as follows:
* walk the directories up to the last one that contains
an ``__init__.py`` file.
* perform ``sys.path.insert(0, basedir)``.
* import the root package as ``root``
* determine the fully qualified name for ``fspath`` by either:
* calling ``root.__pkg__.getimportname(fspath)`` if the
``__pkg__`` exists.` or
* otherwise use the relative path of the module path to
the base dir and turn slashes into dots and strike
the trailing ``.py``.
XXX REVIEW and remove the below XXX
Customizing the testing process
@ -140,79 +36,10 @@ and modules the default collectors will produce
custom Collectors and Items if they are found
in a local ``conftest.py`` file.
example: perform additional ReST checks
.......................................
With your custom collectors or items you can completely
derive from the standard way of collecting and running
tests in a localized manner. Let's look at an example.
If you invoke ``py.test --collectonly py/documentation``
then you get::
<DocDirectory 'documentation'>
<DocDirectory 'example'>
<DocDirectory 'pytest'>
<Module 'test_setup_flow_example.py' (test_setup_flow_example)>
<Class 'TestStateFullThing'>
<Instance '()'>
<Function 'test_42'>
<Function 'test_23'>
<ReSTChecker 'TODO.txt'>
<ReSTSyntaxTest 'TODO.txt'>
<LinkCheckerMaker 'checklinks'>
<ReSTChecker 'api.txt'>
<ReSTSyntaxTest 'api.txt'>
<LinkCheckerMaker 'checklinks'>
<CheckLink 'getting-started.html'>
...
In ``py/documentation/conftest.py`` you find the following
customization::
class DocDirectory(py.test.collect.Directory):
def run(self):
results = super(DocDirectory, self).run()
for x in self.fspath.listdir('*.txt', sort=True):
results.append(x.basename)
return results
def join(self, name):
if not name.endswith('.txt'):
return super(DocDirectory, self).join(name)
p = self.fspath.join(name)
if p.check(file=1):
return ReSTChecker(p, parent=self)
Directory = DocDirectory
The existence of the 'Directory' name in the
``pypy/documentation/conftest.py`` module makes the collection
process defer to our custom "DocDirectory" collector. We extend
the set of collected test items by ``ReSTChecker`` instances
which themselves create ``ReSTSyntaxTest`` and ``LinkCheckerMaker``
items. All of this instances (need to) follow the `collector API`_.
Customizing the reporting of Test Failures
--------------------------------------------
XXX implement Item.repr_run and Item.repr_path for your test items
Writing new assertion methods
-------------------------------------
XXX __tracebackhide__, and use "print"
Customizing the collection process in a module
----------------------------------------------
REPEATED WARNING: details of the collection and running process are
still subject to refactorings and thus details will change.
If you are customizing py.test at "Item" level then you
definitely want to be subscribed to the `py-dev mailing list`_
to follow ongoing development.
If you have a module where you want to take responsibility for
collecting your own test Items and possibly even for executing
a test then you can provide `generative tests`_ that yield
@ -232,14 +59,13 @@ The collection process dynamically consults the *chain of conftest.py*
modules to determine collectors and items at ``Directory``, ``Module``,
``Class``, ``Function`` or ``Generator`` level respectively.
Customizing execution of Functions
----------------------------------
Customizing execution of Items and Functions
----------------------------------------------------
- ``py.test.collect.Function`` test items control execution
of a test function. ``function.run()`` will get called by the
session in order to actually run a test. The method is responsible
for performing proper setup/teardown ("Test Fixtures") for a
Function test.
of a test function through its ``function.runtest()`` method.
This method is responsible for performing setup and teardown
("Test Fixtures") for a test Function.
- ``Function.execute(target, *args)`` methods are invoked by
the default ``Function.run()`` to actually execute a python

View File

@ -44,7 +44,6 @@ Full Contents
.. toctree::
:maxdepth: 2
:numbered:
test
execnet

View File

@ -46,11 +46,78 @@ behind the ``--`` double-dash.
IOW, you can set default values for options per project, per
home-directoray, per shell session or per test-run.
.. _`collection process`:
Test Collection process
======================================================
.. module:: py.test.collect
:synopsis: basic test collection classes
The collecting process is iterative so that distribution
and execution of tests can start as soon as the first test
item is collected. Collection nodes with children are
called "Collectors" and terminal nodes are called "Items".
Here is an example of such a tree, generated with the
command ``py.test --collectonly py/xmlobj``::
<Directory 'xmlobj'>
<Directory 'testing'>
<Module 'test_html.py' (py.__.xmlobj.testing.test_html)>
<Function 'test_html_name_stickyness'>
<Function 'test_stylenames'>
<Function 'test_class_None'>
<Function 'test_alternating_style'>
<Module 'test_xml.py' (py.__.xmlobj.testing.test_xml)>
<Function 'test_tag_with_text'>
<Function 'test_class_identity'>
<Function 'test_tag_with_text_and_attributes'>
<Function 'test_tag_with_subclassed_attr_simple'>
<Function 'test_tag_nested'>
<Function 'test_tag_xmlname'>
By default all directories not starting with a dot are traversed,
looking for ``test_*.py`` and ``*_test.py`` files. Those Python
files are imported under their `package name`_.
The Module collector looks for test functions
and test classes and methods. Test functions and methods
are prefixed ``test`` by default. Test classes must
start with a capitalized ``Test`` prefix.
.. _`package name`:
constructing the package name for test modules
-------------------------------------------------
Test modules are imported under their fully qualified
name. Given a filesystem ``fspath`` it is constructed as follows:
* walk the directories up to the last one that contains
an ``__init__.py`` file.
* perform ``sys.path.insert(0, basedir)``.
* import the root package as ``root``
* determine the fully qualified name for ``fspath`` by either:
* calling ``root.__pkg__.getimportname(fspath)`` if the
``__pkg__`` exists.` or
* otherwise use the relative path of the module path to
the base dir and turn slashes into dots and strike
the trailing ``.py``.
Plugin methods
=======================================
.. module:: py.__.test.pluginapi
A Plugin class may implement the following attributes and methods:
XXX

View File

@ -1,182 +0,0 @@
==============================================
Why, who, what and how do you do *the py lib*?
==============================================
Why did we start the py lib?
============================
Among the main motivation for the py lib and its flagship
py.test tool were:
- to test applications with a testing tool that provides
advanced features out of the box, yet allows full customization
per-project.
- distribute applications in an ad-hoc way both for testing
and for application integration purposes.
- help with neutralizing platform and python version differences
- offer a uniform way to access local and remote file resources
- offer some unique features like micro-threads (greenlets)
What is the py libs current focus?
==================================
testing testing testing
-----------------------
Currently, the main focus of the py lib is to get a decent
`test environment`_, indeed to produce the best one out there.
Writing, distributing and deploying tests should become
a snap ... and fun!
On a side note: automated tests fit very well to the dynamism
of Python. Automated tests ease development and allow fast
refactoring cycles. Automated tests are a means of
communication as well.
ad-hoc distribution of programs
------------------------------------
The py lib through its `py.execnet`_ namespaces offers
support for ad-hoc distributing programs across
a network and subprocesses. We'd like to generalize
this approach further to instantiate and let whole
ad-hoc networks communicate with each other while
keeping to a simple programming model.
.. _`py.execnet`: execnet.html
allowing maximum refactoring in the future ...
----------------------------------------------
explicit name export control
............................
In order to allow a fast development pace across versions of
the py lib there is **explicit name export control**. You
should only see names which make sense to use from the outside
and which the py lib developers want to guarantee across versions.
However, you don't need to treat the ``py`` lib as
anything special. You can simply use the usual ``import``
statement and will not notice much of a difference - except that
the namespaces you'll see from the ``py`` lib are relatively
clean and have no clutter.
Release policy & API maintenance
........................................
We'll talk about major, minor and micro numbers as the three
numbers in "1.2.3" respectively. These are the
the rough release policies:
- Micro-releases are bug fix releases and should not introduce
new names to the public API. They may add tests and thus
further define the behaviour of the py lib. They may
completly change the implementation but the public API
tests should continue to run (unless they needed to
get fixed themselves).
- No **tested feature** of the exported py API shall vanish
across minor releases until it is marked deprecated.
For example, pure API tests of a future version 1.0 are to
continue to fully run on 1.1 and so on. If an API gets
deprecated with a minor release it goes with the next minor
release. Thus if you don't use deprecated APIs you should
be able to use the next two minor releases. However, if
you relied on some untested implementation behaviour,
you may still get screwed. Solution: add API tests to the
py lib :-) It's really the tests that make the difference.
- Pure API tests are not allowed to access any implementation
level details. For example, accessing names starting with
a single leading '_' is generally seen as an implementation
level detail.
- major releases *should*, but are not required to, pass
all API tests of the previous latest major released
version.
the need to find the right *paths* ...
--------------------------------------
Another focus are well tested so called *path* implementations
that allow you to seemlessly work with different backends,
currently a local filesystem, subversion working copies and
subversion remote URLs.
How does py development work?
=============================
Communication and coding style
------------------------------
We are discussing things on our `py-dev mailing list`_
and collaborate via the codespeak subversion repository.
We follow a `coding style`_ which strongly builds on `PEP 8`_,
the basic python coding style document.
It's easy to get commit rights especially if you are an
experienced python developer and share some of the
frustrations described above.
Licensing
-----------------
The Py lib is released under the MIT license and all
contributors need to release their contributions
under this license as well.
connections with PyPy_
---------------------------------
A major motivation for writing the py lib stems from needs
during PyPy_ development, most importantly testing and
file system access issues. PyPy puts a lot of pressure
on a testing environment and thus is a good **reality test**.
Who is "we"?
=============================
Some initial code was written from *Jens-Uwe Mager* and *Holger
Krekel*, after which Holger continued on a previous
incarnations of the py.test tool (known first as 'utest', then
as 'std.utest', now for some 2 years 'py.test').
Helpful discussions took place with *Martijn Faassen*, *Stephan
Schwarzer*, *Brian Dorsey*, *Grigh Gheorghiu* and then
*Armin Rigo* who contributed important parts.
He and Holger came up with a couple of iterations of the
testing-code that reduced the API to basically nothing: just the
plain assert statement and a ``py.test.raises`` method to
check for occuring exceptions within tests.
Currently (as of 2007), there are more people involved
and also have worked funded through merlinux_ and the
PyPy EU project, Carl Friedrich Bolz, Guido Wesdorp
and Maciej Fijalkowski who contributed particularly
in 2006 and 2007 major parts of the py lib.
.. _`talk at EP2004`: http://codespeak.net/svn/user/hpk/talks/std-talk.txt
.. _`coding style`: coding-style.html
.. _`PEP 8`: http://www.python.org/peps/pep-0008.html
.. _`py-dev mailing list`: http://codespeak.net/mailman/listinfo/py-dev
.. _`test environment`: test.html
.. _`PyPy`: http://codespeak.net/pypy
.. _future: future.html
.. _`py.test tool and library`: test.html
.. _merlinux: http://merlinux.de
--
.. [#] FOSS is an evolving acronym for Free and Open Source Software