test_ok1/hacking/execnet-impl-notes.txt

123 lines
4.9 KiB
Plaintext

=============================================================================
Channel implementation notes
=============================================================================
The public API of channels make them appear either opened or closed.
When a channel is closed, we can't send any more items, and it will not
receive any more items than already queued.
Callbacks make the situation slightly more subtle. Callbacks are
attached to the ChannelFactory object, so that Channel objects can be
garbage-collected and still leave behind an active callback that can
continue to receive items.
The CHANNEL_CLOSE message is sent when a channel id is about to be removed
from the ChannelFactory, which means when the Channel object has been
garbage-collected *and* there is no callback any more.
If a Channel object is garbage-collected but the ChannelFactory has a
callback for it, a CHANNEL_LAST_MESSAGE message is sent. It is only useful
if both sides' Channel objects have an associated callback. In this
situation, CHANNEL_LAST_MESSAGE allows its receiver to un-register its own
callback; if/when in addition the receiver side also looses the last
reference to its Channel object, the Channel is closed. So in this particular
situation both sides must forget about the Channel object for it to be
automatically closed.
gateway <---> channelfactory ---> {id: weakref(channel)}
---> {id: callback}
State and invariants of Channel objects
---------------------------------------
_channels and _callbacks are dictionaries on the ChannelFactory.
Other attributes are on the Channel objects.
All states are valid at any time (even with multithreading) unless
marked with {E}, which means that they may be temporary invalid.
They are eventually restored.
States ("sendonly" means opened but won't receive any more items):
opened sendonly closed deleted
================= ============== ================== ===============
not _closed not _closed _closed <no ref left>
not _receiveclosed _receiveclosed {E} _receiveclosed
In the presence of callbacks, "deleted" does not imply "closed" nor "sendonly".
It only means that no more items can be sent. The (logical) channel can
continue to receive data via the call-back even if the channel object no
longer exists.
The two kinds of channels, with or without callback:
items read by receive() has a callback
============================= =======================================
_items is a Queue _items is None
id not in _callbacks
state==opened: id in _callbacks
{E} state==sendonly: there is {E} state!=opened: id not in _callbacks
an ENDMARKER in _items
{E} state==closed: there is
an ENDMARKER in _items
Callback calls should be considered asynchronuous. The channel can be in any
state and change its state while the callback runs.
The ChannelFactory's WeakValueDictionary _channels maps some ids to their
channel object, depending on their state:
opened sendonly closed deleted
================= ============== ================ ===============
id in _channels {E} not in {E} not in not in
All received RemoteErrors are handled exactly once: they are normally
re-raised once in waitclose() or receive(). If it is not possible, they are
at the moment dumped to stderr. (XXX should use logging/tracing)
Only channels in {E} "closed" state can hold RemoteErrors.
Methods:
* close() returns with the channel in "closed" state
* send() either send the data or raise if "closed"
* receive() wait for the next item. If no item left and the state
changes to non-"opened", raise
* waitclose() wait for a non-"opened" state
Assuming the channel is connected and the connection is alive, the local state
eventually influences the state of the corresponding remote channel object:
local | opened sendonly closed deleted
remote |
=======================================================
|
opened | ok n/a (1) (2)
|
sendonly | n/a n/a n/a ok
|
closed | (1) n/a ok ok
|
deleted | (2) ok ok ok
(1) The side with the closed channel object must send a CHANNEL_CLOSE message,
which will eventually put the other side's channel in "closed" state if
it is still "opened".
(2) If the deleted channel has no callback, this is equivalent to (1).
Otherwide, the side with the deleted channel must send a
CHANNEL_LAST_MESSAGE, which will eventually put the other side's channel in
"sendonly" state if it is still "opened".
n/a These configuration should never occur.