/// A source of tokens must provide a sequence of tokens via
///
/// and also must reveal it's source of characters;
///
/// 's text is
/// computed from a
///
/// ; it only store indices into the char
/// stream.
/// Errors from the lexer are never passed to the parser. Either you want to keep
/// going or you do not upon token recognition error. If you do not want to
/// continue lexing then you do not want to continue parsing. Just throw an
/// exception not under
///
/// and Java will naturally toss
/// you all the way out of the recognizers. If you want to continue lexing then
/// you should not throw an exception to the parser--it has already requested a
/// token. Keep lexing until you get a valid one. Just report errors and keep
/// going, looking for a valid token.
///
public interface ITokenSource
{
///