forgetest1
/
antlr
forked from jasder/antlr
1
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity
1,191 Commits 4 Branches 0 Tags 82 MiB
Commit Graph

608 Commits

Author SHA1 Message Date
Sam Harwell dcdcad8feb Add IntegerList.addAll overload for Collection<Integer>, add contains, sort, hashCode, equals, toString. 2012-07-30 15:17:02 -05:00
Sam Harwell 3bf99d6d88 Add IntervalSet.toIntegerList(), use IntegerStack to hold mode stack in Lexer 2012-07-30 15:13:26 -05:00
Sam Harwell fa62570737 Add IntegerList and IntegerStack, behaves like ArrayList<Integer> and ArrayDeque<Integer> (mostly) except doesn't require boxing 2012-07-30 15:13:26 -05:00
Sam Harwell 885f6530ad Use chained calls to append instead of string concatenation 2012-07-20 14:52:49 -05:00
Sam Harwell 44ef41ff29 Remove unnecessary variable initializations 2012-07-20 14:52:46 -05:00
Sam Harwell 3b9940b02a Use isEmpty() instead of comparing size() with 0 2012-07-20 14:52:43 -05:00
Sam Harwell 2f0029a040 Improved handling of (potentially) null fields 2012-07-20 14:52:14 -05:00
Sam Harwell 4c4f767d17 Prevent NPE after reporting error 2012-07-20 14:52:13 -05:00
Sam Harwell 45e42d7243 Parameter to equals can be null 2012-07-20 14:51:54 -05:00
Sam Harwell 3a35f3cb08 Check object type in equals 2012-07-20 14:51:53 -05:00
Sam Harwell 29d71acef9 Update documentation 2012-07-18 16:12:37 -05:00
Sam Harwell a37f8cf4f1 Add missing license notices 2012-07-18 15:57:48 -05:00
Terence Parr f220212a95 couldn't get Horstmann's routine to do EPS not PS so had to backtrack. 2012-07-14 16:32:04 -07:00
Terence Parr 47362b2951 use Horstmann's PS save not mine; it'll handle Japanese char etc.. this way. 2012-07-14 15:38:33 -07:00
Terence Parr a3371ac9ea rm unneeded override. 2012-07-09 17:32:20 -07:00
Terence Parr 9a4227841d -print -> -tree 2012-07-08 13:38:56 -07:00
Terence Parr 707ff615b4 tweak javadoc 2012-07-08 12:18:43 -07:00
Terence Parr 3b2c0a6177 fix bug in get hidden channel stuff. rewrote. update test. 2012-07-07 18:48:48 -07:00
Terence Parr ec47251bb2 unbuffered release() can get markers in weird order since we reset p to beginning of buffer. Might mark at 1 and then at release p = 0 etc... Don't look for errors. Just reset earliestMarker if needed. 2012-07-04 11:32:41 -07:00
Terence Parr 6b6274b0c8 add type narrowed getters 2012-07-02 15:48:45 -07:00
Terence Parr 71bd16020f Merge branch 'master' into nogenerics-in-error-listener-2nd-try 2012-07-02 12:45:48 -07:00
Terence Parr 111c44b3fd add default ctor to Lexer 2012-07-02 12:45:30 -07:00
Terence Parr 4329f00186 Merge branch 'master' into nogenerics-in-error-listener-2nd-try 2012-07-02 12:19:51 -07:00
Terence Parr ae7313f36a factor out input.read() and make ctor for easy subclass. 2012-07-02 12:19:20 -07:00
Terence Parr 3ad87ba12c rm generic parameter from ANTLRErrorListener, ripple effect. 2012-07-01 22:34:35 -07:00
Terence Parr bb5790d6a9 Add convenience method. 2012-07-01 17:36:22 -07:00
Terence Parr 37396d6040 clean up 2012-07-01 12:20:54 -07:00
Terence Parr db66257d7e cleanup so Lexer lets us subclass to support multiple token emit()s per lex rule match. 2012-07-01 12:19:44 -07:00
Terence Parr 0b4fa240c5 alter comment 2012-07-01 10:55:31 -07:00
Terence Parr f80166b39c Rewrote the unbuffered token stream to use a type-adjusted version of the unbuffered character stream. Simpler and I get to remove fast queue and lookahead string classes. These unbuffered streams always prime the pump with the 1st single. 
Added a unit test for the unbuffered token stream.

Made sure that the unbuffered streams move forward always on a consume

Removed the reset method from the unbuffered streams because it's meaningless to reset to the beginning of the buffer.
 2012-07-01 09:39:11 -07:00
Terence Parr 5c69d31e88 CommonTokenFactory now knows how to copy the text out of the character stream buffer before they disappear in unbuffered character strengths; added ctor. 
Lexer now guarantees that the text of the current token is always available to the emit() method even if the character stream is unbuffered.

Added some hooks to see some of the internal data in the unbuffered character stream so that I can test it better.

Updated LexerInterpreter so that it uses the token factory.

Improved/added unit tests for the unbuffered character string.

Updated various comments
 2012-06-30 16:40:16 -07:00
Terence Parr 44355d6ff6 oops; don't need these afterall 2012-06-30 16:36:25 -07:00
Terence Parr 3900606024 no need for range field. removed. overrirde tostring so nothing weird comes out. 2012-06-30 16:36:01 -07:00
Terence Parr 0c22d12870 updated comments, remove duplicate code, add new functionality. 2012-06-29 13:02:44 -07:00
Terence Parr e861902a10 set hidden channel num to 1 not 99. default is chan 0 2012-06-29 12:03:54 -07:00
Terence Parr f396b4dc82 error->syntaxError in listener 2012-06-25 13:25:13 -07:00
Terence Parr b18475113b Merge branch 'master' into interactive 2012-06-16 17:20:04 -07:00
Terence Parr 13d9f6ed53 add comment 2012-06-08 17:04:46 -07:00
Terence Parr abc0e2ef87 add getSpeculativeText(); text matched so far in a lexer rule. Use this in predicates not actions. add unit tests. 2012-06-07 18:31:36 -07:00
Terence Parr 018e3c03e8 look for T not just TLexer if no parser just tokens. 2012-06-07 18:27:42 -07:00
Terence Parr b255509e96 fix a bug related to semantic predicates in the lexer and generally cleaned up variable and method names in the simulator. I moved all of the predicates to the right side of lexer rules in the unit tests. Later, we should ensure that predicates only occur on the right edge of lexer rules. We should state that the rule is not been accepted so we can't test things like getText(), we have to use more raw indexes into the character stream. In the lexer simulator, the addDFAState() method now does not try to compute whether there is a predicate in the configurations. That information has already been set into the ATNConfigSet by the getEpsilonTarget() method. [I should also point out that I have not tested the Java parsing in a while and now it hits a landmine on a number of common Java files in jdk :(. I'm not sure where that crept in] 2012-06-07 17:31:18 -07:00
Terence Parr 5a7b3b0cab fixed a bug where some decisions that could see predicates thought they were LL(1). Added HIT_PRED invalid token type to the lookahead set to indicate we had a predicate; later we wipe out that entire set before returning from "get token lookahead sets". 2012-06-01 18:09:25 -07:00
Terence Parr 59af8ab76b clean up constants 2012-06-01 17:38:54 -07:00
Terence Parr aab8806783 added setter for _token 2012-05-26 11:32:32 -07:00
Terence Parr e737653b4f lexer getText() included lookahead char, which it shouldn't. 2012-05-23 14:09:32 -07:00
Terence Parr bd87ec95e2 add setters for line/charpos to lexer/lexersimulator 2012-05-22 16:21:41 -07:00
Terence Parr ed879ab4e3 working towards interactive lexer 2012-05-21 10:16:37 -07:00
Terence Parr 6215390408 add ctor for consistency 2012-05-20 17:28:14 -07:00
Terence Parr 5fc148381a rm toString from token stream. it affects buffer. 2012-05-20 16:49:44 -07:00
Terence Parr 990da7d1d8 add unit test 2012-05-19 18:08:56 -07:00