forgetest1
/
antlr
forked from jasder/antlr
1
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity
1,666 Commits 4 Branches 0 Tags 82 MiB
Commit Graph

875 Commits

Author SHA1 Message Date
Terence Parr eda95f7478 DFA cache on again; dropped from 25 to 16M!!! 2012-07-28 10:34:34 -07:00
Terence Parr 8a301f59f8 try making smaller config list. 2012-07-28 10:29:04 -07:00
Terence Parr 2ee5a2f3ae release entire ConfigHashSet upon readonly. sames some mem and is faster. 2012-07-27 22:38:43 -07:00
Terence Parr 7ff5544573 wipe out config lookup in ATNConfigSet when it goes readonly. 2012-07-27 22:20:23 -07:00
Terence Parr 54fd28c29d add() was wrong 2012-07-27 21:03:41 -07:00
Terence Parr 8279c3da11 turn on local context cache. 2012-07-27 20:53:10 -07:00
Terence Parr 14372f2515 only 14 unit tests fail. 2012-07-27 17:38:02 -07:00
Terence Parr b8f072e231 rebuilt new hashtable as T[][] 2012-07-27 17:18:33 -07:00
Terence Parr b843d6cf59 initial new hash table impl. 2012-07-27 14:51:07 -07:00
Terence Parr b83c90af2f play with atnconfigset fields 2012-07-27 14:50:36 -07:00
Terence Parr 9445d1fdd6 update comment 2012-07-27 10:38:43 -07:00
Terence Parr 26165c5d54 atnconfigsets can be readonly now. no copying to put into new DFA state, set readonly instead. made atnconfigset.equals() more accurate. add -notree option to test java lr 2012-07-27 10:34:59 -07:00
Terence Parr 093af0edce update comment 2012-07-26 18:26:17 -07:00
Terence Parr 6322ff3abd add comment 2012-07-26 18:17:05 -07:00
Terence Parr 0e24a66f07 reduce memory usage by creating LexerATNConfig a subclass with the lexer action index. saves about 2M on java.* with javalr. 2012-07-26 17:51:02 -07:00
Terence Parr 3c7b4c2a33 big cleanup. 2012-07-26 17:28:10 -07:00
Terence Parr 6b8e6cf0a7 reduces a few equals calls. 2012-07-26 10:58:11 -07:00
Terence Parr faafe8dfa2 no local/shared predctx caches. remove spurious array creation. 2012-07-26 10:35:52 -07:00
Terence Parr 86a4f26596 still has dfa and local predict ctx cache, but doesn't help much, and takes time. Eval preds as we pass them in full ctx mode; no sempred ctx in configs then. removed LANDMINE!!!!!!!!! some lame code that uses fields like _startIndex; fix that. added bail option to testjavalr. 18s to parse java.* with javalr full ctx now. 2012-07-25 18:47:34 -07:00
Terence Parr 05de714987 add chk for null contextCache. 2012-07-25 10:58:04 -07:00
Terence Parr 3774c5752b add more tracking var 2012-07-25 10:30:19 -07:00
Terence Parr 3cec976f7e Don't pursue the closure if there is just one state or it's predicting unique alt. Fixed up computeReachSet() to do this. Dropped from 6s to 4.5s for TestJavaLR -SLL on java.* with 45M max footprint. 2012-07-24 18:45:23 -07:00
Terence Parr 2495ce6b20 add SLL option to the ATN simulator and use in TestJavaLR. 2012-07-24 17:42:47 -07:00
Terence Parr 2ae4d4eebd add ctx cache to merge op. now i get optimal graph it seems. doesn't seem longer for java.* with JavaLR. still landmine in Pattern.java, Foo.java. Made toDOTString sort predctx nodes by id for repeatable tests. updated unit tests. 2012-07-24 16:26:01 -07:00
Terence Parr ac4f00524e add ctx cache. 2012-07-24 14:19:43 -07:00
Terence Parr 9627652b67 tweak to compile 2012-07-24 12:20:56 -07:00
Terence Parr 320868d661 added code chking size of ctxs. changed configsets -> configs. added recur getallnodes. 2012-07-24 10:04:51 -07:00
Terence Parr 344a56fd1e finished inter-dfa-state-context-cache but doesn't fix landmine. add debug_list_atn_decisions field. 2012-07-23 18:21:53 -07:00
Terence Parr 1966379265 shelve to look back at sam's. added optimizeConfigs() stuff. 2012-07-23 17:50:22 -07:00
Terence Parr 935fd2831e add writeFile util. improve dbg msg 2012-07-23 10:29:12 -07:00
Terence Parr d8a9207041 improve combine common parents, return a or b in merge array of merged is a or b; new unit test. 2012-07-22 20:07:46 -07:00
Terence Parr e18b9132d9 cleanup and merge common parents for array merge; fixes unit test 2012-07-22 16:39:53 -07:00
Terence Parr f1e4d85d0a fix cut/paste error. 2012-07-22 16:17:07 -07:00
Terence Parr f7eeca274f reorg closure and fix bug where $ in arrayctx wouldn't perform global follow. fix case in array merge that didn't check both a[i], b[i] as $ (only matters in full ctx). unit tests for graph show fewer ctx's created. 2012-07-22 16:09:45 -07:00
Terence Parr 44ae1dad0b if merged array is 1 node, return singleton. more unit tests 2012-07-22 15:05:12 -07:00
Terence Parr 3edb35d95e cleanup; avoid array creation 2012-07-22 12:59:26 -07:00
Terence Parr 829ad9191c didn't join a$ + bx as [a,b]$ for local ctx. new unit test to chk 2012-07-22 12:57:30 -07:00
Terence Parr 8695210903 cleanup, simplify array ctx equals 2012-07-22 12:17:34 -07:00
Terence Parr 43424cd5df do check for a==b in merge(); improve DOT output 2012-07-22 12:10:40 -07:00
Terence Parr 43e986db8b small hash code cleanup. 2012-07-22 11:27:44 -07:00
Terence Parr 54d23066b2 merge of array and $ for rootIsWildcard was broken. Fixed JavaLR parsing. Still sticks on Pattern.java but not 5 hours anymore--faster up til that landmine. 2012-07-21 19:24:03 -07:00
Terence Parr 36a725765e cleanup; hashcode for single node pred ctx was wrong. failed to see [46] and [46] as same. 2012-07-21 17:13:16 -07:00
Terence Parr 991014d3f2 clean up, add comments, fix graph unit tests, updated toDotString(). 2012-07-21 15:10:15 -07:00
Terence Parr f78bf4d097 rm comment; looks inappropriate for branch 2012-07-21 10:45:59 -07:00
Terence Parr 0141bc058a merge from master,In preparation for continuing on this experiment branch. 2012-07-21 10:38:55 -07:00
Sam Harwell 885f6530ad Use chained calls to append instead of string concatenation 2012-07-20 14:52:49 -05:00
Sam Harwell 44ef41ff29 Remove unnecessary variable initializations 2012-07-20 14:52:46 -05:00
Sam Harwell 3b9940b02a Use isEmpty() instead of comparing size() with 0 2012-07-20 14:52:43 -05:00
Sam Harwell 2f0029a040 Improved handling of (potentially) null fields 2012-07-20 14:52:14 -05:00
Sam Harwell 4c4f767d17 Prevent NPE after reporting error 2012-07-20 14:52:13 -05:00
Sam Harwell 45e42d7243 Parameter to equals can be null 2012-07-20 14:51:54 -05:00
Sam Harwell 3a35f3cb08 Check object type in equals 2012-07-20 14:51:53 -05:00
Sam Harwell 29d71acef9 Update documentation 2012-07-18 16:12:37 -05:00
Sam Harwell a37f8cf4f1 Add missing license notices 2012-07-18 15:57:48 -05:00
Terence Parr f220212a95 couldn't get Horstmann's routine to do EPS not PS so had to backtrack. 2012-07-14 16:32:04 -07:00
Terence Parr 47362b2951 use Horstmann's PS save not mine; it'll handle Japanese char etc.. this way. 2012-07-14 15:38:33 -07:00
Sam Harwell d626c4acd6 Simplify ParserRuleContext.toString 2012-07-13 14:26:56 -05:00
Terence Parr a3371ac9ea rm unneeded override. 2012-07-09 17:32:20 -07:00
Terence Parr 9a4227841d -print -> -tree 2012-07-08 13:38:56 -07:00
Terence Parr 707ff615b4 tweak javadoc 2012-07-08 12:18:43 -07:00
Terence Parr 3b2c0a6177 fix bug in get hidden channel stuff. rewrote. update test. 2012-07-07 18:48:48 -07:00
Terence Parr ec47251bb2 unbuffered release() can get markers in weird order since we reset p to beginning of buffer. Might mark at 1 and then at release p = 0 etc... Don't look for errors. Just reset earliestMarker if needed. 2012-07-04 11:32:41 -07:00
Terence Parr 6b6274b0c8 add type narrowed getters 2012-07-02 15:48:45 -07:00
Terence Parr 71bd16020f Merge branch 'master' into nogenerics-in-error-listener-2nd-try 2012-07-02 12:45:48 -07:00
Terence Parr 111c44b3fd add default ctor to Lexer 2012-07-02 12:45:30 -07:00
Terence Parr 4329f00186 Merge branch 'master' into nogenerics-in-error-listener-2nd-try 2012-07-02 12:19:51 -07:00
Terence Parr ae7313f36a factor out input.read() and make ctor for easy subclass. 2012-07-02 12:19:20 -07:00
Terence Parr 3ad87ba12c rm generic parameter from ANTLRErrorListener, ripple effect. 2012-07-01 22:34:35 -07:00
Terence Parr bb5790d6a9 Add convenience method. 2012-07-01 17:36:22 -07:00
Terence Parr 37396d6040 clean up 2012-07-01 12:20:54 -07:00
Terence Parr db66257d7e cleanup so Lexer lets us subclass to support multiple token emit()s per lex rule match. 2012-07-01 12:19:44 -07:00
Terence Parr 0b4fa240c5 alter comment 2012-07-01 10:55:31 -07:00
Terence Parr f80166b39c Rewrote the unbuffered token stream to use a type-adjusted version of the unbuffered character stream. Simpler and I get to remove fast queue and lookahead string classes. These unbuffered streams always prime the pump with the 1st single. 
Added a unit test for the unbuffered token stream.

Made sure that the unbuffered streams move forward always on a consume

Removed the reset method from the unbuffered streams because it's meaningless to reset to the beginning of the buffer.
 2012-07-01 09:39:11 -07:00
Terence Parr 5c69d31e88 CommonTokenFactory now knows how to copy the text out of the character stream buffer before they disappear in unbuffered character strengths; added ctor. 
Lexer now guarantees that the text of the current token is always available to the emit() method even if the character stream is unbuffered.

Added some hooks to see some of the internal data in the unbuffered character stream so that I can test it better.

Updated LexerInterpreter so that it uses the token factory.

Improved/added unit tests for the unbuffered character string.

Updated various comments
 2012-06-30 16:40:16 -07:00
Terence Parr 44355d6ff6 oops; don't need these afterall 2012-06-30 16:36:25 -07:00
Terence Parr 3900606024 no need for range field. removed. overrirde tostring so nothing weird comes out. 2012-06-30 16:36:01 -07:00
Terence Parr 0c22d12870 updated comments, remove duplicate code, add new functionality. 2012-06-29 13:02:44 -07:00
Terence Parr e861902a10 set hidden channel num to 1 not 99. default is chan 0 2012-06-29 12:03:54 -07:00
Terence Parr f396b4dc82 error->syntaxError in listener 2012-06-25 13:25:13 -07:00
Terence Parr b18475113b Merge branch 'master' into interactive 2012-06-16 17:20:04 -07:00
Terence Parr 13d9f6ed53 add comment 2012-06-08 17:04:46 -07:00
Terence Parr 3ece2c8640 merge from master; this pulls in the cleanup and bug fix in the lexer related to semantic predicates. 2012-06-08 10:18:49 -07:00
Terence Parr abc0e2ef87 add getSpeculativeText(); text matched so far in a lexer rule. Use this in predicates not actions. add unit tests. 2012-06-07 18:31:36 -07:00
Terence Parr 018e3c03e8 look for T not just TLexer if no parser just tokens. 2012-06-07 18:27:42 -07:00
Terence Parr b255509e96 fix a bug related to semantic predicates in the lexer and generally cleaned up variable and method names in the simulator. I moved all of the predicates to the right side of lexer rules in the unit tests. Later, we should ensure that predicates only occur on the right edge of lexer rules. We should state that the rule is not been accepted so we can't test things like getText(), we have to use more raw indexes into the character stream. In the lexer simulator, the addDFAState() method now does not try to compute whether there is a predicate in the configurations. That information has already been set into the ATNConfigSet by the getEpsilonTarget() method. [I should also point out that I have not tested the Java parsing in a while and now it hits a landmine on a number of common Java files in jdk :(. I'm not sure where that crept in] 2012-06-07 17:31:18 -07:00
Terence Parr 5a7b3b0cab fixed a bug where some decisions that could see predicates thought they were LL(1). Added HIT_PRED invalid token type to the lookahead set to indicate we had a predicate; later we wipe out that entire set before returning from "get token lookahead sets". 2012-06-01 18:09:25 -07:00
Terence Parr 59af8ab76b clean up constants 2012-06-01 17:38:54 -07:00
Terence Parr aab8806783 added setter for _token 2012-05-26 11:32:32 -07:00
Terence Parr e737653b4f lexer getText() included lookahead char, which it shouldn't. 2012-05-23 14:09:32 -07:00
Terence Parr bd87ec95e2 add setters for line/charpos to lexer/lexersimulator 2012-05-22 16:21:41 -07:00
Terence Parr ed879ab4e3 working towards interactive lexer 2012-05-21 10:16:37 -07:00
Terence Parr 6215390408 add ctor for consistency 2012-05-20 17:28:14 -07:00
Terence Parr 5fc148381a rm toString from token stream. it affects buffer. 2012-05-20 16:49:44 -07:00
Terence Parr 990da7d1d8 add unit test 2012-05-19 18:08:56 -07:00
Terence Parr 03d7b51d3c throw exception upon bad token indexes, add getOffChannelTokensToLeft, right. pull up some off channel token methods from CommonTokenStream. renamed. 2012-05-19 14:22:52 -07:00
Terence Parr 7be2ba9d42 remove unused type param 2012-05-19 13:06:27 -07:00
Terence Parr d46c7db52c merge master 2012-05-15 12:19:48 -07:00
Terence Parr c590ba8fd8 don't look backwards for err msg if EOF is entire input. make sure we don't use -1 rule index for ruleNames[] 2012-04-29 12:12:42 -07:00
Terence Parr 3ea6a5cb2f Merge branch 'master' into stack-graphs-integration 
Conflicts:
	tool/playground/T.g
 2012-04-26 12:01:28 -07:00
Terence Parr 30f61b17e3 comment tweak 2012-04-15 18:30:28 -07:00
Terence Parr f5501979c6 Merge branch 'master' into stack-graphs-integration 2012-04-09 10:56:14 -07:00
Terence Parr 35aeeb7c5c turn on building parse trees by default since we do -listener by default. 2012-04-07 11:27:18 -07:00
Terence Parr d22eb34fe7 Merge branch 'master' into stack-graphs-integration 2012-04-05 16:35:56 -07:00
Sam Harwell 1ade15392b Add license header 2012-04-04 15:33:39 -05:00
Sam Harwell e11dc07275 Remove TokenRewriteStream 2012-04-04 15:33:03 -05:00
Terence Parr 08e3ddefc0 Merge branch 'master' into stack-graphs-integration 2012-04-04 13:22:00 -07:00
Terence Parr 585aa0a14b Merge branch 'tokenstreamrewriter' of git://github.com/sharwell/antlr4 2012-04-04 13:20:58 -07:00
Sam Harwell ff2655d379 Rename getSource() to getTokenStream() 2012-04-04 15:20:45 -05:00
Sam Harwell bbd61dca35 Remove TokenStreamRewriter.getOriginalText() (use rewriter.getSource().getText() instead) 2012-04-04 10:15:29 -05:00
Sam Harwell a85f1c17a1 Add TokenStreamRewriter.getSource() 2012-04-04 10:14:09 -05:00
Sam Harwell e4612935f2 Only need TokenStream (was overly restricted to BufferedTokenStream) 2012-04-04 10:10:33 -05:00
Sam Harwell 16046a7ec1 Proper maven reference for org.abego.treelayout.core dependency 2012-04-04 09:52:06 -05:00
Terence Parr bf9c7c3a28 Remove old TokenRewriteStream in favor of new TokenStreamRewriter that is not a stream; it feeds off of the stream to provide transformations of that original stream. Updated the comment at the top of the file. Instead of toString() we use getText(). Altered the unit tests to create a buffered stream and then create the rewriter attached to that. 2012-04-02 18:10:32 -07:00
Terence Parr 3849cd572d rm SymbolStream<T>, change all to TokenStream. all but 5 tests pass. 2012-04-02 16:49:04 -07:00
Terence Parr 9ef6127982 merging from master 2012-04-01 18:13:40 -07:00
Sam Harwell 53c5aecb85 Implement UnbufferedTokenStream.get(int) within marked intervals 2012-04-01 18:23:32 -05:00
Sam Harwell 8ec9ba14c9 Implement getText within marked intervals of unbuffered streams 2012-04-01 18:22:43 -05:00
Sam Harwell d6e2e41c1f Fix stop index 2012-04-01 14:29:06 -05:00
Terence Parr e1870d16e9 added convenience getText(ctx) method to TokenStream. unlikley to need in CharStream and can get another way 2012-04-01 12:23:45 -07:00
Terence Parr 588e30bfe1 getText(...) moved to CharStream and TokenStream; cleaned up names. removed getInputString from parser. 2012-04-01 12:12:49 -07:00
Terence Parr f238d7579e added getText() to ParseTree. getText(tokens) is still really what you want but convenient for ctx.type().getText() when it's just one token. 2012-03-31 17:27:11 -07:00
Terence Parr cc20a52cdd allow special "tokens" start rule name so we can test pure lexer grammars. 2012-03-31 15:54:00 -07:00
Terence Parr c9aef6fdbe Merge branch 'master' into stack-graphs-integration 2012-03-31 10:33:22 -07:00
Sam Harwell e5d7c27b09 Updated comments to indicate preference of .g4 over .g 2012-03-30 11:37:22 -05:00
Terence Parr 447a5620f0 add -diagnostics option to test rig. 2012-03-28 11:07:23 -07:00
Terence Parr a363e70582 pull from master 2012-03-27 16:22:01 -07:00
Terence Parr bf07f7907f Merge branch 'override-ruleindex' of github.com:sharwell/antlr4 2012-03-27 15:09:50 -07:00
Terence Parr 41e97791bb Merge branch 'set-stop-token' of github.com:sharwell/antlr4 2012-03-27 15:08:57 -07:00
Terence Parr f307945d2a Merge branch 'lexer-npe' of github.com:sharwell/antlr4 2012-03-27 15:06:17 -07:00
Terence Parr 8465920343 Merge branch 'atnstate-reachability-test' of github.com:sharwell/antlr4 2012-03-27 15:05:39 -07:00
Sam Harwell e8765ef241 Fix errors tracking current state on rule return 2012-03-26 23:29:53 -05:00
Terence Parr 169f58a3ff fix def of EMPTY->INVALID 2012-03-25 21:19:39 -07:00
Sam Harwell aa34f33846 Move ParserRuleContext._s to a field in Recognizer 2012-03-25 21:51:48 -05:00
Sam Harwell df2e24a228 Override getRuleIndex() in generated context objects so ParserRuleContext.ruleIndex field can be removed 2012-03-25 21:50:55 -05:00
Sam Harwell bdda174af6 Set stop token even when RecognitionException occurs. Add ParserRuleContext.exception field to hold exception if one occurs. Resolves antlr/antlr4#49. 2012-03-25 21:50:12 -05:00
Terence Parr 2b5896e83b Merge branch 'master' into stack-graphs-integration 2012-03-25 10:08:29 -07:00
Terence Parr 542e700644 fix antlr/antlr4/#48 (getText from parse tree node). Don't NPE when start/stop null in getSourceInterval. 2012-03-25 10:01:27 -07:00
Terence Parr 4ea3c73d1f cleaned up interval stuff. moved getSourceInterval to ParserRuleContext; renamed create to of() in Interval. 2012-03-24 18:17:35 -07:00
Terence Parr c6365fb5e2 removed config list (huge) from default message. 2012-03-24 16:56:36 -07:00
Terence Parr e228e88ac5 tweak to remove insuff pred method 2012-03-24 16:48:26 -07:00
Terence Parr 6791bf60cf Merge commit '3f1f76d' into parrt-default-error-listener 
Conflicts:
	runtime/Java/src/org/antlr/v4/runtime/ANTLRErrorStrategy.java
	runtime/Java/src/org/antlr/v4/runtime/DefaultErrorStrategy.java
	runtime/Java/src/org/antlr/v4/runtime/DiagnosticErrorListener.java
	runtime/Java/src/org/antlr/v4/runtime/atn/ParserATNSimulator.java
 2012-03-24 16:44:47 -07:00
Terence Parr b1bcde76b7 Merge commit '768bfc0' into parrt-default-error-listener 2012-03-24 16:40:09 -07:00
Terence Parr b035ceec9e EMPTY gives "" not "$" now. rename closure->closure_. had return in wrong spot in closure for-loop. fix fromRuleContext. 2012-03-20 19:06:52 -07:00
Terence Parr 0f969af947 added in Sam's popAll to deal with loop end push/pop stuff. passes LR tests and all but 2 sempred tests. does some JavaLR.g4 parsing too. 2012-03-20 18:06:00 -07:00
Terence Parr 9318391664 tweak to fromRuleContext. more sempred tests work. 2012-03-20 12:32:28 -07:00
Terence Parr 55ed1a45bf lexer wasn't using new prediction context. weird. put in functionality for $ merge in full ctx mode. removed configs array in ATNConfigSet. ATNConfigSet wasn't tracking lexerActionIndex from config. ATNConfigSet now maps key to ATNConfig. 2012-03-20 12:12:18 -07:00
Terence Parr 5945b29a86 got more working 2012-03-19 19:58:04 -07:00
Terence Parr 7e9a86a3e1 got TestATNInterpreter working 2012-03-19 18:47:41 -07:00
Terence Parr 48d663667b pull from master 2012-03-19 18:03:32 -07:00
Terence Parr 2232ea5101 was not computing lookahead correctly in _LOOK. It assumed all epsilons were predicates. 2012-03-19 17:50:51 -07:00
Terence Parr 85b40c7d2e shelve 2012-03-19 17:47:57 -07:00
Sam Harwell bd7796544d Update comments 2012-03-19 08:33:55 -05:00
Sam Harwell 5e0f9a4490 Ensure target non-null in Transition constructor (additional runtime-side consistency check). 2012-03-19 08:33:36 -05:00
Sam Harwell a70cb6f36a Fix NPE in LexerATNSimulator.execDFA error recovery 2012-03-19 08:17:59 -05:00
Terence Parr 974ae65cb9 shelve 2012-03-18 20:21:39 -07:00
Terence Parr 90516272fe snapshot 2012-03-18 15:39:08 -07:00
Terence Parr 7233177441 got integrated graph stacks to compile. commit so i can look at sam's 2012-03-18 15:20:52 -07:00
Terence Parr 3a79aa15ba move new classes into proper package locations 2012-03-18 13:45:13 -07:00
Terence Parr ea7037dd2d missing semicolon 2012-03-16 14:29:07 -07:00
Terence Parr 9a0aaacbee rm k=1 chk to report early ambiguity. 2012-03-16 14:11:21 -07:00
Sam Harwell ae871dcc07 Rename exec/exec to execDFA/execATN 2012-03-13 19:27:52 -05:00
Sam Harwell aeb43069aa Move reach variable inside loop. Remove unnecessary ATNConfigSet clone. 2012-03-13 19:25:39 -05:00
Sam Harwell 51a15ac4a7 ATNConfigSet prior to error is closure, not reach. Reach is empty. 2012-03-13 19:25:38 -05:00
Sam Harwell 83c680850e Only create a new ATNConfigSet for reach when no DFA edge is available 2012-03-13 19:25:38 -05:00
Sam Harwell 9d774c6543 Separate handling of DFA and ATN transitions during lexer atn sim. 2012-03-13 19:25:37 -05:00
Sam Harwell 1f60da9cfd Handle DFA error states in lexer atn exec 2012-03-13 19:25:37 -05:00
Sam Harwell 590558c4d8 Combine DFAExecState and ATNExecState so we no longer need to throw exceptions in LexerATNSimulator.failOrAccept for valid inputs 2012-03-13 19:25:31 -05:00
Terence Parr aca3e0ddb2 make trim parse trees singular 2012-03-12 15:40:22 -07:00
Terence Parr dc627158e6 loop invar. code motion 2012-03-12 14:19:19 -07:00
Terence Parr 2fc9760ba7 Merge remote-tracking branch 'sharwell/ambig-sempred' 2012-03-12 13:50:45 -07:00
Terence Parr 79f204ff94 Merge commit 'fa9ec19' 2012-03-12 13:22:55 -07:00
Terence Parr 754408da27 Merge commit '86671d3' 2012-03-12 13:19:35 -07:00
Sam Harwell 3f1f76df7d Move reportAmbiguity, reportContextSensitivity, reportAttemptingFullContext, reportInsufficientPredicates from ANTLRErrorStrategy to ANTLRErrorListener. 
Add BaseErrorListener to allow implementing ANTLRErrorListener without implementing every method (e.g. ConsoleErrorListener).
DiagnosticErrorStrategy is now DiagnosticErrorListener, updated tests.
 2012-03-12 15:07:48 -05:00
Sam Harwell 768bfc0cf2 Add ProxyErrorListener to allow dispatching error reporting to multiple listeners without manually iterating over the list of listeners. 2012-03-12 15:07:48 -05:00
Sam Harwell dce72dcbfb * Recognizer._listeners initialized to have a single ConsoleErrorListener 
* Use CopyOnWriteArrayList so listeners can be added/removed in callbacks
* Remove special handling for _listeners null or empty (never null, ConsoleErrorListener will be present if feature is desired)
 2012-03-12 15:01:21 -05:00
Sam Harwell 50180825d9 Add boolean field ParserATNSimulator.reportAmbiguities (default true). When false, ambiguous alts aren't reported which allows usage of (the faster) incomplete semantic predicate evaluation even within execATN. 2012-03-10 16:41:57 -06:00
Sam Harwell 04cd48e88d No need to fully evaluate predicates when called from execDFA because ambiguous alts aren't reported anyway. 2012-03-10 16:32:10 -06:00
Sam Harwell 2cc39c2f9b Evaluate predicates for all ambiguous alternatives and report if still ambiguous after predicate evaluation. Remove misleading message insufficient predicates. Fixes antlr/antlr4#31. Partially addresses antlr/antlr4#39. 2012-03-10 16:16:39 -06:00
Sam Harwell 86671d30fb Tweak comments 2012-03-10 13:23:02 -06:00
Sam Harwell 7525bb6e1d Semantic predicates now evaluate with standard min-alt selection 2012-03-08 13:53:45 -06:00
Terence Parr a9e74ce399 Merge branch 'failed-predicate-message' of git://github.com/sharwell/antlr4 2012-03-06 13:28:47 -08:00
Sam Harwell 5deab167a1 ParseTreeVisitor.visit takes a ParseTree... only visitChildren takes a RuleNode 2012-03-04 12:52:43 -06:00
Sam Harwell b570641e2b Simplify ParseTree.visitChildren; ParseTree.visit and visitChildren take a RuleNode parameter; update comments 2012-03-04 12:17:43 -06:00
Sam Harwell a948736729 TerminalNodeImpl.isErrorNode now tests for ErrorNode interface instead of ErrorNodeImpl implementation 2012-03-03 21:57:49 -06:00
Sam Harwell 80b3ec8c62 Dispatch accept() for terminal nodes to visitor 2012-03-03 21:57:08 -06:00
Terence Parr 0375f8318a rename a parameter 2012-03-01 18:01:27 -08:00
Terence Parr 2ba5a1f12a Made the parse tree visitor walk ParseTree objects not ParserRuleContext objects. This necessitated pulling up the accept() and toStringTree(Parser) methods into ParseTree. 2012-03-01 17:57:49 -08:00
Sam Harwell fa9ec191bd Add TrimToSizeListener and Parser.get/setTrimParseTrees 2012-02-29 11:22:54 -06:00
Sam Harwell c3cd99858c Reduce size of generated code by emitting predicate text instead of full failed predicate message 2012-02-29 08:14:42 -06:00
Terence Parr dd69a7532d getToken/s returns TerminalNode now per Sam's suggestion. altered tests. seems correct. 2012-02-27 21:39:30 -08:00
Terence Parr 9ae92a5cef reformat 2012-02-27 21:32:25 -08:00
Terence Parr 67b2e6d7c1 forgot to move Pair etc... 2012-02-26 21:55:23 -08:00
Terence Parr e9a3de645a add useful method to get a->b pairs out 2012-02-26 15:55:46 -08:00
Terence Parr 41be88dcd5 mv to runtime 2012-02-26 15:49:45 -08:00
Sam Harwell bedc502d33 Set the listeners list back to null if all listeners are removed from it 2012-02-26 17:33:52 -06:00
Terence Parr 1e438746be added interface for error nodes, method in terminalnode. 2012-02-26 15:32:18 -08:00
Terence Parr db1b86af91 alter visitTerminal interface, add visitErrorNode. 2012-02-25 17:03:41 -08:00
Terence Parr 528e9c7b86 fix cut/paste error 2012-02-25 16:09:29 -08:00
Terence Parr 600c4bbd91 crap. sun already made IdentityHashMap; rm'ing my version 2012-02-25 16:07:34 -08:00
Terence Parr 73ae2b0bfc Added ParseTreeProperty and added generically useful Map that ignores key.hashCode and equals() and simply uses key identity instead. In particular, this is useful for ANTLR because parse trees are actually interfaces laid on top of the rule context objects used by ANTLR's grammar analysis. These context objects use special equals methods and so on. Plus, when we want to annotate trees with values, we want a specific note to have a value. Identity is what we intend. 2012-02-25 13:43:32 -08:00
Sam Harwell a74da2df60 Use ==ParserRuleContext.EMPTY instead of .isEmpty() 2012-02-24 15:57:43 -06:00
Sam Harwell 9bf6f284df Code cleanup (little things like using .isEmpty() and StringBuilder, and specify some small-scale generic arguments) 2012-02-23 16:42:36 -06:00
Sam Harwell 1e3e092326 Fix unchecked warnings in clone() 2012-02-23 16:13:15 -06:00
Sam Harwell e400f5b1a1 Move Recognizer._errHandler to Parser class 2012-02-23 16:13:14 -06:00
Sam Harwell 1c8efd6a48 Allow contravariant error listeners 2012-02-23 16:13:13 -06:00
Sam Harwell 6c26917b90 Use generic List instead of generic array for listeners 2012-02-23 16:13:12 -06:00
Sam Harwell 7dcb148d73 CommonTokenStream now extends BufferedTokenStream<Token> instead of BufferedTokenStream<CommonToken>. Token interface provides the same information and the change helps the work on improving generics. 2012-02-23 16:13:11 -06:00
Sam Harwell d7dbef2782 Can get token start and stop index from Token interface now 2012-02-23 16:13:11 -06:00
Sam Harwell 1492648d26 Add missing @Override annotations 2012-02-23 16:13:06 -06:00
Sam Harwell 96c08bec52 Remove unused imports 2012-02-23 16:12:02 -06:00
Terence Parr b459dafeb9 empty alternatives didn't trigger listener events. overrode getParent in ParserRuleContext to make type more specific. 2012-02-23 12:16:14 -08:00
Terence Parr 8a34176d82 added listener unit tests. fixed bug that didn't create ctx getters properly for recursive rules. added Symbol extends Token to parse tree stuff. added visitTerminal to Visitor. recursive alts now track their original, unedited AltAST subtree so we can properly count rule refs etc... later. dup of RuleRefAST was making wrong node. don't gen dispatch methods if no listener. 2012-02-22 12:44:33 -08:00
Terence Parr 58ef729be5 renamed next to target. factored out to methods, commented code. 2012-02-21 12:47:34 -08:00
Terence Parr 0e23c7e2a3 Merge commit '611728a3806d6a8c9fe44a538d34b448784986c1' 2012-02-21 12:14:11 -08:00
Sam Harwell 5fb389b6e4 Improve lexer performance during warm-up by tracking DFA states within exec (ATN) and using existing edges when available 2012-02-20 19:20:50 -06:00
Sam Harwell 611728a380 Update testPerformance to show information about DFA states in the lexer 2012-02-20 18:51:26 -06:00
Sam Harwell a81b1a17eb Extract abstract method in Recognizer for Parser.getGrammarFileName() and generated lexer's getGrammarFileName(). Make Recognizer.getTokenNames(), getRuleNames(), and getATN() abstract - implementations are always generated. 2012-02-20 17:18:57 -06:00
Sam Harwell 2982f4b5a6 Add comments describing the semantics of an ATNConfig set with partially predicated alts in ParserATNSimulator 2012-02-20 15:43:02 -06:00
Sam Harwell 16fa8ba4f4 Add comments describing the context depth latch in ParserATNSimulator.closure 2012-02-20 15:25:58 -06:00
Terence Parr e9122cafa3 Merge commit '46d5458' 2012-02-20 12:22:37 -08:00
Terence Parr 1a2094b2dd merging Sam\'s fix to full ctx pred etc...; effectively did pull parrt/antlr4#15, includes fixes antlr/antlr4#14, includes fixes antlr/antlr4#15 2012-02-20 11:48:03 -08:00
Terence Parr 4e8931519c added -parse-listener option and generated code if option on. parse listener differs from tree listener 2012-02-18 16:49:05 -08:00
Terence Parr 725b105135 Added visitor gen mechanism, runtime support 2012-02-16 17:49:57 -08:00
Sam Harwell 27806dc490 Merge remote-tracking branch 'parrt/master' 2012-02-15 14:58:16 -06:00
Sam Harwell 28919bbdc5 Fix ParserATNSimulator.execDFA consuming an unnecessary symbol after an accept or context-sensitive state is reached 2012-02-15 14:02:50 -06:00
Sam Harwell 685cf7b7d5 Further improvements to auto-label generic usage, return Collections.emptyList() instead of null when no children are found 2012-02-15 12:31:43 -06:00
Sam Harwell 0bbb7101ba Include input character in LexerNoViableAltException.toString() 2012-02-15 12:02:44 -06:00
Sam Harwell bc87562aff Remove unused exception LexerRecognitionExeption [sic] 2012-02-15 11:01:01 -06:00
Sam Harwell 5ab082967f Fix generic usage 2012-02-15 10:50:37 -06:00
Terence Parr fa3a664696 got some good examples working that play around with the getter methods. 2012-02-14 16:33:12 -08:00
Terence Parr cdb420fdde got implementation of getters in! Fixes #8 2012-02-14 15:01:03 -08:00
Sam Harwell 9600a70724 Fix NPE in CommonToken when TokenSource is not available 2012-02-13 08:41:57 -06:00
Terence Parr 6e4cba131a Allow var args in ctor of IntervalSet 2012-02-11 15:30:54 -08:00
Terence Parr f278d3b453 add unit test for left-recur grammars 2012-02-11 11:35:27 -08:00
Sam Harwell cd3adb140d Fix ParserATNSimulator.execDFA adding incorrect error edges in the DFA 2012-02-10 00:52:24 -06:00
Sam Harwell 30c58bf8eb Specify <?> type parameter on raw types 2012-02-10 00:51:23 -06:00
Sam Harwell 99ce3cba5c Fix ParserATNSimulator.closure not tracking proper context for context-sensitive predicates during full-context parsing 2012-02-09 19:47:56 -06:00
Sam Harwell 467797785f Fix generics in ParserATNSimulator 2012-02-09 15:37:56 -06:00
Sam Harwell ddf946b067 Token.EOF instead of -1, add <?> to generic type usage, make getUniqueAlt an instance method (need to override it for some apps), @NotNull annotations 2012-02-09 15:33:03 -06:00
Sam Harwell 7fb73a3849 Construct DFA instances with the decision number 2012-02-09 13:29:14 -06:00
Sam Harwell bf5df30708 Remove unnecessary initializations, use isEmpty() instead of size()==0 2012-02-09 12:45:46 -06:00
sharwell 46d5458617 Fix handling of alts with some configs predicated and some configs unpredicated (should be treated as always true) 2012-02-09 01:11:09 -06:00
sharwell 9c554a002b Add LexerATNSimulator.copyState (allows copying state information from another simulator instance) 2012-02-09 00:15:30 -06:00
sharwell 40a8759276 Add LexerATNSimulator.getTraceStream 2012-02-09 00:14:37 -06:00
sharwell 3b4ae2a742 Use trivial implementation of mark/release for BufferedTokenStream 2012-02-08 18:58:57 -06:00
sharwell f43f9f2218 Override CommonTokenStream.reset() to make sure the stream is left on an on-channel token 2012-02-08 18:58:19 -06:00
sharwell a34fd6330b Use isEmpty() instead of size()==0 2012-02-08 18:57:32 -06:00
sharwell a9e44f3aab Use isEmpty() instead of size()==0, use StringBuilder instead of StringBuffer, use chained append() calls instead of string concatenation, add @Override annotations 2012-02-08 18:57:05 -06:00
sharwell d181b230dc Specify generic type parameter 2012-02-08 18:55:57 -06:00
sharwell 64203312ac Improved SemanticContext.hashCode(), add @Override annotations 2012-02-08 18:54:40 -06:00