forgetest1
/
antlr
forked from jasder/antlr
1
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity
1,482 Commits 4 Branches 0 Tags 82 MiB
Commit Graph

970 Commits

Author SHA1 Message Date
Terence Parr c7d1ea7e23 Merge remote-tracking branch 'sharwell/atn-optimization' 2012-08-04 12:08:43 -07:00
Sam Harwell d504d21010 Do not emit token types for lexer rules with a 'type(...)' or 'more' command 2012-08-04 14:05:17 -05:00
Terence Parr 20d96b0a86 Merge remote-tracking branch 'sharwell/automata-cleanup' 2012-08-04 12:04:09 -07:00
Terence Parr 88dedbd1e0 added mergeArray cache, which didnt help java much but made massive diff to Go grammar. a file, nat.go, went from 2min to 3s in speed. Fixed bug where SLL had preds but they ORd to NONE. made it resolve to min alt. DoubleKeyMap moved to runtime. 2012-08-03 20:27:23 -07:00
Terence Parr e7b65057a6 added var for sll loop tail recursion default value; updated unit tests 2012-08-03 18:12:52 -07:00
Sam Harwell 674471c090 Add NotNull annotations, null check, tweak documentation 2012-08-03 08:46:07 -05:00
Sam Harwell 43da92e117 Remove old tree parser code 2012-08-03 08:45:36 -05:00
Sam Harwell 5019278204 Add ATN optimization to collapse multiple alternatives to a single SetTransition where possible. Currently disabled for parser grammars since the code generator doesn't support set transitions. 2012-08-03 08:33:11 -05:00
Sam Harwell 1f4df2ea5a Add ATN optimization to remove null state entries and renumber the remaining states 2012-08-03 08:33:10 -05:00
Sam Harwell f9bacf5060 Add ATN optimizer (currently doesn't provide optimizations) 2012-08-03 08:33:10 -05:00
Sam Harwell eff9e0c47f Move TailEpsilonRemover to top-level class 2012-08-03 08:33:09 -05:00
Terence Parr 0cfe3b4206 tweak to leave code for printing DFA DOT 2012-08-01 14:36:19 -07:00
Terence Parr feb41eee15 fix a few unit tests. 2012-07-31 19:09:40 -07:00
Terence Parr abb268416e Merge branch 'master' into main 2012-07-31 18:34:33 -07:00
Sam Harwell 170a8347bb Emit a RangeTransition instead of SetTransition if a set has only one interval 2012-07-31 09:07:04 -05:00
Terence Parr c893f2af08 Pulled Sam's new IntegerList 2012-07-30 15:20:43 -07:00
Sam Harwell 7d4f71d829 Use IntegerList in the tool 2012-07-30 15:18:07 -05:00
Terence Parr 489f3d17cb tweak 2012-07-29 17:04:05 -07:00
Terence Parr 8836e13954 use ReentrantReadWriteLock to sync. works much better but still occasional illegal monitor state :( 2012-07-29 17:01:41 -07:00
Terence Parr 4fe9efeb85 tweak tester 2012-07-29 14:28:59 -07:00
Terence Parr c97f36bd89 reorg workers 2012-07-29 13:45:41 -07:00
Terence Parr 230b687b03 add barrier 2012-07-29 13:03:10 -07:00
Terence Parr 5712424156 create 2 worker threads for testjavalr 2012-07-29 12:28:29 -07:00
Terence Parr 7181483bbf test javalr does -2x now 2012-07-29 10:57:51 -07:00
Terence Parr 510c1c0dd9 clean up 2012-07-29 10:50:14 -07:00
Terence Parr 79594a942e synchronized lexer. just as fast now to build new parser/lexer each iter in TestJavaLR 2012-07-29 10:48:08 -07:00
Terence Parr e3e739dfc7 The lexer and parser ATN simulators' adaptivePredict now synchronize on the specific DFA of the decision to be simulated. This should prevent a lot of contention that would occur if we synchronize the entire adaptivePredict method. When the individual DFA are created, we also synchronize on the shared DFA[] table quickly to create a DFA and insert it into the array. Code generation modified to have _decisionToDFA generated at the top of both the parser and the lexer. Simulators created now with the recognizer, ATN, DFA[]. Not sure the LexerInterp/ParserInterp work but pushing ahead anyway for the moment. 2012-07-29 09:49:35 -07:00
Terence Parr 109880b7ab added ALIAS_REASSIGNMENT warning so redef of string literal among rules caught. First literal goes to .tokens. 2012-07-28 14:24:39 -07:00
Terence Parr 680df17280 rm transient context cache for closure computations; slight speed improvement maybe. less complex. 2012-07-28 10:51:07 -07:00
Terence Parr 26165c5d54 atnconfigsets can be readonly now. no copying to put into new DFA state, set readonly instead. made atnconfigset.equals() more accurate. add -notree option to test java lr 2012-07-27 10:34:59 -07:00
Terence Parr 3c7b4c2a33 big cleanup. 2012-07-26 17:28:10 -07:00
Terence Parr 48ddc0bcee add gc() 2012-07-26 12:42:10 -07:00
Terence Parr 86a4f26596 still has dfa and local predict ctx cache, but doesn't help much, and takes time. Eval preds as we pass them in full ctx mode; no sempred ctx in configs then. removed LANDMINE!!!!!!!!! some lame code that uses fields like _startIndex; fix that. added bail option to testjavalr. 18s to parse java.* with javalr full ctx now. 2012-07-25 18:47:34 -07:00
Terence Parr 3774c5752b add more tracking var 2012-07-25 10:30:19 -07:00
Terence Parr 3cec976f7e Don't pursue the closure if there is just one state or it's predicting unique alt. Fixed up computeReachSet() to do this. Dropped from 6s to 4.5s for TestJavaLR -SLL on java.* with 45M max footprint. 2012-07-24 18:45:23 -07:00
Terence Parr 2495ce6b20 add SLL option to the ATN simulator and use in TestJavaLR. 2012-07-24 17:42:47 -07:00
Terence Parr 2ae4d4eebd add ctx cache to merge op. now i get optimal graph it seems. doesn't seem longer for java.* with JavaLR. still landmine in Pattern.java, Foo.java. Made toDOTString sort predctx nodes by id for repeatable tests. updated unit tests. 2012-07-24 16:26:01 -07:00
Terence Parr ac4f00524e add ctx cache. 2012-07-24 14:19:43 -07:00
Terence Parr 320868d661 added code chking size of ctxs. changed configsets -> configs. added recur getallnodes. 2012-07-24 10:04:51 -07:00
Terence Parr 00c4f98134 add test file that kills antlr 2012-07-23 10:35:02 -07:00
Terence Parr d8a9207041 improve combine common parents, return a or b in merge array of merged is a or b; new unit test. 2012-07-22 20:07:46 -07:00
Terence Parr e18b9132d9 cleanup and merge common parents for array merge; fixes unit test 2012-07-22 16:39:53 -07:00
Terence Parr f7eeca274f reorg closure and fix bug where $ in arrayctx wouldn't perform global follow. fix case in array merge that didn't check both a[i], b[i] as $ (only matters in full ctx). unit tests for graph show fewer ctx's created. 2012-07-22 16:09:45 -07:00
Terence Parr eb78593469 rm comment 2012-07-22 15:43:01 -07:00
Terence Parr 44ae1dad0b if merged array is 1 node, return singleton. more unit tests 2012-07-22 15:05:12 -07:00
Terence Parr 829ad9191c didn't join a$ + bx as [a,b]$ for local ctx. new unit test to chk 2012-07-22 12:57:30 -07:00
Terence Parr 43424cd5df do check for a==b in merge(); improve DOT output 2012-07-22 12:10:40 -07:00
Terence Parr 93945d47f5 Merge branch 'master' into stack-graphs-integration 2012-07-21 16:27:36 -07:00
Terence Parr 9539572ee7 simplify test. 2012-07-21 16:27:00 -07:00
Terence Parr 7a4a269615 fix test so no error due to EOF. 2012-07-21 15:36:56 -07:00
Terence Parr 8c03dbacf8 Resolves #41. EOF cannot follow s. would need s' : s EOF ; for that. updating unit test TestParserExec.testOptional 2012-07-21 15:29:07 -07:00
Terence Parr 991014d3f2 clean up, add comments, fix graph unit tests, updated toDotString(). 2012-07-21 15:10:15 -07:00
Terence Parr 0141bc058a merge from master,In preparation for continuing on this experiment branch. 2012-07-21 10:38:55 -07:00
Sam Harwell 885f6530ad Use chained calls to append instead of string concatenation 2012-07-20 14:52:49 -05:00
Sam Harwell 6259ab5c9e Use StringBuilder instead of StringBuffer 2012-07-20 14:52:48 -05:00
Sam Harwell 2056e019a5 Use HashMap instead of Hashtable 2012-07-20 14:52:47 -05:00
Sam Harwell 44ef41ff29 Remove unnecessary variable initializations 2012-07-20 14:52:46 -05:00
Sam Harwell 6045bd4eb5 Add null check 2012-07-20 14:52:45 -05:00
Sam Harwell 84e1025801 Remove extraneous ';' (empty statement), remove unnecessary null checks and dead code 2012-07-20 14:52:44 -05:00
Sam Harwell 3b9940b02a Use isEmpty() instead of comparing size() with 0 2012-07-20 14:52:43 -05:00
Sam Harwell 4c4f767d17 Prevent NPE after reporting error 2012-07-20 14:52:13 -05:00
Sam Harwell 4c1e9b4119 Improve performance of equals 2012-07-20 14:51:55 -05:00
Sam Harwell 3a35f3cb08 Check object type in equals 2012-07-20 14:51:53 -05:00
Sam Harwell 8b4a461262 Merge branch 'ModelElement' 2012-07-18 16:14:14 -05:00
Sam Harwell 29d71acef9 Update documentation 2012-07-18 16:12:37 -05:00
Sam Harwell 589d6bf187 Remove field which hides another field with the same name; add check for this condition to OutputModelWalker 2012-07-18 16:06:33 -05:00
Sam Harwell 45815d97f7 Specifically test for the ModelElement annotation on fields (not just any annotation will do) 2012-07-18 16:06:32 -05:00
Sam Harwell a37f8cf4f1 Add missing license notices 2012-07-18 15:57:48 -05:00
Terence Parr f220212a95 couldn't get Horstmann's routine to do EPS not PS so had to backtrack. 2012-07-14 16:32:04 -07:00
Sam Harwell 96654531b7 Use isEmpty() instead of size()==0, specify generic arguments 2012-07-13 14:33:42 -05:00
Terence Parr 683b915507 rm prints 2012-07-12 13:25:41 -07:00
Terence Parr 4923cb3874 reorg r : '...' ; matching. deals with -> cmds better (and more of em) 2012-07-12 13:25:18 -07:00
Terence Parr de744f912a allow A : '<' -> skip ; style token name/literal aliases. 2012-07-12 12:25:12 -07:00
Terence Parr 9738658126 rm unneeded code to set line/charpos 2012-07-09 17:32:03 -07:00
Terence Parr 3b2c0a6177 fix bug in get hidden channel stuff. rewrote. update test. 2012-07-07 18:48:48 -07:00
Terence Parr 00808ef11f off by 1 error. 2012-07-03 13:44:44 -07:00
Terence Parr d0e7e79b2b make sure tokenNames puts tokens at right spot; didn't handle wholes in token type sequence. 2012-07-03 13:37:43 -07:00
Terence Parr 9455f5cd30 bug fix; .tokens didn't allow ' ' or 'a b'. 2012-07-03 13:15:35 -07:00
Terence Parr 2e9c449ec4 fix a few tests 2012-07-03 12:42:10 -07:00
Terence Parr 1d9aef0a5e replace .tokens file parser with regex to avoid \t becoming tab char. 2012-07-03 12:40:36 -07:00
Terence Parr 3ad87ba12c rm generic parameter from ANTLRErrorListener, ripple effect. 2012-07-01 22:34:35 -07:00
Terence Parr 6053ac5269 rm gunit; hmm...my commits are wacked on this/last one. sorry! 2012-07-01 09:45:35 -07:00
Terence Parr 5c69d31e88 CommonTokenFactory now knows how to copy the text out of the character stream buffer before they disappear in unbuffered character strengths; added ctor. 
Lexer now guarantees that the text of the current token is always available to the emit() method even if the character stream is unbuffered.

Added some hooks to see some of the internal data in the unbuffered character stream so that I can test it better.

Updated LexerInterpreter so that it uses the token factory.

Improved/added unit tests for the unbuffered character string.

Updated various comments
 2012-06-30 16:40:16 -07:00
Terence Parr 44355d6ff6 oops; don't need these afterall 2012-06-30 16:36:25 -07:00
Terence Parr 0c22d12870 updated comments, remove duplicate code, add new functionality. 2012-06-29 13:02:44 -07:00
Terence Parr f396b4dc82 error->syntaxError in listener 2012-06-25 13:25:13 -07:00
Terence Parr 740208ee4d test code. 2012-06-17 16:56:26 -07:00
Terence Parr b18475113b Merge branch 'master' into interactive 2012-06-16 17:20:04 -07:00
Terence Parr bfc90d473c fix help msg slightly. 2012-06-13 12:42:21 -07:00
Terence Parr 3ece2c8640 merge from master; this pulls in the cleanup and bug fix in the lexer related to semantic predicates. 2012-06-08 10:18:49 -07:00
Terence Parr abc0e2ef87 add getSpeculativeText(); text matched so far in a lexer rule. Use this in predicates not actions. add unit tests. 2012-06-07 18:31:36 -07:00
Terence Parr b255509e96 fix a bug related to semantic predicates in the lexer and generally cleaned up variable and method names in the simulator. I moved all of the predicates to the right side of lexer rules in the unit tests. Later, we should ensure that predicates only occur on the right edge of lexer rules. We should state that the rule is not been accepted so we can't test things like getText(), we have to use more raw indexes into the character stream. In the lexer simulator, the addDFAState() method now does not try to compute whether there is a predicate in the configurations. That information has already been set into the ATNConfigSet by the getEpsilonTarget() method. [I should also point out that I have not tested the Java parsing in a while and now it hits a landmine on a number of common Java files in jdk :(. I'm not sure where that crept in] 2012-06-07 17:31:18 -07:00
Terence Parr d3c238569f fix unit test. 2012-06-06 15:44:10 -07:00
Terence Parr 5a2197a74d fix bug where X : 'x' {pred}? ; wasn't seen as alias from X to 'x'. 2012-06-02 11:44:27 -07:00
Terence Parr 7192cc9a68 rm deadcode 2012-06-02 11:44:01 -07:00
Terence Parr e1524fe86c add test 2012-06-02 11:28:25 -07:00
Terence Parr 0e9bcfbeca change in token type broke many tests; fixed. says 5 failures now. 2012-06-02 11:26:18 -07:00
Terence Parr 5a2fe297e3 ack. more cleanup work will be required for change of token types from 3..n to 1..n 2012-06-01 18:08:00 -07:00
Terence Parr 8481979f69 rename method 2012-06-01 18:06:48 -07:00
Terence Parr 94071d7d82 prevent negative indexes 2012-06-01 17:36:21 -07:00
Terence Parr de83aacb65 add unit test 2012-05-31 17:02:15 -07:00
Terence Parr 3b1d91e010 fix unit tests 2012-05-24 13:08:15 -07:00
Terence Parr 6539c5f464 remove inappropriate attributes of lexer rules 2012-05-24 13:07:43 -07:00
Terence Parr cd82465830 updated [...] parser so it handles < > and [ ] properly. no escapes now. 2012-05-23 12:48:07 -07:00
Terence Parr bc236a6e0b no tree attribute 2012-05-22 16:54:45 -07:00
Terence Parr 8078d39053 rm "st" attribute 2012-05-22 16:42:05 -07:00
Terence Parr 868ca7ca6d named actions in combined get copied to lexer now, but lexer::{...} are MOVED to lexer grammar. 2012-05-22 16:21:06 -07:00
Terence Parr ed879ab4e3 working towards interactive lexer 2012-05-21 10:16:37 -07:00
Terence Parr d378469c9c rename 2012-05-20 18:06:43 -07:00
Terence Parr 990da7d1d8 add unit test 2012-05-19 18:08:56 -07:00
Terence Parr c590ba8fd8 don't look backwards for err msg if EOF is entire input. make sure we don't use -1 rule index for ruleNames[] 2012-04-29 12:12:42 -07:00
Terence Parr 3ea6a5cb2f Merge branch 'master' into stack-graphs-integration 
Conflicts:
	tool/playground/T.g
 2012-04-26 12:01:28 -07:00
Terence Parr 6314b7d31b -> becomes # for alt labels 2012-04-26 11:59:57 -07:00
Terence Parr f4abc84054 MutableInt's compare method was messed up. 2012-04-10 10:51:51 -07:00
Terence Parr d22eb34fe7 Merge branch 'master' into stack-graphs-integration 2012-04-05 16:35:56 -07:00
Sam Harwell 9d92446125 Merge remote-tracking branch 'parrt/master' into basetest-error-reporting 2012-04-04 15:38:03 -05:00
Terence Parr 08e3ddefc0 Merge branch 'master' into stack-graphs-integration 2012-04-04 13:22:00 -07:00
Terence Parr 585aa0a14b Merge branch 'tokenstreamrewriter' of git://github.com/sharwell/antlr4 2012-04-04 13:20:58 -07:00
Sam Harwell ff2655d379 Rename getSource() to getTokenStream() 2012-04-04 15:20:45 -05:00
Terence Parr 8155b1138e Merge branch 'implicit-tokens' of git://github.com/sharwell/antlr4 2012-04-04 13:11:02 -07:00
Terence Parr 8658be69ba Merge branch 'test-reporting' of git://github.com/sharwell/antlr4 2012-04-04 13:08:07 -07:00
Sam Harwell bbd61dca35 Remove TokenStreamRewriter.getOriginalText() (use rewriter.getSource().getText() instead) 2012-04-04 10:15:29 -05:00
Sam Harwell d054a358a4 Don't run compiler if tool reported error(s) 2012-04-04 09:57:38 -05:00
Sam Harwell 7082590240 Allow tests to use the default tool listener for error reporting (prints formatted error messages). Use this in TestPerformance. 2012-04-04 09:56:32 -05:00
Sam Harwell 64e0a8734f Report warnings as well as errors 2012-04-04 09:53:10 -05:00
Sam Harwell 2c1f673f62 Mark ErrorType fields final 2012-04-04 09:51:23 -05:00
Sam Harwell 1c9e52b5d6 Use CopyOnWriteArrayList instead of Collections.synchronizedList 2012-04-04 09:50:51 -05:00
Sam Harwell 251490cfc9 Fix unit tests which included implicitly defined string literal tokens 2012-04-04 09:39:44 -05:00
Sam Harwell 9ce85cc6e4 Add error 126: implicit definition of string literal token in non-combined grammar 2012-04-04 09:38:28 -05:00
Sam Harwell c3e5c42c51 Fix (broken) unit tests which included implicitly defined tokens 2012-04-03 16:42:22 -05:00
Sam Harwell b433c2ae34 Add warning 125: implicit definition of token in parser 2012-04-03 16:40:52 -05:00
Terence Parr bf9c7c3a28 Remove old TokenRewriteStream in favor of new TokenStreamRewriter that is not a stream; it feeds off of the stream to provide transformations of that original stream. Updated the comment at the top of the file. Instead of toString() we use getText(). Altered the unit tests to create a buffered stream and then create the rewriter attached to that. 2012-04-02 18:10:32 -07:00
Terence Parr 3849cd572d rm SymbolStream<T>, change all to TokenStream. all but 5 tests pass. 2012-04-02 16:49:04 -07:00
Sam Harwell 78fcc134cd Fix error codes in unit tests 2012-04-02 18:22:05 -05:00
Sam Harwell 5fd8b223b6 Fix ErrorManager ignoring error code in messages 2012-04-02 18:13:47 -05:00
Terence Parr 186293ae86 tweak 2012-04-02 10:13:13 -07:00
Terence Parr 9ef6127982 merging from master 2012-04-01 18:13:40 -07:00
Sam Harwell adad53ee18 Fix unit tests that called TokenStream.toString (now TokenStream.getText) 2012-04-01 18:33:49 -05:00
Terence Parr e1870d16e9 added convenience getText(ctx) method to TokenStream. unlikley to need in CharStream and can get another way 2012-04-01 12:23:45 -07:00
Terence Parr 588e30bfe1 getText(...) moved to CharStream and TokenStream; cleaned up names. removed getInputString from parser. 2012-04-01 12:12:49 -07:00
Terence Parr c9aef6fdbe Merge branch 'master' into stack-graphs-integration 2012-03-31 10:33:22 -07:00
Terence Parr 9fbe9b6e21 op=(x|y) works as left-recur binary op now. 2012-03-30 13:00:31 -07:00
Sam Harwell f166df7b94 Updated unit tests to use .g4 as the preferred grammar extension 2012-03-30 11:44:34 -05:00
Sam Harwell e5d7c27b09 Updated comments to indicate preference of .g4 over .g 2012-03-30 11:37:22 -05:00
Sam Harwell cfcb2a1a55 Add Tool.GRAMMAR_EXTENSION (.g4) and Tool.LEGACY_GRAMMAR_EXTENSION (.g), allow imported grammars to use either extension (favoring .g4 over .g). 2012-03-30 11:36:20 -05:00
Terence Parr 11cd720cc6 added failing test so i go back and fix issue with op=('+'|'-') in LR rules 2012-03-29 21:37:56 -07:00
Terence Parr 42706485e9 was looking for imports with .g not .g4 2012-03-29 14:59:43 -07:00
Terence Parr 48b2e237d1 Merge branch 'no-rules-npe' of https://github.com/sharwell/antlr4 2012-03-27 21:23:47 -07:00
Sam Harwell cc7e9c1b0c Don't strip error numbers from lines because the numbers are constants of the public API 2012-03-27 21:52:34 -05:00
Sam Harwell 8d16912fb9 Fix expected output for testAmbigLR 2012-03-27 21:51:55 -05:00