e3e739dfc7
The lexer and parser ATN simulators' adaptivePredict now synchronize on the specific DFA of the decision to be simulated. This should prevent a lot of contention that would occur if we synchronize the entire adaptivePredict method. When the individual DFA are created, we also synchronize on the shared DFA[] table quickly to create a DFA and insert it into the array. Code generation modified to have _decisionToDFA generated at the top of both the parser and the lexer. Simulators created now with the recognizer, ATN, DFA[]. Not sure the LexerInterp/ParserInterp work but pushing ahead anyway for the moment.
2012-07-29 09:49:35 -07:00
109880b7ab
added ALIAS_REASSIGNMENT warning so redef of string literal among rules caught. First literal goes to .tokens.
2012-07-28 14:24:39 -07:00
3c7b4c2a33
big cleanup.
2012-07-26 17:28:10 -07:00
f7eeca274f
reorg closure and fix bug where $ in arrayctx wouldn't perform global follow. fix case in array merge that didn't check both a[i], b[i] as $ (only matters in full ctx). unit tests for graph show fewer ctx's created.
2012-07-22 16:09:45 -07:00
9539572ee7
simplify test.
2012-07-21 16:27:00 -07:00
f220212a95
couldn't get Horstmann's routine to do EPS not PS so had to backtrack.
2012-07-14 16:32:04 -07:00
1d9aef0a5e
replace .tokens file parser with regex to avoid \t becoming tab char.
2012-07-03 12:40:36 -07:00
5c69d31e88
CommonTokenFactory now knows how to copy the text out of the character stream buffer before they disappear in unbuffered character strengths; added ctor.
...
Lexer now guarantees that the text of the current token is always available to the emit() method even if the character stream is unbuffered.
Added some hooks to see some of the internal data in the unbuffered character stream so that I can test it better.
Updated LexerInterpreter so that it uses the token factory.
Improved/added unit tests for the unbuffered character string.
Updated various comments
2012-06-30 16:40:16 -07:00
740208ee4d
test code.
2012-06-17 16:56:26 -07:00
b255509e96
fix a bug related to semantic predicates in the lexer and generally cleaned up variable and method names in the simulator. I moved all of the predicates to the right side of lexer rules in the unit tests. Later, we should ensure that predicates only occur on the right edge of lexer rules. We should state that the rule is not been accepted so we can't test things like getText(), we have to use more raw indexes into the character stream. In the lexer simulator, the addDFAState() method now does not try to compute whether there is a predicate in the configurations. That information has already been set into the ATNConfigSet by the getEpsilonTarget() method. [I should also point out that I have not tested the Java parsing in a while and now it hits a landmine on a number of common Java files in jdk :(. I'm not sure where that crept in]
2012-06-07 17:31:18 -07:00
c590ba8fd8
don't look backwards for err msg if EOF is entire input. make sure we don't use -1 rule index for ruleNames[]
2012-04-29 12:12:42 -07:00
6314b7d31b
-> becomes # for alt labels
2012-04-26 11:59:57 -07:00
9c1e58db7c
add {} in primary alt block to prevent ID|INT from becoming SET, which breaks code gen needs.
2012-03-27 16:21:01 -07:00
9a0aaacbee
rm k=1 chk to report early ambiguity.
2012-03-16 14:11:21 -07:00
102980dffd
make T.g same
2012-03-14 13:20:24 -07:00
ae08867ff3
alter visitTerminal interface, add visitErrorNode.
2012-02-26 22:07:45 -08:00
8a34176d82
added listener unit tests. fixed bug that didn't create ctx getters properly for recursive rules. added Symbol extends Token to parse tree stuff. added visitTerminal to Visitor. recursive alts now track their original, unedited AltAST subtree so we can properly count rule refs etc... later. dup of RuleRefAST was making wrong node. don't gen dispatch methods if no listener.
2012-02-22 12:44:33 -08:00
a923ad8765
Major update to v4. I backed out a change I made on Christmas then mistakenly prevented any lexer DFA creation. Per http://www.antlr.org/wiki/display/~admin/2011/12/29/Flaw+in+ANTLR+v3+LL%28*%29+analysis+algorithm I fixed a major flaw in ANTLR's notion of context. To do that, I needed to create a new LoopEndState, with all of its fanout to the serialization and parser ATN construction. got a very good start on ParserATNPathFinder, which uses basic recursion to find all possible paths and return a tree with the possibilities. I left it in the condition where he would sometimes loop forever; it needs to track sets of configurations in the busy set; it using states at the moment. added a new signal from the interpreter: reportAttemptingFullContext. I fixed a bug where configuration sets derived from a configuration that had reachesIntoOuterContext>0 were not being considered as dipping into the outer context. The ambiguity checker needed to switch so that a check for exact matches not suffixes when doing full context. It's faster at the very least for full context. added some more support routines to DFA. Added TraceTree in support of the new ParserATNPathFinder.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9764]
2011-12-29 17:04:40 -08:00
299c29d927
more lexer rule specialization in parser. got antlr almost back to working with new [Aa] notation in lexer.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9753]
2011-12-26 17:09:01 -08:00
d9efffd104
Add [abc] syntax to allow set of char in lexer; args aren't allowed so unambig.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9750]
2011-12-26 15:58:40 -08:00
6fa5e52d5e
added => skip, channel(99), more, mode(xx), push(xx), pop lexer syntax. separated lexer rules from others in parser / AST now.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9749]
2011-12-26 15:14:49 -08:00
bb48deb354
tweak to dotgenerator, make parserinterp using new atn sim
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9645]
2011-12-16 18:15:56 -08:00
ebd1fbb63d
within 2 or 3 unit test of where I was before I got it the ATN simulator
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9642]
2011-12-16 15:07:28 -08:00
3d133e9417
broke out fullctx tests, some fixes.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9636]
2011-12-16 09:43:29 -08:00
5ad1505fdb
almost got new ATN engine working; separated .* nongreedy tests, reorg args on reporting methods
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9627]
2011-12-15 11:03:41 -08:00
92279bd6db
almost got prediction working
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9600]
2011-12-13 18:10:04 -08:00
131e9f7686
added comments, working on parser interpreter (not prediction) reorg. adding ParserInterpreter. adding unit tests.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9546]
2011-12-09 16:35:21 -08:00
63168c5577
added more dbg prints, turned on context sensitivity in test java lr.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9545]
2011-12-07 17:47:08 -08:00
cab803d5f3
tweak comment
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9528]
2011-12-05 09:44:40 -08:00
80fd90d363
rm'd epsilon transition in ATN after rule refs.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9518]
2011-12-03 14:52:24 -08:00
f08e8fc098
predTransition was not passing along predicate properly. rm'd unneeded parameter. pass null if ctx independent pred. added 2 unit tests for non outer ctx check.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9516]
2011-12-03 14:12:06 -08:00
ae74881de6
strip epsilons between alternative elements and also stripped the final epsilons before block ends. added an ATN visitor for general use. updated all of the unit tests so they pass new smaller ATNs. had to do some work in the serialization and deserialization to handle state numbers without state information. did not want to reorder state numbers during optimization.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9512]
2011-12-01 17:46:12 -08:00
a8219d943e
got java LR parser working :) allows e A B C e type LR alts now
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9506]
2011-11-30 18:57:45 -08:00
8d7a97fb68
fixing some pred bugs. fixed unit testSimple in pred eval. optimized sem ctx.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9503]
2011-11-30 17:44:49 -08:00
e00fa44996
Got validating and disambiguating predicates the way I want; widespread changes. DFAStates now have a list of predicate/alt pairs that they can test at except states. renamed Java.g4 to JavaLR.g4. LL(1) analysis did not fail when it's all predicates, which it must.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9482]
2011-11-29 15:46:51 -08:00
9a1a8428b5
got ambig pred thing working i think; unit tests. missing validating pred thing.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9480]
2011-11-28 19:16:45 -08:00
324884585b
add slider
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9306]
2011-11-13 11:35:11 -08:00
6c104b7724
pull apart treeviewer; rename ANTLRParserListener
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9148]
2011-10-20 19:12:32 -08:00
a1c682434d
removes the individual stacks to track rule invocations. $a::x now looks up the context stack for rule index RULE_a. they're not done very often and so it's better to say all of the overhead pushing and popping the contexts.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9066]
2011-09-11 16:12:01 -08:00
bf19465437
Got a rule labels added to the tree and also altered code generation so that it pays attention to the labels. fairly major surgery but it's a nice refactoring.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9062]
2011-09-09 17:08:13 -08:00
730794128e
parse tree visiting; now, we ALWAYS generate a return struct definition: rule_ctx.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9049]
2011-09-05 17:08:26 -08:00
5c6f35abef
did some reorganization
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9038]
2011-09-03 11:52:23 -08:00
085dd05bf1
got parser nongreedy loops going
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9032]
2011-08-12 19:51:12 -08:00
d926ec9661
WOOT! all left-recursive tests pass
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8972]
2011-07-31 18:14:47 -08:00
dcfb804ed3
added method to get context for non-arg rules
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8921]
2011-07-28 19:36:56 -08:00
cce6c70b2e
fixed context in sempred. don't exec forced actions either if in outer context and dependent.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8920]
2011-07-28 15:15:41 -08:00
3d15daccad
got predicate issue resolved. executes preds if not dependent on context and no dipping into outer context
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8919]
2011-07-28 12:43:28 -08:00
a29feac1bf
a bit of clean up
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8906]
2011-07-24 16:18:21 -08:00
66da1f723a
got unit tests working again for interp; got args funcs in output
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8902]
2011-07-24 11:48:43 -08:00
3b52eb4e6a
snapshot
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 8899]
2011-07-22 17:19:26 -08:00
Terence Parr