e4a9a44671
grammar option cleanup. was a mess. -Doption=value works to override grammar options on cmd-line now.
2012-09-05 18:37:28 -07:00
201db8b6d0
merge sam's pulls
2012-09-04 18:59:20 -07:00
1eb258b8f8
tweak test
2012-08-27 11:25:54 -07:00
60d99e62dc
rm ParseListener; tested the tracer with left recursive rules; weird but deterministic for entry events.
2012-08-27 11:22:42 -07:00
e33e355d66
tweak comment
2012-08-26 16:32:28 -07:00
ad737ebdf6
tweak
2012-08-26 16:03:59 -07:00
64c050f233
sets at top level work now: s : A | B ; in lexer and parser.
2012-08-05 10:17:00 -07:00
799b7afc1c
playwith grammar
2012-08-05 09:51:52 -07:00
8637f31837
rm prints
2012-08-05 09:21:59 -07:00
1bec176eaa
Impl Sam's no viable alt avoidance idea that chooses min alt that dips into outer context. unit test
2012-08-04 14:18:57 -07:00
4090621beb
same literal different modes gens no literal in .tokens, rm warning
2012-08-04 13:51:18 -07:00
c2b49bd94e
pull in Sam's ATN alt collapsing optimizations with optimizeStates off as it causes a class cast exception.
2012-08-04 13:32:07 -07:00
e7b65057a6
added var for sll loop tail recursion default value; updated unit tests
2012-08-03 18:12:52 -07:00
0cfe3b4206
tweak to leave code for printing DFA DOT
2012-08-01 14:36:19 -07:00
489f3d17cb
tweak
2012-07-29 17:04:05 -07:00
8836e13954
use ReentrantReadWriteLock to sync. works much better but still occasional illegal monitor state :(
2012-07-29 17:01:41 -07:00
4fe9efeb85
tweak tester
2012-07-29 14:28:59 -07:00
c97f36bd89
reorg workers
2012-07-29 13:45:41 -07:00
230b687b03
add barrier
2012-07-29 13:03:10 -07:00
5712424156
create 2 worker threads for testjavalr
2012-07-29 12:28:29 -07:00
7181483bbf
test javalr does -2x now
2012-07-29 10:57:51 -07:00
510c1c0dd9
clean up
2012-07-29 10:50:14 -07:00
79594a942e
synchronized lexer. just as fast now to build new parser/lexer each iter in TestJavaLR
2012-07-29 10:48:08 -07:00
e3e739dfc7
The lexer and parser ATN simulators' adaptivePredict now synchronize on the specific DFA of the decision to be simulated. This should prevent a lot of contention that would occur if we synchronize the entire adaptivePredict method. When the individual DFA are created, we also synchronize on the shared DFA[] table quickly to create a DFA and insert it into the array. Code generation modified to have _decisionToDFA generated at the top of both the parser and the lexer. Simulators created now with the recognizer, ATN, DFA[]. Not sure the LexerInterp/ParserInterp work but pushing ahead anyway for the moment.
2012-07-29 09:49:35 -07:00
109880b7ab
added ALIAS_REASSIGNMENT warning so redef of string literal among rules caught. First literal goes to .tokens.
2012-07-28 14:24:39 -07:00
26165c5d54
atnconfigsets can be readonly now. no copying to put into new DFA state, set readonly instead. made atnconfigset.equals() more accurate. add -notree option to test java lr
2012-07-27 10:34:59 -07:00
3c7b4c2a33
big cleanup.
2012-07-26 17:28:10 -07:00
48ddc0bcee
add gc()
2012-07-26 12:42:10 -07:00
86a4f26596
still has dfa and local predict ctx cache, but doesn't help much, and takes time. Eval preds as we pass them in full ctx mode; no sempred ctx in configs then. removed LANDMINE!!!!!!!!! some lame code that uses fields like _startIndex; fix that. added bail option to testjavalr. 18s to parse java.* with javalr full ctx now.
2012-07-25 18:47:34 -07:00
3774c5752b
add more tracking var
2012-07-25 10:30:19 -07:00
3cec976f7e
Don't pursue the closure if there is just one state or it's predicting unique alt. Fixed up computeReachSet() to do this. Dropped from 6s to 4.5s for TestJavaLR -SLL on java.* with 45M max footprint.
2012-07-24 18:45:23 -07:00
2495ce6b20
add SLL option to the ATN simulator and use in TestJavaLR.
2012-07-24 17:42:47 -07:00
2ae4d4eebd
add ctx cache to merge op. now i get optimal graph it seems. doesn't seem longer for java.* with JavaLR. still landmine in Pattern.java, Foo.java. Made toDOTString sort predctx nodes by id for repeatable tests. updated unit tests.
2012-07-24 16:26:01 -07:00
ac4f00524e
add ctx cache.
2012-07-24 14:19:43 -07:00
00c4f98134
add test file that kills antlr
2012-07-23 10:35:02 -07:00
f7eeca274f
reorg closure and fix bug where $ in arrayctx wouldn't perform global follow. fix case in array merge that didn't check both a[i], b[i] as $ (only matters in full ctx). unit tests for graph show fewer ctx's created.
2012-07-22 16:09:45 -07:00
9539572ee7
simplify test.
2012-07-21 16:27:00 -07:00
f220212a95
couldn't get Horstmann's routine to do EPS not PS so had to backtrack.
2012-07-14 16:32:04 -07:00
1d9aef0a5e
replace .tokens file parser with regex to avoid \t becoming tab char.
2012-07-03 12:40:36 -07:00
5c69d31e88
CommonTokenFactory now knows how to copy the text out of the character stream buffer before they disappear in unbuffered character strengths; added ctor.
...
Lexer now guarantees that the text of the current token is always available to the emit() method even if the character stream is unbuffered.
Added some hooks to see some of the internal data in the unbuffered character stream so that I can test it better.
Updated LexerInterpreter so that it uses the token factory.
Improved/added unit tests for the unbuffered character string.
Updated various comments
2012-06-30 16:40:16 -07:00
740208ee4d
test code.
2012-06-17 16:56:26 -07:00
b255509e96
fix a bug related to semantic predicates in the lexer and generally cleaned up variable and method names in the simulator. I moved all of the predicates to the right side of lexer rules in the unit tests. Later, we should ensure that predicates only occur on the right edge of lexer rules. We should state that the rule is not been accepted so we can't test things like getText(), we have to use more raw indexes into the character stream. In the lexer simulator, the addDFAState() method now does not try to compute whether there is a predicate in the configurations. That information has already been set into the ATNConfigSet by the getEpsilonTarget() method. [I should also point out that I have not tested the Java parsing in a while and now it hits a landmine on a number of common Java files in jdk :(. I'm not sure where that crept in]
2012-06-07 17:31:18 -07:00
c590ba8fd8
don't look backwards for err msg if EOF is entire input. make sure we don't use -1 rule index for ruleNames[]
2012-04-29 12:12:42 -07:00
6314b7d31b
-> becomes # for alt labels
2012-04-26 11:59:57 -07:00
e5d7c27b09
Updated comments to indicate preference of .g4 over .g
2012-03-30 11:37:22 -05:00
9c1e58db7c
add {} in primary alt block to prevent ID|INT from becoming SET, which breaks code gen needs.
2012-03-27 16:21:01 -07:00
6791bf60cf
Merge commit '3f1f76d' into parrt-default-error-listener
...
Conflicts:
runtime/Java/src/org/antlr/v4/runtime/ANTLRErrorStrategy.java
runtime/Java/src/org/antlr/v4/runtime/DefaultErrorStrategy.java
runtime/Java/src/org/antlr/v4/runtime/DiagnosticErrorListener.java
runtime/Java/src/org/antlr/v4/runtime/atn/ParserATNSimulator.java
2012-03-24 16:44:47 -07:00
9a0aaacbee
rm k=1 chk to report early ambiguity.
2012-03-16 14:11:21 -07:00
102980dffd
make T.g same
2012-03-14 13:20:24 -07:00
6887c86d5e
shelve
2012-03-14 12:04:18 -07:00
3f1f76df7d
Move reportAmbiguity, reportContextSensitivity, reportAttemptingFullContext, reportInsufficientPredicates from ANTLRErrorStrategy to ANTLRErrorListener.
...
Add BaseErrorListener to allow implementing ANTLRErrorListener without implementing every method (e.g. ConsoleErrorListener).
DiagnosticErrorStrategy is now DiagnosticErrorListener, updated tests.
2012-03-12 15:07:48 -05:00
ae08867ff3
alter visitTerminal interface, add visitErrorNode.
2012-02-26 22:07:45 -08:00
8a34176d82
added listener unit tests. fixed bug that didn't create ctx getters properly for recursive rules. added Symbol extends Token to parse tree stuff. added visitTerminal to Visitor. recursive alts now track their original, unedited AltAST subtree so we can properly count rule refs etc... later. dup of RuleRefAST was making wrong node. don't gen dispatch methods if no listener.
2012-02-22 12:44:33 -08:00
642177f716
forgot to add dispatch methods in context objects. in-parse listeners should work now.
2012-02-19 11:25:15 -08:00
4e8931519c
added -parse-listener option and generated code if option on. parse listener differs from tree listener
2012-02-18 16:49:05 -08:00
71b75c88dd
Fix regression; did not gen enter/exit in alt label structs.
2012-02-18 15:16:42 -08:00
c6fd897287
altered naming scheme for listeners/visitors
...
* names changed. visit() -> visitX(). enter/exit() -> enter/exitX()
* capitalizing automatically now. rule s -> SContext not sContext
* no enter/exit method in generic rule context object if rule has alt labels, nor in interfaces.
* dup labels allowed in same rule
* label X or x illegal if rule x exists
2012-02-18 12:40:47 -08:00
725b105135
Added visitor gen mechanism, runtime support
2012-02-16 17:49:57 -08:00
3d56d40975
update playground
2012-02-14 16:33:34 -08:00
fa3a664696
got some good examples working that play around with the getter methods.
2012-02-14 16:33:12 -08:00
b8f50d46e7
update playground
2012-02-14 13:59:53 -08:00
6c2beb530d
see A.g4 for getter syntax; use e() not gete(). adding ContextGetterDecl.java
2012-02-13 20:47:19 -08:00
2091f2d001
change enterRule to enter in listener interface
2012-02-13 17:35:29 -08:00
8af1e23461
adding tests
2012-02-13 09:16:29 -08:00
59e4a54faa
alter tests for new parser atn name
2012-02-07 13:18:51 -08:00
5eb9ec9582
fixed bug in [a-z]
2012-01-30 18:08:22 -08:00
e90b322dd4
~[] stuff is allowed and works inside sets etc...
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9926]
2012-01-28 14:36:46 -08:00
043afd3767
shelve for pull at home; don't sync ;)
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9914]
2012-01-27 15:38:11 -08:00
15d537ce6e
Added ranges, escapes to [a-z] notation in lexer
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9897]
2012-01-22 11:37:15 -08:00
884472358d
-> popMode was broken
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9893]
2012-01-21 21:21:19 -08:00
9fd280b30e
Lexer fields mv from x to _x
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9883]
2012-01-21 15:36:20 -08:00
9ae5049d44
play with tests
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9879]
2012-01-21 15:15:01 -08:00
1ddf609e25
labels on tokens in left-recursive rules caused codegen exception.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9866]
2012-01-14 10:18:19 -08:00
73793898a9
update grammars
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9856]
2012-01-11 11:21:05 -08:00
e7f6ba5ac1
lr rules now gen only 1 rule e->e not e->e_ etc... altered tests to build parse trees.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9823]
2012-01-04 17:36:24 -08:00
1d6bbb08c3
kill
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9793]
2012-01-03 12:30:21 -08:00
1b04195a2f
removed all template / AST rewrite stuff; massive change; added -encoding tool option
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9780]
2012-01-02 18:13:16 -08:00
ddb68aa948
new example
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9778]
2012-01-02 14:07:11 -08:00
a923ad8765
Major update to v4. I backed out a change I made on Christmas then mistakenly prevented any lexer DFA creation. Per http://www.antlr.org/wiki/display/~admin/2011/12/29/Flaw+in+ANTLR+v3+LL%28*%29+analysis+algorithm I fixed a major flaw in ANTLR's notion of context. To do that, I needed to create a new LoopEndState, with all of its fanout to the serialization and parser ATN construction. got a very good start on ParserATNPathFinder, which uses basic recursion to find all possible paths and return a tree with the possibilities. I left it in the condition where he would sometimes loop forever; it needs to track sets of configurations in the busy set; it using states at the moment. added a new signal from the interpreter: reportAttemptingFullContext. I fixed a bug where configuration sets derived from a configuration that had reachesIntoOuterContext>0 were not being considered as dipping into the outer context. The ambiguity checker needed to switch so that a check for exact matches not suffixes when doing full context. It's faster at the very least for full context. added some more support routines to DFA. Added TraceTree in support of the new ParserATNPathFinder.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9764]
2011-12-29 17:04:40 -08:00
3a3ed27e60
add some tests
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9763]
2011-12-28 19:49:37 -08:00
52396d8f5f
removed method call
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9762]
2011-12-27 16:31:09 -08:00
24c572d055
oops. antlr was ok, it was javalr grammar.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9754]
2011-12-26 17:14:29 -08:00
299c29d927
more lexer rule specialization in parser. got antlr almost back to working with new [Aa] notation in lexer.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9753]
2011-12-26 17:09:01 -08:00
d9efffd104
Add [abc] syntax to allow set of char in lexer; args aren't allowed so unambig.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9750]
2011-12-26 15:58:40 -08:00
6fa5e52d5e
added => skip, channel(99), more, mode(xx), push(xx), pop lexer syntax. separated lexer rules from others in parser / AST now.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9749]
2011-12-26 15:14:49 -08:00
6daa66f83f
no more resolved bit in config. use ATNConfigSet all over. final clean up of ATN sim.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9741]
2011-12-25 12:01:24 -08:00
bb48deb354
tweak to dotgenerator, make parserinterp using new atn sim
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9645]
2011-12-16 18:15:56 -08:00
6b673a3b2e
track stats properly
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9643]
2011-12-16 16:00:11 -08:00
ebd1fbb63d
within 2 or 3 unit test of where I was before I got it the ATN simulator
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9642]
2011-12-16 15:07:28 -08:00
3d133e9417
broke out fullctx tests, some fixes.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9636]
2011-12-16 09:43:29 -08:00
5ad1505fdb
almost got new ATN engine working; separated .* nongreedy tests, reorg args on reporting methods
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9627]
2011-12-15 11:03:41 -08:00
92279bd6db
almost got prediction working
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9600]
2011-12-13 18:10:04 -08:00
b379a56c7f
was playing with test java
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9561]
2011-12-11 13:02:58 -08:00
131e9f7686
added comments, working on parser interpreter (not prediction) reorg. adding ParserInterpreter. adding unit tests.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9546]
2011-12-09 16:35:21 -08:00
63168c5577
added more dbg prints, turned on context sensitivity in test java lr.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9545]
2011-12-07 17:47:08 -08:00
cab803d5f3
tweak comment
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9528]
2011-12-05 09:44:40 -08:00
80fd90d363
rm'd epsilon transition in ATN after rule refs.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9518]
2011-12-03 14:52:24 -08:00
f08e8fc098
predTransition was not passing along predicate properly. rm'd unneeded parameter. pass null if ctx independent pred. added 2 unit tests for non outer ctx check.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9516]
2011-12-03 14:12:06 -08:00
ae74881de6
strip epsilons between alternative elements and also stripped the final epsilons before block ends. added an ATN visitor for general use. updated all of the unit tests so they pass new smaller ATNs. had to do some work in the serialization and deserialization to handle state numbers without state information. did not want to reorder state numbers during optimization.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9512]
2011-12-01 17:46:12 -08:00
a8219d943e
got java LR parser working :) allows e A B C e type LR alts now
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9506]
2011-11-30 18:57:45 -08:00
Terence Parr
Sam Harwell