71b75c88dd
Fix regression; did not gen enter/exit in alt label structs.
2012-02-18 15:16:42 -08:00
c6fd897287
altered naming scheme for listeners/visitors
...
* names changed. visit() -> visitX(). enter/exit() -> enter/exitX()
* capitalizing automatically now. rule s -> SContext not sContext
* no enter/exit method in generic rule context object if rule has alt labels, nor in interfaces.
* dup labels allowed in same rule
* label X or x illegal if rule x exists
2012-02-18 12:40:47 -08:00
725b105135
Added visitor gen mechanism, runtime support
2012-02-16 17:49:57 -08:00
82aac56c98
Fix rule list labels
2012-02-16 17:35:01 -06:00
fa3a664696
got some good examples working that play around with the getter methods.
2012-02-14 16:33:12 -08:00
e30627c967
updated command line options.
2012-02-14 15:14:46 -08:00
a6786743d4
got it generating getters, labels correctly. now to impl getters
2012-02-14 13:58:48 -08:00
6c2beb530d
see A.g4 for getter syntax; use e() not gete(). adding ContextGetterDecl.java
2012-02-13 20:47:19 -08:00
203ad46820
fixed 8 unit tests; only 9 fail now.
2012-02-13 17:47:26 -08:00
0031075da2
Merge branch 'master' into enhance-ctx
2012-02-13 17:36:56 -08:00
7b672c31da
Merge pull request #11 from sharwell/clean-tests
...
Clean tests
2012-02-13 15:38:39 -08:00
146a9b0509
Fix unit test placing output files like "null.java" in the project directory
2012-02-13 16:32:42 -06:00
b68ce1f5aa
almost got ctx stuff working
2012-02-13 11:47:26 -08:00
97ab2c42da
mode wasn't working as lexer command
2012-02-13 11:45:10 -08:00
3dbf4cdabc
mid-job push so i can pull from work to continue. working on updating ctx objects to have getters
2012-02-13 09:15:54 -08:00
f8285bcca0
another useful tuple
2012-02-13 09:15:05 -08:00
dee579a68f
oops; need to alter this when removing up/down stuff
2012-02-11 17:46:02 -08:00
808d23e6d1
add some useful classes
2012-02-11 17:42:02 -08:00
b80b22e4fa
rm remnants of tree parser
2012-02-11 17:39:50 -08:00
4f3e40f0d3
Use List instead of Vector
2012-02-08 19:06:32 -06:00
5279b250ca
Use isEmpty() instead of size()==0, use StringBuilder instead of StringBuffer, use chained append() calls instead of string concatenation, remove unnecessary initializations.
2012-02-08 19:01:28 -06:00
5c79fc600a
["\\] didn't work in lexer
2012-02-07 13:19:12 -08:00
894a77168d
rename atn sim, add error message to TestRig
2012-02-05 10:14:04 -08:00
c47c07299a
ugh. fix "[a-z]" for real now
2012-02-02 10:41:27 -08:00
c94037def5
fixed regression "a returns [int i] : A ;" didn't work.
2012-02-01 15:55:49 -08:00
5eb9ec9582
fixed bug in [a-z]
2012-01-30 18:08:22 -08:00
e90b322dd4
~[] stuff is allowed and works inside sets etc...
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9926]
2012-01-28 14:36:46 -08:00
368b70d53c
get 'x' not x in tokenNames
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9925]
2012-01-28 11:28:42 -08:00
f4ecc75df5
allows " in [] char set now.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9923]
2012-01-28 10:46:08 -08:00
043afd3767
shelve for pull at home; don't sync ;)
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9914]
2012-01-27 15:38:11 -08:00
15d537ce6e
Added ranges, escapes to [a-z] notation in lexer
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9897]
2012-01-22 11:37:15 -08:00
89ad588023
fix regression
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9896]
2012-01-22 09:55:13 -08:00
9fd280b30e
Lexer fields mv from x to _x
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9883]
2012-01-21 15:36:20 -08:00
449c845dc0
more cleanup
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9882]
2012-01-21 15:29:33 -08:00
fd0d280e6a
cleanup
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9881]
2012-01-21 15:23:39 -08:00
4632d7f493
rule name change
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9880]
2012-01-21 15:16:00 -08:00
dc82edad02
Summary: we can have lexer commands like -> skip now.
...
moved model.actions package to chunk
type(foo) didn't match.
ATNBuilder.g now tracks the outer alternative number and calls new factory commands on the lexer commands.
Removed unnecessary resolveWithPredicate field from ATNConfig
Added lexerActionIndex field to ATNConfig since we need to track whether we passed an action in an alternative will rule in the lexer.
Renamed ruleIndex in DFAState and added the lexer action index so that we can execute lexer actions from the DFA.
added functions to the grammar tree visitor for the lexer commands.
Added templates for the lexer commands.
Augmented the lexer ATN factory so that it constructs plain old actions from the lexer commands it finds. That way, the code generator doesn't know any different and generates an action.
Augmented the lexer ATN simulator so that it fires a proper action index now. previously it only used the rule index, which of course doesn't work when you have more than one action in a rule.
rm'd dup code from OutputModelController
altered the epsilon edge removal optimization so that it could not remove actions in lexer rules.
Added list of valid lexer commands in Rule.
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9877]
2012-01-21 15:01:05 -08:00
d321bb7854
fixed comment
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9876]
2012-01-21 14:49:20 -08:00
b90d41b508
wasn't walking lexer rule trees right.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9874]
2012-01-20 15:12:51 -08:00
ec5e3e51b1
rm ForcedAction
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9873]
2012-01-20 15:12:32 -08:00
491744f893
reuse of -> label on multiple alts in rule caused dup ctx object defs.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9868]
2012-01-14 13:12:35 -08:00
1ddf609e25
labels on tokens in left-recursive rules caused codegen exception.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9866]
2012-01-14 10:18:19 -08:00
848c6e4313
This change is a major restructuring of how left recursive rules are transformed. Previously I simply rewrote the grammar and ANTLR was none the wiser. However, it quickly became apparent that ANTLR needed to do many different things for recursive rules so I had to insert transformation later in the pipeline. Specifically, I needed it after Rule object creation. I made a special LeftRecursiveRule object that track information collected during transformation so that I could use it later. I made major changes to the left recursion templates as well as the little code snippets in Java.stg. I created a new template called LeftRecursiveRuleFunction an accompanying model object that handles the special case, even though there is some duplication. the biggest difference in the grammar is the introduction of => ID notation on any outermost alternative. This information is not added to the tree, instead the ALT node is annotated with the ID. Rule.getAltLabels() now looks also in the LeftRecursiveRuleAltInfo objects. I have moved to the left recursion transformation to its own object and have moved some objects into the analysis package. Further, I have split out the Rule object creation into its own RuleCollector. I renamed discoverAlt in the grammar tree visitor to be discoverOuterAlt an added discoverAlt so we can get information about individual alts even inside subrules. Listeners always get an event for the generic rule context, which is used if there is no specific label for an alternative. Added a list of iteration operations for LL(*) subrules. Split buildRuleFunction into buildLeftRecursiveRuleFunction and one for normal rule function creation. I have to insert lots of extra code to manage the contexts, but of course it's all done using the templates. As long as those templates are correct, this code generation mechanism will work. I removed the st field from the parser rule context. I injected the left recursion transformation inside the SemanticPipeline. Visitor dispatch methods are always added to the generated context structures. Fixed some unit tests. About 24 fail.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9854]
2012-01-11 11:00:05 -08:00
742a08862f
Merging dev changes to main
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9853]
2012-01-09 10:26:23 -08:00
e8ae8ee005
labels allowed in left recursive rule refs.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9834]
2012-01-05 18:35:20 -08:00
4ae23f4a64
* 1+2*3 now gives new parse tree: (e (e 1) + (e (e 2) * (e 3))) See CHANGES.txt now too
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9833]
2012-01-05 12:27:14 -08:00
2e8a5f391a
no more -trace
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9826]
2012-01-04 17:41:42 -08:00
7d68546dbe
rm prints
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9824]
2012-01-04 17:37:15 -08:00
e7f6ba5ac1
lr rules now gen only 1 rule e->e not e->e_ etc... altered tests to build parse trees.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9823]
2012-01-04 17:36:24 -08:00
2d9139ca4d
remove % from ActionSplitter.g
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9790]
2012-01-03 11:04:12 -08:00
5c3c8d6e7a
Added TokenFactory, CommonTokenFactory. Update the parser in the lexer with methods to set the factory. Alter the default error strategy and the lexer to use the factory. The parser's set token factory method updates the token source, usually the lexer, and the error handling strategy. I had to add the set token factory method to token source as well to make all of this work.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9789]
2012-01-03 10:58:01 -08:00
1b04195a2f
removed all template / AST rewrite stuff; massive change; added -encoding tool option
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9780]
2012-01-02 18:13:16 -08:00
3d620734af
Merging CL9604 to main (optimized string literals in generated code)
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9777]
2012-01-02 12:08:18 -08:00
353ddc6c2b
using wrong map class
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9767]
2012-01-01 10:27:51 -08:00
dff466be66
literal rules like A:'a' not seen in modes; weren't in .tokens files
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9766]
2011-12-31 17:26:48 -08:00
a923ad8765
Major update to v4. I backed out a change I made on Christmas then mistakenly prevented any lexer DFA creation. Per http://www.antlr.org/wiki/display/~admin/2011/12/29/Flaw+in+ANTLR+v3+LL%28*%29+analysis+algorithm I fixed a major flaw in ANTLR's notion of context. To do that, I needed to create a new LoopEndState, with all of its fanout to the serialization and parser ATN construction. got a very good start on ParserATNPathFinder, which uses basic recursion to find all possible paths and return a tree with the possibilities. I left it in the condition where he would sometimes loop forever; it needs to track sets of configurations in the busy set; it using states at the moment. added a new signal from the interpreter: reportAttemptingFullContext. I fixed a bug where configuration sets derived from a configuration that had reachesIntoOuterContext>0 were not being considered as dipping into the outer context. The ambiguity checker needed to switch so that a check for exact matches not suffixes when doing full context. It's faster at the very least for full context. added some more support routines to DFA. Added TraceTree in support of the new ParserATNPathFinder.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9764]
2011-12-29 17:04:40 -08:00
9853061093
added null chk
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9759]
2011-12-27 14:32:19 -08:00
299c29d927
more lexer rule specialization in parser. got antlr almost back to working with new [Aa] notation in lexer.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9753]
2011-12-26 17:09:01 -08:00
3fae7cc880
rm unnecessary code
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9751]
2011-12-26 16:56:51 -08:00
d9efffd104
Add [abc] syntax to allow set of char in lexer; args aren't allowed so unambig.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9750]
2011-12-26 15:58:40 -08:00
6fa5e52d5e
added => skip, channel(99), more, mode(xx), push(xx), pop lexer syntax. separated lexer rules from others in parser / AST now.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9749]
2011-12-26 15:14:49 -08:00
6daa66f83f
no more resolved bit in config. use ATNConfigSet all over. final clean up of ATN sim.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9741]
2011-12-25 12:01:24 -08:00
bb48deb354
tweak to dotgenerator, make parserinterp using new atn sim
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9645]
2011-12-16 18:15:56 -08:00
c32b703555
quote literal was not translated properly. updated Lexer to convert antlr literals to target literals properly. added unit test.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9557]
2011-12-11 10:37:27 -08:00
d170d89ce6
wasn't using some args
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9547]
2011-12-09 18:15:05 -08:00
131e9f7686
added comments, working on parser interpreter (not prediction) reorg. adding ParserInterpreter. adding unit tests.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9546]
2011-12-09 16:35:21 -08:00
6c5f3655b9
dropped another eps edge; like 5% savings for JavaLR.g4
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9543]
2011-12-07 13:07:04 -08:00
6e79c36861
add verbose dfa state option
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9537]
2011-12-06 12:09:19 -08:00
edc4224b56
v4: updated atn graphs
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9531]
2011-12-06 10:46:33 -08:00
265a189a86
crap. fixing regression. was adding way too many followlinks.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9527]
2011-12-04 19:01:55 -08:00
2ddb8100e9
shrunk serialized ATN a bit
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9526]
2011-12-04 18:39:20 -08:00
80fd90d363
rm'd epsilon transition in ATN after rule refs.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9518]
2011-12-03 14:52:24 -08:00
beec4ab721
v4: Back out changelist 9511 (change DFAState.configs to ATNConfig[])
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9513]
2011-12-02 10:04:34 -08:00
ae74881de6
strip epsilons between alternative elements and also stripped the final epsilons before block ends. added an ATN visitor for general use. updated all of the unit tests so they pass new smaller ATNs. had to do some work in the serialization and deserialization to handle state numbers without state information. did not want to reorder state numbers during optimization.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9512]
2011-12-01 17:46:12 -08:00
682e60b065
v4: DFAState.configs now ATNConfig[] instead of OrderedHashSet<ATNConfig>
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9511]
2011-12-01 17:04:25 -08:00
4400ba169f
refactored RuleContext to move all noncritical fields down into ParserRuleContext. Widespread but minor changes. Using the more specific ParserRuleContext where appropriate.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9509]
2011-12-01 14:01:00 -08:00
38c69af805
v4: Back out changelist 9507 (PredictionContext optimization)
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9508]
2011-12-01 13:14:37 -08:00
11b6fa13f9
v4: PredictionContext optimization
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9507]
2011-12-01 13:13:24 -08:00
a8219d943e
got java LR parser working :) allows e A B C e type LR alts now
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9506]
2011-11-30 18:57:45 -08:00
ad9259fc4f
bad attribute in error msg
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9504]
2011-11-30 18:00:43 -08:00
0bf1aca47f
cleaned up a bit so that TestToolSyntaxErrors works. added some cast checks and some bailout upon error before semantic processing
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9483]
2011-11-29 15:59:23 -08:00
e00fa44996
Got validating and disambiguating predicates the way I want; widespread changes. DFAStates now have a list of predicate/alt pairs that they can test at except states. renamed Java.g4 to JavaLR.g4. LL(1) analysis did not fail when it's all predicates, which it must.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9482]
2011-11-29 15:46:51 -08:00
3b7e984d5f
mv join to runtime Utils
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9477]
2011-11-27 14:28:59 -08:00
21e9f67927
rm print
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9471]
2011-11-26 13:07:08 -08:00
c2145a4120
didn't store parser::members in combined grammar
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9470]
2011-11-26 13:04:15 -08:00
a81ab4cc64
rm more invalid options
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9464]
2011-11-26 10:46:54 -08:00
d2b24da47f
remove incidentTransition from ATNState, add computation for next tokens within rule, add nextTokenWithinRule to ATNState, add EPSILON as -2 token type
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9459]
2011-11-25 16:09:00 -08:00
06b27b8244
added stub for ID*[','] if we want that syntax
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9455]
2011-11-25 12:13:29 -08:00
3b543098aa
refactor grammar transform pipeline
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9454]
2011-11-25 11:55:13 -08:00
7a52d4e304
Back out changelist 9448
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9449]
2011-11-23 12:25:06 -08:00
898138a5e2
v4: Reusable ATN simulator objects
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9448]
2011-11-23 06:42:31 -08:00
d0230aba5b
rm legacy options from grammar, rule, block
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9444]
2011-11-22 18:23:08 -08:00
2b36a19739
rm Rule.scope field. not doing scope {...} anymore
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9442]
2011-11-22 16:48:20 -08:00
72684907bc
add -Xlog, add .ctx to rule refs. parse tree test pass.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9436]
2011-11-22 16:30:50 -08:00
95f3afa562
rm ANTLRStringStream. combined with ANTLRInputStream. causes name change in lots of files. rm encoding ability.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9435]
2011-11-22 15:50:19 -08:00
09ad4bcb0a
v4: Remove dependency on v3 tool (still depends on v3 runtime)
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9431]
2011-11-22 15:32:53 -08:00
cfecf4b109
add CANNOT_FIND_ATTRIBUTE_NAME_IN_DECL
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9429]
2011-11-22 15:26:41 -08:00
0d691db980
added uniq error codes
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9426]
2011-11-22 14:29:06 -08:00
dcedc63dec
added missing error msg
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9422]
2011-11-21 17:22:21 -08:00
4b4a7d9c7a
add LogManager, add InputSymbolType()
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9413]
2011-11-20 14:27:58 -08:00
Terence Parr
Sam Harwell
Terence Parr
sharwell