03d7b51d3c
throw exception upon bad token indexes, add getOffChannelTokensToLeft, right. pull up some off channel token methods from CommonTokenStream. renamed.
2012-05-19 14:22:52 -07:00
7be2ba9d42
remove unused type param
2012-05-19 13:06:27 -07:00
d46c7db52c
merge master
2012-05-15 12:19:48 -07:00
c590ba8fd8
don't look backwards for err msg if EOF is entire input. make sure we don't use -1 rule index for ruleNames[]
2012-04-29 12:12:42 -07:00
3ea6a5cb2f
Merge branch 'master' into stack-graphs-integration
...
Conflicts:
tool/playground/T.g
2012-04-26 12:01:28 -07:00
30f61b17e3
comment tweak
2012-04-15 18:30:28 -07:00
f5501979c6
Merge branch 'master' into stack-graphs-integration
2012-04-09 10:56:14 -07:00
35aeeb7c5c
turn on building parse trees by default since we do -listener by default.
2012-04-07 11:27:18 -07:00
d22eb34fe7
Merge branch 'master' into stack-graphs-integration
2012-04-05 16:35:56 -07:00
1ade15392b
Add license header
2012-04-04 15:33:39 -05:00
e11dc07275
Remove TokenRewriteStream
2012-04-04 15:33:03 -05:00
08e3ddefc0
Merge branch 'master' into stack-graphs-integration
2012-04-04 13:22:00 -07:00
585aa0a14b
Merge branch 'tokenstreamrewriter' of git://github.com/sharwell/antlr4
2012-04-04 13:20:58 -07:00
ff2655d379
Rename getSource() to getTokenStream()
2012-04-04 15:20:45 -05:00
bbd61dca35
Remove TokenStreamRewriter.getOriginalText() (use rewriter.getSource().getText() instead)
2012-04-04 10:15:29 -05:00
a85f1c17a1
Add TokenStreamRewriter.getSource()
2012-04-04 10:14:09 -05:00
e4612935f2
Only need TokenStream (was overly restricted to BufferedTokenStream)
2012-04-04 10:10:33 -05:00
16046a7ec1
Proper maven reference for org.abego.treelayout.core dependency
2012-04-04 09:52:06 -05:00
bf9c7c3a28
Remove old TokenRewriteStream in favor of new TokenStreamRewriter that is not a stream; it feeds off of the stream to provide transformations of that original stream. Updated the comment at the top of the file. Instead of toString() we use getText(). Altered the unit tests to create a buffered stream and then create the rewriter attached to that.
2012-04-02 18:10:32 -07:00
3849cd572d
rm SymbolStream<T>, change all to TokenStream. all but 5 tests pass.
2012-04-02 16:49:04 -07:00
9ef6127982
merging from master
2012-04-01 18:13:40 -07:00
53c5aecb85
Implement UnbufferedTokenStream.get(int) within marked intervals
2012-04-01 18:23:32 -05:00
8ec9ba14c9
Implement getText within marked intervals of unbuffered streams
2012-04-01 18:22:43 -05:00
d6e2e41c1f
Fix stop index
2012-04-01 14:29:06 -05:00
e1870d16e9
added convenience getText(ctx) method to TokenStream. unlikley to need in CharStream and can get another way
2012-04-01 12:23:45 -07:00
588e30bfe1
getText(...) moved to CharStream and TokenStream; cleaned up names. removed getInputString from parser.
2012-04-01 12:12:49 -07:00
f238d7579e
added getText() to ParseTree. getText(tokens) is still really what you want but convenient for ctx.type().getText() when it's just one token.
2012-03-31 17:27:11 -07:00
cc20a52cdd
allow special "tokens" start rule name so we can test pure lexer grammars.
2012-03-31 15:54:00 -07:00
c9aef6fdbe
Merge branch 'master' into stack-graphs-integration
2012-03-31 10:33:22 -07:00
e5d7c27b09
Updated comments to indicate preference of .g4 over .g
2012-03-30 11:37:22 -05:00
447a5620f0
add -diagnostics option to test rig.
2012-03-28 11:07:23 -07:00
a363e70582
pull from master
2012-03-27 16:22:01 -07:00
bf07f7907f
Merge branch 'override-ruleindex' of github.com:sharwell/antlr4
2012-03-27 15:09:50 -07:00
41e97791bb
Merge branch 'set-stop-token' of github.com:sharwell/antlr4
2012-03-27 15:08:57 -07:00
f307945d2a
Merge branch 'lexer-npe' of github.com:sharwell/antlr4
2012-03-27 15:06:17 -07:00
8465920343
Merge branch 'atnstate-reachability-test' of github.com:sharwell/antlr4
2012-03-27 15:05:39 -07:00
169f58a3ff
fix def of EMPTY->INVALID
2012-03-25 21:19:39 -07:00
df2e24a228
Override getRuleIndex() in generated context objects so ParserRuleContext.ruleIndex field can be removed
2012-03-25 21:50:55 -05:00
bdda174af6
Set stop token even when RecognitionException occurs. Add ParserRuleContext.exception field to hold exception if one occurs. Resolves antlr/antlr4#49 .
2012-03-25 21:50:12 -05:00
2b5896e83b
Merge branch 'master' into stack-graphs-integration
2012-03-25 10:08:29 -07:00
542e700644
fix antlr/antlr4/#48 (getText from parse tree node). Don't NPE when start/stop null in getSourceInterval.
2012-03-25 10:01:27 -07:00
4ea3c73d1f
cleaned up interval stuff. moved getSourceInterval to ParserRuleContext; renamed create to of() in Interval.
2012-03-24 18:17:35 -07:00
c6365fb5e2
removed config list (huge) from default message.
2012-03-24 16:56:36 -07:00
e228e88ac5
tweak to remove insuff pred method
2012-03-24 16:48:26 -07:00
6791bf60cf
Merge commit '3f1f76d' into parrt-default-error-listener
...
Conflicts:
runtime/Java/src/org/antlr/v4/runtime/ANTLRErrorStrategy.java
runtime/Java/src/org/antlr/v4/runtime/DefaultErrorStrategy.java
runtime/Java/src/org/antlr/v4/runtime/DiagnosticErrorListener.java
runtime/Java/src/org/antlr/v4/runtime/atn/ParserATNSimulator.java
2012-03-24 16:44:47 -07:00
b1bcde76b7
Merge commit '768bfc0' into parrt-default-error-listener
2012-03-24 16:40:09 -07:00
b035ceec9e
EMPTY gives "" not "$" now. rename closure->closure_. had return in wrong spot in closure for-loop. fix fromRuleContext.
2012-03-20 19:06:52 -07:00
0f969af947
added in Sam's popAll to deal with loop end push/pop stuff. passes LR tests and all but 2 sempred tests. does some JavaLR.g4 parsing too.
2012-03-20 18:06:00 -07:00
9318391664
tweak to fromRuleContext. more sempred tests work.
2012-03-20 12:32:28 -07:00
55ed1a45bf
lexer wasn't using new prediction context. weird. put in functionality for $ merge in full ctx mode. removed configs array in ATNConfigSet. ATNConfigSet wasn't tracking lexerActionIndex from config. ATNConfigSet now maps key to ATNConfig.
2012-03-20 12:12:18 -07:00
5945b29a86
got more working
2012-03-19 19:58:04 -07:00
7e9a86a3e1
got TestATNInterpreter working
2012-03-19 18:47:41 -07:00
48d663667b
pull from master
2012-03-19 18:03:32 -07:00
2232ea5101
was not computing lookahead correctly in _LOOK. It assumed all epsilons were predicates.
2012-03-19 17:50:51 -07:00
85b40c7d2e
shelve
2012-03-19 17:47:57 -07:00
bd7796544d
Update comments
2012-03-19 08:33:55 -05:00
5e0f9a4490
Ensure target non-null in Transition constructor (additional runtime-side consistency check).
2012-03-19 08:33:36 -05:00
a70cb6f36a
Fix NPE in LexerATNSimulator.execDFA error recovery
2012-03-19 08:17:59 -05:00
974ae65cb9
shelve
2012-03-18 20:21:39 -07:00
90516272fe
snapshot
2012-03-18 15:39:08 -07:00
7233177441
got integrated graph stacks to compile. commit so i can look at sam's
2012-03-18 15:20:52 -07:00
3a79aa15ba
move new classes into proper package locations
2012-03-18 13:45:13 -07:00
ea7037dd2d
missing semicolon
2012-03-16 14:29:07 -07:00
9a0aaacbee
rm k=1 chk to report early ambiguity.
2012-03-16 14:11:21 -07:00
ae871dcc07
Rename exec/exec to execDFA/execATN
2012-03-13 19:27:52 -05:00
aeb43069aa
Move reach variable inside loop. Remove unnecessary ATNConfigSet clone.
2012-03-13 19:25:39 -05:00
51a15ac4a7
ATNConfigSet prior to error is closure, not reach. Reach is empty.
2012-03-13 19:25:38 -05:00
83c680850e
Only create a new ATNConfigSet for reach when no DFA edge is available
2012-03-13 19:25:38 -05:00
9d774c6543
Separate handling of DFA and ATN transitions during lexer atn sim.
2012-03-13 19:25:37 -05:00
1f60da9cfd
Handle DFA error states in lexer atn exec
2012-03-13 19:25:37 -05:00
590558c4d8
Combine DFAExecState and ATNExecState so we no longer need to throw exceptions in LexerATNSimulator.failOrAccept for valid inputs
2012-03-13 19:25:31 -05:00
aca3e0ddb2
make trim parse trees singular
2012-03-12 15:40:22 -07:00
dc627158e6
loop invar. code motion
2012-03-12 14:19:19 -07:00
2fc9760ba7
Merge remote-tracking branch 'sharwell/ambig-sempred'
2012-03-12 13:50:45 -07:00
79f204ff94
Merge commit 'fa9ec19'
2012-03-12 13:22:55 -07:00
754408da27
Merge commit '86671d3'
2012-03-12 13:19:35 -07:00
3f1f76df7d
Move reportAmbiguity, reportContextSensitivity, reportAttemptingFullContext, reportInsufficientPredicates from ANTLRErrorStrategy to ANTLRErrorListener.
...
Add BaseErrorListener to allow implementing ANTLRErrorListener without implementing every method (e.g. ConsoleErrorListener).
DiagnosticErrorStrategy is now DiagnosticErrorListener, updated tests.
2012-03-12 15:07:48 -05:00
768bfc0cf2
Add ProxyErrorListener to allow dispatching error reporting to multiple listeners without manually iterating over the list of listeners.
2012-03-12 15:07:48 -05:00
dce72dcbfb
* Recognizer._listeners initialized to have a single ConsoleErrorListener
...
* Use CopyOnWriteArrayList so listeners can be added/removed in callbacks
* Remove special handling for _listeners null or empty (never null, ConsoleErrorListener will be present if feature is desired)
2012-03-12 15:01:21 -05:00
50180825d9
Add boolean field ParserATNSimulator.reportAmbiguities (default true). When false, ambiguous alts aren't reported which allows usage of (the faster) incomplete semantic predicate evaluation even within execATN.
2012-03-10 16:41:57 -06:00
04cd48e88d
No need to fully evaluate predicates when called from execDFA because ambiguous alts aren't reported anyway.
2012-03-10 16:32:10 -06:00
2cc39c2f9b
Evaluate predicates for all ambiguous alternatives and report if still ambiguous after predicate evaluation. Remove misleading message insufficient predicates. Fixes antlr/antlr4#31 . Partially addresses antlr/antlr4#39 .
2012-03-10 16:16:39 -06:00
86671d30fb
Tweak comments
2012-03-10 13:23:02 -06:00
7525bb6e1d
Semantic predicates now evaluate with standard min-alt selection
2012-03-08 13:53:45 -06:00
a9e74ce399
Merge branch 'failed-predicate-message' of git://github.com/sharwell/antlr4
2012-03-06 13:28:47 -08:00
5deab167a1
ParseTreeVisitor.visit takes a ParseTree... only visitChildren takes a RuleNode
2012-03-04 12:52:43 -06:00
b570641e2b
Simplify ParseTree.visitChildren; ParseTree.visit and visitChildren take a RuleNode parameter; update comments
2012-03-04 12:17:43 -06:00
a948736729
TerminalNodeImpl.isErrorNode now tests for ErrorNode interface instead of ErrorNodeImpl implementation
2012-03-03 21:57:49 -06:00
80b3ec8c62
Dispatch accept() for terminal nodes to visitor
2012-03-03 21:57:08 -06:00
0375f8318a
rename a parameter
2012-03-01 18:01:27 -08:00
2ba5a1f12a
Made the parse tree visitor walk ParseTree objects not ParserRuleContext objects. This necessitated pulling up the accept() and toStringTree(Parser) methods into ParseTree.
2012-03-01 17:57:49 -08:00
fa9ec191bd
Add TrimToSizeListener and Parser.get/setTrimParseTrees
2012-02-29 11:22:54 -06:00
c3cd99858c
Reduce size of generated code by emitting predicate text instead of full failed predicate message
2012-02-29 08:14:42 -06:00
dd69a7532d
getToken/s returns TerminalNode now per Sam's suggestion. altered tests. seems correct.
2012-02-27 21:39:30 -08:00
9ae92a5cef
reformat
2012-02-27 21:32:25 -08:00
67b2e6d7c1
forgot to move Pair etc...
2012-02-26 21:55:23 -08:00
e9a3de645a
add useful method to get a->b pairs out
2012-02-26 15:55:46 -08:00
41be88dcd5
mv to runtime
2012-02-26 15:49:45 -08:00
bedc502d33
Set the listeners list back to null if all listeners are removed from it
2012-02-26 17:33:52 -06:00
1e438746be
added interface for error nodes, method in terminalnode.
2012-02-26 15:32:18 -08:00
db1b86af91
alter visitTerminal interface, add visitErrorNode.
2012-02-25 17:03:41 -08:00
528e9c7b86
fix cut/paste error
2012-02-25 16:09:29 -08:00
600c4bbd91
crap. sun already made IdentityHashMap; rm'ing my version
2012-02-25 16:07:34 -08:00
73ae2b0bfc
Added ParseTreeProperty and added generically useful Map that ignores key.hashCode and equals() and simply uses key identity instead. In particular, this is useful for ANTLR because parse trees are actually interfaces laid on top of the rule context objects used by ANTLR's grammar analysis. These context objects use special equals methods and so on. Plus, when we want to annotate trees with values, we want a specific note to have a value. Identity is what we intend.
2012-02-25 13:43:32 -08:00
a74da2df60
Use ==ParserRuleContext.EMPTY instead of .isEmpty()
2012-02-24 15:57:43 -06:00
9bf6f284df
Code cleanup (little things like using .isEmpty() and StringBuilder, and specify some small-scale generic arguments)
2012-02-23 16:42:36 -06:00
1e3e092326
Fix unchecked warnings in clone()
2012-02-23 16:13:15 -06:00
e400f5b1a1
Move Recognizer._errHandler to Parser class
2012-02-23 16:13:14 -06:00
1c8efd6a48
Allow contravariant error listeners
2012-02-23 16:13:13 -06:00
6c26917b90
Use generic List instead of generic array for listeners
2012-02-23 16:13:12 -06:00
7dcb148d73
CommonTokenStream now extends BufferedTokenStream<Token> instead of BufferedTokenStream<CommonToken>. Token interface provides the same information and the change helps the work on improving generics.
2012-02-23 16:13:11 -06:00
d7dbef2782
Can get token start and stop index from Token interface now
2012-02-23 16:13:11 -06:00
1492648d26
Add missing @Override annotations
2012-02-23 16:13:06 -06:00
96c08bec52
Remove unused imports
2012-02-23 16:12:02 -06:00
b459dafeb9
empty alternatives didn't trigger listener events. overrode getParent in ParserRuleContext to make type more specific.
2012-02-23 12:16:14 -08:00
8a34176d82
added listener unit tests. fixed bug that didn't create ctx getters properly for recursive rules. added Symbol extends Token to parse tree stuff. added visitTerminal to Visitor. recursive alts now track their original, unedited AltAST subtree so we can properly count rule refs etc... later. dup of RuleRefAST was making wrong node. don't gen dispatch methods if no listener.
2012-02-22 12:44:33 -08:00
58ef729be5
renamed next to target. factored out to methods, commented code.
2012-02-21 12:47:34 -08:00
0e23c7e2a3
Merge commit '611728a3806d6a8c9fe44a538d34b448784986c1'
2012-02-21 12:14:11 -08:00
5fb389b6e4
Improve lexer performance during warm-up by tracking DFA states within exec (ATN) and using existing edges when available
2012-02-20 19:20:50 -06:00
611728a380
Update testPerformance to show information about DFA states in the lexer
2012-02-20 18:51:26 -06:00
a81b1a17eb
Extract abstract method in Recognizer for Parser.getGrammarFileName() and generated lexer's getGrammarFileName(). Make Recognizer.getTokenNames(), getRuleNames(), and getATN() abstract - implementations are always generated.
2012-02-20 17:18:57 -06:00
2982f4b5a6
Add comments describing the semantics of an ATNConfig set with partially predicated alts in ParserATNSimulator
2012-02-20 15:43:02 -06:00
16fa8ba4f4
Add comments describing the context depth latch in ParserATNSimulator.closure
2012-02-20 15:25:58 -06:00
e9122cafa3
Merge commit '46d5458'
2012-02-20 12:22:37 -08:00
1a2094b2dd
merging Sam\'s fix to full ctx pred etc...; effectively did pull parrt/antlr4#15 , includes fixes antlr/antlr4#14 , includes fixes antlr/antlr4#15
2012-02-20 11:48:03 -08:00
4e8931519c
added -parse-listener option and generated code if option on. parse listener differs from tree listener
2012-02-18 16:49:05 -08:00
725b105135
Added visitor gen mechanism, runtime support
2012-02-16 17:49:57 -08:00
27806dc490
Merge remote-tracking branch 'parrt/master'
2012-02-15 14:58:16 -06:00
28919bbdc5
Fix ParserATNSimulator.execDFA consuming an unnecessary symbol after an accept or context-sensitive state is reached
2012-02-15 14:02:50 -06:00
685cf7b7d5
Further improvements to auto-label generic usage, return Collections.emptyList() instead of null when no children are found
2012-02-15 12:31:43 -06:00
0bbb7101ba
Include input character in LexerNoViableAltException.toString()
2012-02-15 12:02:44 -06:00
bc87562aff
Remove unused exception LexerRecognitionExeption [sic]
2012-02-15 11:01:01 -06:00
5ab082967f
Fix generic usage
2012-02-15 10:50:37 -06:00
fa3a664696
got some good examples working that play around with the getter methods.
2012-02-14 16:33:12 -08:00
cdb420fdde
got implementation of getters in! Fixes #8
2012-02-14 15:01:03 -08:00
9600a70724
Fix NPE in CommonToken when TokenSource is not available
2012-02-13 08:41:57 -06:00
6e4cba131a
Allow var args in ctor of IntervalSet
2012-02-11 15:30:54 -08:00
f278d3b453
add unit test for left-recur grammars
2012-02-11 11:35:27 -08:00
cd3adb140d
Fix ParserATNSimulator.execDFA adding incorrect error edges in the DFA
2012-02-10 00:52:24 -06:00
30c58bf8eb
Specify <?> type parameter on raw types
2012-02-10 00:51:23 -06:00
99ce3cba5c
Fix ParserATNSimulator.closure not tracking proper context for context-sensitive predicates during full-context parsing
2012-02-09 19:47:56 -06:00
467797785f
Fix generics in ParserATNSimulator
2012-02-09 15:37:56 -06:00
ddf946b067
Token.EOF instead of -1, add <?> to generic type usage, make getUniqueAlt an instance method (need to override it for some apps), @NotNull annotations
2012-02-09 15:33:03 -06:00
7fb73a3849
Construct DFA instances with the decision number
2012-02-09 13:29:14 -06:00
bf5df30708
Remove unnecessary initializations, use isEmpty() instead of size()==0
2012-02-09 12:45:46 -06:00
46d5458617
Fix handling of alts with some configs predicated and some configs unpredicated (should be treated as always true)
2012-02-09 01:11:09 -06:00
9c554a002b
Add LexerATNSimulator.copyState (allows copying state information from another simulator instance)
2012-02-09 00:15:30 -06:00
40a8759276
Add LexerATNSimulator.getTraceStream
2012-02-09 00:14:37 -06:00
3b4ae2a742
Use trivial implementation of mark/release for BufferedTokenStream
2012-02-08 18:58:57 -06:00
f43f9f2218
Override CommonTokenStream.reset() to make sure the stream is left on an on-channel token
2012-02-08 18:58:19 -06:00
a34fd6330b
Use isEmpty() instead of size()==0
2012-02-08 18:57:32 -06:00
a9e44f3aab
Use isEmpty() instead of size()==0, use StringBuilder instead of StringBuffer, use chained append() calls instead of string concatenation, add @Override annotations
2012-02-08 18:57:05 -06:00
d181b230dc
Specify generic type parameter
2012-02-08 18:55:57 -06:00
64203312ac
Improved SemanticContext.hashCode(), add @Override annotations
2012-02-08 18:54:40 -06:00
773e9e1c7e
Add generic type argument
2012-02-08 18:12:01 -06:00
dcdfac4bcd
Improved implementation of equals() and hashCode() for ATNConfig
2012-02-08 18:09:32 -06:00
64210f4b72
Add @NotNull annotations, make ATNConfig.semanticContext field final
2012-02-08 17:21:09 -06:00
71e0c66477
Rename NUM_EDGES to MAX_DFA_EDGE, reduce max value to 127
2012-02-08 17:15:34 -06:00
2249135c21
Specify generic type parameters
2012-02-08 17:12:22 -06:00
1df70924f5
Use ATNConfigSet instead of OrderedHashSet<ATNConfig>
2012-02-08 17:06:41 -06:00
52256044a9
Use for each instead of indexed for; use isEmpty() instead of size()==0
2012-02-08 17:06:03 -06:00
326cb259a0
Construct Parser with TokenStream (restriction from IntStream)
2012-02-08 16:34:04 -06:00
18a6414fb3
Use ATNConfigSet for parameter instead of OrderedHashSet<ATNConfig>
2012-02-08 16:26:14 -06:00
a6524d8b38
Add pom.xml to support maven builds
2012-02-08 15:07:06 -06:00
894a77168d
rename atn sim, add error message to TestRig
2012-02-05 10:14:04 -08:00
1cc7ac3410
added -trace to TestRig. fixed code gen issue for trace listener
2012-01-30 16:51:55 -08:00
e90b322dd4
~[] stuff is allowed and works inside sets etc...
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9926]
2012-01-28 14:36:46 -08:00
09da63cfd4
rollback and do a real fix for whitespace escaping
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9911]
2012-01-27 12:00:30 -08:00
7498908d62
rollback and do a real fix for whitespace escaping
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9906]
2012-01-26 14:18:57 -08:00
adcf72b9a2
whitespace was being escaped to special characters even when printing to string should only do so when doing GUI tree
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9904]
2012-01-26 14:11:30 -08:00
9fd280b30e
Lexer fields mv from x to _x
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9883]
2012-01-21 15:36:20 -08:00
6013c4c97d
Added modeNames to gen'd lexers
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9878]
2012-01-21 15:10:46 -08:00
dc82edad02
Summary: we can have lexer commands like -> skip now.
...
moved model.actions package to chunk
type(foo) didn't match.
ATNBuilder.g now tracks the outer alternative number and calls new factory commands on the lexer commands.
Removed unnecessary resolveWithPredicate field from ATNConfig
Added lexerActionIndex field to ATNConfig since we need to track whether we passed an action in an alternative will rule in the lexer.
Renamed ruleIndex in DFAState and added the lexer action index so that we can execute lexer actions from the DFA.
added functions to the grammar tree visitor for the lexer commands.
Added templates for the lexer commands.
Augmented the lexer ATN factory so that it constructs plain old actions from the lexer commands it finds. That way, the code generator doesn't know any different and generates an action.
Augmented the lexer ATN simulator so that it fires a proper action index now. previously it only used the rule index, which of course doesn't work when you have more than one action in a rule.
rm'd dup code from OutputModelController
altered the epsilon edge removal optimization so that it could not remove actions in lexer rules.
Added list of valid lexer commands in Rule.
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9877]
2012-01-21 15:01:05 -08:00
093a4f951b
leave start/stop char index alone in CommonTokenFactory; refers to original text.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9867]
2012-01-14 11:18:23 -08:00
5bd780c235
in tree views, spaces/newlines were blanks. I converted to \n and middle dot
...
for space.
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9857]
2012-01-11 17:25:44 -08:00
848c6e4313
This change is a major restructuring of how left recursive rules are transformed. Previously I simply rewrote the grammar and ANTLR was none the wiser. However, it quickly became apparent that ANTLR needed to do many different things for recursive rules so I had to insert transformation later in the pipeline. Specifically, I needed it after Rule object creation. I made a special LeftRecursiveRule object that track information collected during transformation so that I could use it later. I made major changes to the left recursion templates as well as the little code snippets in Java.stg. I created a new template called LeftRecursiveRuleFunction an accompanying model object that handles the special case, even though there is some duplication. the biggest difference in the grammar is the introduction of => ID notation on any outermost alternative. This information is not added to the tree, instead the ALT node is annotated with the ID. Rule.getAltLabels() now looks also in the LeftRecursiveRuleAltInfo objects. I have moved to the left recursion transformation to its own object and have moved some objects into the analysis package. Further, I have split out the Rule object creation into its own RuleCollector. I renamed discoverAlt in the grammar tree visitor to be discoverOuterAlt an added discoverAlt so we can get information about individual alts even inside subrules. Listeners always get an event for the generic rule context, which is used if there is no specific label for an alternative. Added a list of iteration operations for LL(*) subrules. Split buildRuleFunction into buildLeftRecursiveRuleFunction and one for normal rule function creation. I have to insert lots of extra code to manage the contexts, but of course it's all done using the templates. As long as those templates are correct, this code generation mechanism will work. I removed the st field from the parser rule context. I injected the left recursion transformation inside the SemanticPipeline. Visitor dispatch methods are always added to the generated context structures. Fixed some unit tests. About 24 fail.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9854]
2012-01-11 11:00:05 -08:00
742a08862f
Merging dev changes to main
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9853]
2012-01-09 10:26:23 -08:00
4ae23f4a64
* 1+2*3 now gives new parse tree: (e (e 1) + (e (e 2) * (e 3))) See CHANGES.txt now too
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9833]
2012-01-05 12:27:14 -08:00
cab41e41e3
had non-ascii char
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9812]
2012-01-04 09:05:31 -08:00
963ccd2700
added comment
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9797]
2012-01-03 12:46:51 -08:00
224a4bc9e1
changed formatting
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9796]
2012-01-03 12:46:39 -08:00
8099cec3bd
Added ctx to visitTerminal in parse tree listener. That should be useful information. made commensurate change in the parse tree walker to make sure I have a proper context to send to the visitTerminal method. renamed a few fields of parser/lexer to have _ in front to avoid name collisions with user actions. change the name of the listener method so that they're more explicit using the terms Error and Parser to identify what kind of listener we are adding or removing. Added a default TraceListener to Parser so that we can call setTrace(true) to have it start dumping out information as a first step in the debugging process. there are now multiple parse tree listeners possible because I made a list. we may want to pass in multiple actions to the parser.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9795]
2012-01-03 12:46:18 -08:00
5c328c7e3f
Bug: didn't stop at end of stream during consume()
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9794]
2012-01-03 12:40:42 -08:00
5d60e85617
rm trace ATN state stuff. Will think about when we do debugging.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9792]
2012-01-03 11:07:49 -08:00
c716a87544
move method down in file
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9791]
2012-01-03 11:05:40 -08:00
5c3c8d6e7a
Added TokenFactory, CommonTokenFactory. Update the parser in the lexer with methods to set the factory. Alter the default error strategy and the lexer to use the factory. The parser's set token factory method updates the token source, usually the lexer, and the error handling strategy. I had to add the set token factory method to token source as well to make all of this work.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9789]
2012-01-03 10:58:01 -08:00
3aeeb2b277
rename BaseREcognizer as Parser
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9781]
2012-01-02 18:36:59 -08:00
1b04195a2f
removed all template / AST rewrite stuff; massive change; added -encoding tool option
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9780]
2012-01-02 18:13:16 -08:00
8846b98ee7
fix compile error
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9765]
2011-12-29 17:05:34 -08:00
a923ad8765
Major update to v4. I backed out a change I made on Christmas then mistakenly prevented any lexer DFA creation. Per http://www.antlr.org/wiki/display/~admin/2011/12/29/Flaw+in+ANTLR+v3+LL%28*%29+analysis+algorithm I fixed a major flaw in ANTLR's notion of context. To do that, I needed to create a new LoopEndState, with all of its fanout to the serialization and parser ATN construction. got a very good start on ParserATNPathFinder, which uses basic recursion to find all possible paths and return a tree with the possibilities. I left it in the condition where he would sometimes loop forever; it needs to track sets of configurations in the busy set; it using states at the moment. added a new signal from the interpreter: reportAttemptingFullContext. I fixed a bug where configuration sets derived from a configuration that had reachesIntoOuterContext>0 were not being considered as dipping into the outer context. The ambiguity checker needed to switch so that a check for exact matches not suffixes when doing full context. It's faster at the very least for full context. added some more support routines to DFA. Added TraceTree in support of the new ParserATNPathFinder.
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9764]
2011-12-29 17:04:40 -08:00
3b51bb478d
rm unneeded field
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9761]
2011-12-27 15:52:47 -08:00
d01bde9125
added new reportAttemptingFullContext to strategy
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9760]
2011-12-27 15:25:01 -08:00
bb960e3c37
improved data to report insuff. preds. also, it was detecting it improperly
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9758]
2011-12-27 14:11:41 -08:00
638969bed9
add '...' to diagnostics input
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9757]
2011-12-27 13:37:47 -08:00
c1e199d77f
added -tokens flag
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9756]
2011-12-26 18:10:29 -08:00
bf8473e0fd
add cmt
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9755]
2011-12-26 17:57:00 -08:00
ce9638e59c
was still creating dfa with sempreds
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9752]
2011-12-26 17:07:45 -08:00
c1923a107e
-tree => -print option
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9748]
2011-12-26 14:05:58 -08:00
db655f8f2c
add -gui option to view in gui window
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9747]
2011-12-26 13:06:05 -08:00
5c50e5f049
Wasn't using Udo's lib correctly
...
[git-p4: depot-paths = "//depot/code/antlr4/main/": change = 9746]
2011-12-26 13:05:46 -08:00
Terence Parr
Sam Harwell
sharwell