forgetest1
/
antlr
forked from jasder/antlr
1
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Merge branch 'master' into feature/templates

This commit is contained in:
Mike Lischke 2016-04-23 16:05:10 +02:00
parent 815856bdf0 ac8384354c
commit 014d9fd593
107 changed files with 1172 additions and 215 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
.travis.yml
View File

@ -11,8 +11,9 @@ before_install:
- sudo add-apt-repository ppa:fkrull/deadsnakes -y
- sudo add-apt-repository ppa:rwky/nodejs -y
- sudo apt-get update -qq
- sudo apt-get install -qq python3.4
- sudo apt-get install -qq python3.5
- sudo apt-get install -qq nodejs
- echo "deb http://download.mono-project.com/repo/debian wheezy/snapshots/3.12.1 main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
- sudo apt-get install -qq mono-complete
- python --version
- python3 --version

2
antlr4-maven-plugin/pom.xml
View File

@ -34,7 +34,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
</parent>
<artifactId>antlr4-maven-plugin</artifactId>
<packaging>maven-plugin</packaging>

4
contributors.txt
View File

@ -88,4 +88,8 @@ YYYY/MM/DD, github id, Full name, email
2015/12/17, sebadur, Sebastian Badur, sebadur@users.noreply.github.com
2015/12/23, pboyer, Peter Boyer, peter.b.boyer@gmail.com
2015/12/24, dtymon, David Tymon, david.tymon@gmail.com
2016/02/18, reitzig, Raphael Reitzig, reitzig[at]cs.uni-kl.de
2016/03/10, mike-lischke, Mike Lischke, mike@lischke-online.de
2016/03/27, beardlybread, Bradley Steinbacher, bradley.j.steinbacher@gmail.com
2016/03/29, msteiger, Martin Steiger, antlr@martin-steiger.de
2016/03/28, gagern, Martin von Gagern, gagern@ma.tum.de

2
doc/actions.md
View File

@ -2,7 +2,7 @@
In Chapter 10, Attributes and Actions, we learned how to embed actions within grammars and looked at the most common token and rule attributes. This section summarizes the important syntax and semantics from that chapter and provides a complete list of all available attributes. (You can learn more about actions in the grammar from the free excerpt on listeners and actions.)
Actions are blocks of text written in the target language and enclosed in curly braces. The recognizer triggers them according to their locations within the grammar. For example, the following rule emits found a decl after the parser has seen a valid declaration:
Actions are blocks of text written in the target language and enclosed in curly braces. The recognizer triggers them according to their locations within the grammar. For example, the following rule emits "found a decl" after the parser has seen a valid declaration:
```
decl: type ID ';' {System.out.println("found a decl");} ;

2
doc/grammars.md
View File

@ -83,7 +83,7 @@ $ grun MyELang stat
If there were any `tokens` specifications, the main grammar would merge the token sets. Any named actions such as `@members` would be merged. In general, you should avoid named actions and actions within rules in imported grammars since that limits their reuse. ANTLR also ignores any options in imported grammars.
Imported grammars can also import other grammars. ANTLR pursues all imported grammars in a depth-first fashion. If two or more imported grammars define ruler, ANTLR chooses the first version of `r` it finds. In the following diagram, ANTLR examines grammars in the following order `Nested`, `G1`, `G3`, `G2`.
Imported grammars can also import other grammars. ANTLR pursues all imported grammars in a depth-first fashion. If two or more imported grammars define rule `r`, ANTLR chooses the first version of `r` it finds. In the following diagram, ANTLR examines grammars in the following order `Nested`, `G1`, `G3`, `G2`.
<img src=images/nested.png width=350>

2
doc/left-recursion.md
View File

@ -1,6 +1,6 @@
# Left-recursive rules
The most natural expression of a some common language constructs is left recursive. For example C declarators and arithmetic expressions. Unfortunately, left recursive specifications of arithmetic expressions are typically ambiguous but much easier to write out than the multiple levels required in a typical top-down grammar. Here is a sample ANTLR 4 grammar with a left recursive expression rule:
The most natural expression of some common language constructs is left recursive. For example C declarators and arithmetic expressions. Unfortunately, left recursive specifications of arithmetic expressions are typically ambiguous but much easier to write out than the multiple levels required in a typical top-down grammar. Here is a sample ANTLR 4 grammar with a left recursive expression rule:
```
stat: expr '=' expr ';' // e.g., x=y; or x=f(x);

5
doc/lexer-rules.md
View File

@ -171,7 +171,7 @@ error(126): P.g4:3:4: cannot create implicit token for string literal '&' in non
## Lexer Rule Actions
An ANTLR lexer creates a Token object after matching a lexical rule. Each request for a token starts in Lexer.nextToken, which calls emit once it has identified a token.emit collects information from the current state of the lexer to build the token. It accesses fields `_type`, `_text`, `_channel`, `_tokenStartCharIndex`, `_tokenStartLine`, and `_tokenStartCharPositionInLine`. You can set the state of these with the various setter methods such as `setType`. For example, the following rule turns `enum` into an identifier if `enumIsKeyword` is false.
An ANTLR lexer creates a Token object after matching a lexical rule. Each request for a token starts in `Lexer.nextToken`, which calls `emit` once it has identified a token. `emit` collects information from the current state of the lexer to build the token. It accesses fields `_type`, `_text`, `_channel`, `_tokenStartCharIndex`, `_tokenStartLine`, and `_tokenStartCharPositionInLine`. You can set the state of these with the various setter methods such as `setType`. For example, the following rule turns `enum` into an identifier if `enumIsKeyword` is false.
```
ENUM : 'enum' {if (!enumIsKeyword) setType(Identifier);} ;
@ -255,7 +255,8 @@ WS : [ \r\t\n]+ -> skip ;
```
For multiple 'type()' commands, only the rightmost has an effect.
channel()
### channel()
```
BLOCK_COMMENT

53
doc/options.md
View File

@ -12,6 +12,55 @@ where a value can be an identifier, a qualified identifier (for example, a.b.c),
All grammars can use the following options. In combined grammars, all options except language pertain only to the generated parser. Options may be set either within the grammar file using the options syntax (described above) or when invoking ANTLR on the command line, using the `-D` option. (see Section 15.9, [ANTLR Tool Command Line Options](tool-options.md).) The following examples demonstrate both mechanisms; note that `-D` overrides options within the grammar.
* `superClass`. Set the superclass of the generated parser or lexer. For combined grammars, it sets the superclass of the parser.
```
$ cat Hi.g4
grammar Hi;
a : 'hi' ;
$ antlr4 -DsuperClass=XX Hi.g4
$ grep 'public class' HiParser.java
public class HiParser extends XX {
$ grep 'public class' HiLexer.java
public class HiLexer extends Lexer {
```
* `language` Generate code in the indicated language, if ANTLR is able to do so. Otherwise, you will see an error message like this:
```
$ antlr4 -Dlanguage=C MyGrammar.g4
error(31): ANTLR cannot generate C code as of version 4.0
```
* `tokenVocab` ANTLR assigns token type numbers to the tokens as it encounters them in a file. To use different token type values, such as with a separate lexer, use this option to have ANTLR pull in the <fileextension>tokens</fileextension> file. ANTLR generates a <fileextension>tokens</fileextension> file from each grammar.
```
$ cat SomeLexer.g4
lexer grammar SomeLexer;
ID : [a-z]+ ;
$ cat R.g4
parser grammar R;
options {tokenVocab=SomeLexer;}
tokens {A,B,C} // normally, these would be token types 1, 2, 3
a : ID ;
$ antlr4 SomeLexer.g4
$ cat SomeLexer.tokens
ID=1
$ antlr4 R.g4
$ cat R.tokens
A=2
B=3
C=4
ID=1
```
* `TokenLabelType` ANTLR normally uses type <class>Token</class> when it generates variables referencing tokens. If you have passed a <class>TokenFactory</class> to your parser and lexer so that they create custom tokens, you should set this option to your specific type. This ensures that the context objects know your type for fields and method return values.
```
$ cat T2.g4
grammar T2;
options {TokenLabelType=MyToken;}
a : x=ID ;
$ antlr4 T2.g4
$ grep MyToken T2Parser.java
public MyToken x;
```
* `contextSuperClass`. Specify the super class of parse tree internal nodes. Default is `ParserRuleContext`. Should derive from ultimately `RuleContext` at minimum.
Java target can use `contextSuperClass=org.antlr.v4.runtime.RuleContextWithAltNum` for convenience. It adds a backing field for `altNumber`, the alt matched for the associated rule node.
## Rule Options
There are currently no valid rule-level options, but the tool still supports the following syntax for future use:
@ -25,7 +74,7 @@ options {...}
## Rule Element Options
Token options have the form `T<name=value>` as we saw in Section 5.4, [Dealing with Precedence, Left Recursion, and Associativity](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference). The only token option is assocand it accepts values left and right. Here’s a sample grammar with a left-recursive expression rule that specifies a token option on the `^` exponent operator token:
Token options have the form `T<name=value>` as we saw in Section 5.4, [Dealing with Precedence, Left Recursion, and Associativity](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference). The only token option is `assoc`, and it accepts values `left` and `right`. Here’s a sample grammar with a left-recursive expression rule that specifies a token option on the `^` exponent operator token:
```
grammar ExprLR;
@ -40,7 +89,7 @@ INT : '0'..'9'+ ;
WS : [ \n]+ -> skip ;
```
Semantic predicates also accept an option, per [Catching failed semantic predicates](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference). The only valid option is the fail option, which takes either a string literal in double-quotes or an action that evaluates to a string. The string literal or string result from the action should be the message to emit upon predicate failure.
Semantic predicates also accept an option, per [Catching failed semantic predicates](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference). The only valid option is the `fail` option, which takes either a string literal in double-quotes or an action that evaluates to a string. The string literal or string result from the action should be the message to emit upon predicate failure.
```
ints[int max]

4
doc/predicates.md
View File

@ -29,7 +29,7 @@ expr: {istype()}? ID '(' expr ')' // ctor-style typecast
;
```
The parser will only predict an expr from stat when `istype()||isfunc()` evaluates to true. This makes sense because the parser should only choose to match an expression if the upcoming `ID` is a type name or function name. It wouldn't make sense to just test one of the predicates in this case. Note that, when the parser gets to expritself, the parsing decision tests the predicates individually, one for each alternative.
The parser will only predict an expr from stat when `istype()||isfunc()` evaluates to true. This makes sense because the parser should only choose to match an expression if the upcoming `ID` is a type name or function name. It wouldn't make sense to just test one of the predicates in this case. Note that, when the parser gets to `expr` itself, the parsing decision tests the predicates individually, one for each alternative.
If multiple predicates occur in a sequence, the parser joins them with the `&&` operator. For example, consider changing `stat` to include a predicate before the call `toexpr`:
@ -72,7 +72,7 @@ stat: {System.out.println("goto"); allowgoto=true;} {java5}? 'goto' ID ';'
If we can't execute the action during prediction, we shouldn't evaluate the `{java5}?` predicate because it depends on that action.
The prediction process also can't see through token references. Token references have the side effect of advancing the input one symbol. A predicate that tested the current input symbol would find itself out of sync if the parser shifted it over the token reference. For example, in the following grammar, the predicates expectgetCurrentToken to return an ID token.
The prediction process also can't see through token references. Token references have the side effect of advancing the input one symbol. A predicate that tested the current input symbol would find itself out of sync if the parser shifted it over the token reference. For example, in the following grammar, the predicates expect `getCurrentToken` to return an `ID` token.
```
stat: '{' decl '}'

9
doc/releasing-antlr.md
View File

@ -135,16 +135,21 @@ cp ~/.m2/repository/org/antlr/antlr4/4.5.2/antlr4-4.5.2.jar ~/antlr/sites/websit
cd ~/antlr/sites/website-antlr4/download
git add antlr-4.5.2-complete.jar
git add antlr-runtime-4.5.2.jar
git commit -a -m 'add 4.5.2 jars'
git push origin gh-pages
```
Update on site:
* download.html
* index.html
* api/index.html
* download/index.html
* scripts/topnav.js
```
git commit -a -m 'add 4.5.2 jars'
git push origin gh-pages
```
## Deploying Targets
### JavaScript

3
doc/targets.md
View File

@ -18,7 +18,6 @@ The [ANTLR v4 book](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-ref
New features generally appear in the Java target and then migrate to the other targets, but these other targets don't always get updated in the same overall tool release. This section tries to identify features added to Java that have not been added to the other targets.
|Feature|Java|C&sharp;|JavaScript|Python2|Python3|Swift|C++|
|-|-|-|-|-|-|-|-|
|---|---|---|---|---|---|---|---|
|Ambiguous tree construction|4.5.1|-|-|-|-|-|-|

2
doc/tree-matching.md
View File

@ -29,7 +29,7 @@ ParseTreeMatch m = p.match(t);
if ( m.succeeded() ) {...}
```
We can also test for specific expressions or token values. For example, the following checks to see if t is an expression consisting of an identifier added to 0:
We can also test for specific expressions or token values. For example, the following checks to see if `t` is an expression consisting of an identifier added to 0:
```java
ParseTree t = ...; // assume t is an expression

2
pom.xml
View File

@ -7,7 +7,7 @@
</parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
<packaging>pom</packaging>
<name>ANTLR 4</name>

2
runtime-testsuite/pom.xml
View File

@ -4,7 +4,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
</parent>
<artifactId>antlr4-runtime-testsuite</artifactId>
<name>ANTLR 4 Runtime Test Generator</name>

13
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/CSharp.test.stg
View File

@ -327,6 +327,19 @@ ParseTreeWalker walker = new ParseTreeWalker();
walker.Walk(new LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
public class MyRuleNode : ParserRuleContext {
public int altNum;
public MyRuleNode(ParserRuleContext parent, int invokingStateNumber): base(parent, invokingStateNumber)
{
}
public override int getAltNumber() { return altNum; }
public override void setAltNumber(int altNum) { this.altNum = altNum; }
}
}
>>
TokenGetterListener(X) ::= <<
public class LeafListener : TBaseListener {
public override void ExitA(TParser.AContext ctx) {

13
runtime-testsuite/resources/org/antlr/v4/test/runtime/java/Java.test.stg
View File

@ -335,6 +335,19 @@ ParseTreeWalker walker = new ParseTreeWalker();
walker.walk(new LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
public static class MyRuleNode extends ParserRuleContext {
public int altNum;
public MyRuleNode(ParserRuleContext parent, int invokingStateNumber) {
super(parent, invokingStateNumber);
}
@Override public int getAltNumber() { return altNum; }
@Override public void setAltNumber(int altNum) { this.altNum = altNum; }
}
}
>>
TokenGetterListener(X) ::= <<
public static class LeafListener extends TBaseListener {
public void exitA(TParser.AContext ctx) {

14
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/chrome/Chrome.test.stg
View File

@ -322,6 +322,20 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
this.LeafListener = function() {
this.exitA = function(ctx) {

14
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/explorer/Explorer.test.stg
View File

@ -322,6 +322,20 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
this.LeafListener = function() {
this.exitA = function(ctx) {

14
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/firefox/Firefox.test.stg
View File

@ -324,6 +324,20 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
this.LeafListener = function() {
this.exitA = function(ctx) {

18
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/node/Node.test.stg
View File

@ -322,6 +322,24 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
MyRuleNode.prototype.getAltNumber = function() { return this.altNum; }
MyRuleNode.prototype.setAltNumber = function(altNumber) { this.altNum = altNumber; }
}
>>
TokenGetterListener(X) ::= <<
this.LeafListener = function() {
this.exitA = function(ctx) {

15
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/safari/Safari.test.stg
View File

@ -322,6 +322,21 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
this.LeafListener = function() {
this.exitA = function(ctx) {

13
runtime-testsuite/resources/org/antlr/v4/test/runtime/python2/Python2.test.stg
View File

@ -312,6 +312,19 @@ walker = ParseTreeWalker()
walker.walk(TParser.LeafListener(), <s>)
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
class MyRuleNode(ParserRuleContext):
def __init__(self, parent = None, invokingStateNumber = None ):
super(<X>Parser.MyRuleNode, self).__init__(parent, invokingStateNumber)
self.altNum = 0;
def getAltNumber(self):
return self.altNum
def setAltNumber(self, altNum):
self.altNum = altNum
}
>>
TokenGetterListener(X) ::= <<
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener

13
runtime-testsuite/resources/org/antlr/v4/test/runtime/python3/Python3.test.stg
View File

@ -314,6 +314,19 @@ walker = ParseTreeWalker()
walker.walk(TParser.LeafListener(), <s>)
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
class MyRuleNode(ParserRuleContext):
def __init__(self, parent:ParserRuleContext = None, invokingStateNumber:int = None ):
super(<X>Parser.MyRuleNode, self).__init__(parent, invokingStateNumber)
self.altNum = 0;
def getAltNumber(self):
return self.altNum
def setAltNumber(self, altNum):
self.altNum = altNum
}
>>
TokenGetterListener(X) ::= <<
class LeafListener(MockListener):
def exitA(self, ctx):

2
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/LeftRecursion/Index.stg
View File

@ -59,6 +59,8 @@ TestTemplates ::= [
"MultipleAlternativesWithCommonLabel_2": [],
"MultipleAlternativesWithCommonLabel_3": [],
"MultipleAlternativesWithCommonLabel_4": [],
"PrefixAndOtherAlt_1": [],
"PrefixAndOtherAlt_2": [],
"PrefixOpWithActionAndLabel_1": [],
"PrefixOpWithActionAndLabel_2": [],
"PrefixOpWithActionAndLabel_3": [],

24
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/LeftRecursion/PrefixAndOtherAlt.stg Normal file
View File

@ -0,0 +1,24 @@
TestType() ::= "Parser"
Options ::= [
"Debug": false
]
Grammar ::= [
"T": {<grammar("T")>}
]
Rule() ::= "s"
grammar(grammarName) ::= <<
grammar <grammarName>;
s @after {<ToStringTree("$ctx"):writeln()>} : expr EOF ;
expr : literal
| op expr
| expr op expr
;
literal : '-'? Integer ;
op : '+' | '-' ;
Integer : [0-9]+ ;
WS : (' '|'\n') -> skip ;
>>

9
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/LeftRecursion/PrefixAndOtherAlt_1.stg Normal file
View File

@ -0,0 +1,9 @@
import "PrefixAndOtherAlt.stg"
Input() ::= "-1"
Output() ::= <<
(s (expr (literal - 1)) \<EOF>)<\n>
>>
Errors() ::= ""

9
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/LeftRecursion/PrefixAndOtherAlt_2.stg Normal file
View File

@ -0,0 +1,9 @@
import "PrefixAndOtherAlt.stg"
Input() ::= "-1 + -1"
Output() ::= <<
(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) \<EOF>)<\n>
>>
Errors() ::= ""

40
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/ParseTrees/AltNum.stg Normal file
View File

@ -0,0 +1,40 @@
TestType() ::= "Parser"
Grammar ::= [
"T": {<grammar("T")>}
]
Input() ::= "xyz"
Rule() ::= "s"
Output() ::= <<
(a:3 x (b:2 y) z)<\n>
>>
Errors() ::= ""
grammar(grammarName) ::= <<
grammar <grammarName>;
options { contextSuperClass=MyRuleNode; }
<TreeNodeWithAltNumField(X=grammarName)>
s
@init {
<BuildParseTrees()>
}
@after {
<ToStringTree("$r.ctx"):writeln()>
}
: r=a ;
a : 'f'
| 'g'
| 'x' b 'z'
;
b : 'e' {} | 'y'
;
>>

3
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/ParseTrees/Index.stg
View File

@ -6,5 +6,6 @@ TestTemplates ::= [
"RuleRef": [],
"ExtraToken": [],
"NoViableAlt": [],
"Sync": []
"Sync": [],
"AltNum": []
]

44
runtime-testsuite/test/org/antlr/v4/test/runtime/csharp/TestLeftRecursion.java
View File

@ -1801,6 +1801,50 @@ public class TestLeftRecursion extends BaseTest {
assertEquals("(prog (statement (letterA a)) (statement (letterA a)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_1() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(223);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {Console.WriteLine($ctx.ToStringTree(this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(s (expr (literal - 1)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_2() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(223);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {Console.WriteLine($ctx.ToStringTree(this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1 + -1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test

44
runtime-testsuite/test/org/antlr/v4/test/runtime/csharp/TestParseTrees.java
View File

@ -2,7 +2,6 @@
package org.antlr.v4.test.runtime.csharp;
import org.junit.Test;
import org.junit.Ignore;
@SuppressWarnings("unused")
public class TestParseTrees extends BaseTest {
@ -52,6 +51,49 @@ public class TestParseTrees extends BaseTest {
assertEquals("(a y)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testAltNum() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(547);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("\n");
grammarBuilder.append("options { contextSuperClass=MyRuleNode; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("@parser::members {\n");
grammarBuilder.append("public class MyRuleNode : ParserRuleContext {\n");
grammarBuilder.append(" public int altNum;\n");
grammarBuilder.append(" public MyRuleNode(ParserRuleContext parent, int invokingStateNumber): base(parent, invokingStateNumber)\n");
grammarBuilder.append(" {\n");
grammarBuilder.append(" }\n");
grammarBuilder.append(" public override int getAltNumber() { return altNum; }\n");
grammarBuilder.append(" public override void setAltNumber(int altNum) { this.altNum = altNum; }\n");
grammarBuilder.append("}\n");
grammarBuilder.append("}\n");
grammarBuilder.append("\n");
grammarBuilder.append("\n");
grammarBuilder.append("s\n");
grammarBuilder.append("@init {\n");
grammarBuilder.append("this.BuildParseTree = true;\n");
grammarBuilder.append("}\n");
grammarBuilder.append("@after {\n");
grammarBuilder.append("Console.WriteLine($r.ctx.ToStringTree(this));\n");
grammarBuilder.append("}\n");
grammarBuilder.append(" : r=a ;\n");
grammarBuilder.append("\n");
grammarBuilder.append("a : 'f'\n");
grammarBuilder.append(" | 'g'\n");
grammarBuilder.append(" | 'x' b 'z'\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("b : 'e' {} | 'y'\n");
grammarBuilder.append(" ;");
String grammar = grammarBuilder.toString();
String input ="xyz";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(a:3 x (b:2 y) z)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test

52
runtime-testsuite/test/org/antlr/v4/test/runtime/java/TestLeftRecursion.java
View File

@ -2007,6 +2007,58 @@ public class TestLeftRecursion extends BaseTest {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_1() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(224);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {System.out.println($ctx.toStringTree(this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(s (expr (literal - 1)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_2() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(224);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {System.out.println($ctx.toStringTree(this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1 + -1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixOpWithActionAndLabel_1() throws Exception {

51
runtime-testsuite/test/org/antlr/v4/test/runtime/java/TestParseTrees.java
View File

@ -1,10 +1,10 @@
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.java;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
public class TestParseTrees extends BaseTest {
@ -62,6 +62,53 @@ public class TestParseTrees extends BaseTest {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testAltNum() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(562);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("\n");
grammarBuilder.append("options { contextSuperClass=MyRuleNode; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("@parser::members {\n");
grammarBuilder.append("public static class MyRuleNode extends ParserRuleContext {\n");
grammarBuilder.append(" public int altNum;\n");
grammarBuilder.append(" public MyRuleNode(ParserRuleContext parent, int invokingStateNumber) {\n");
grammarBuilder.append(" super(parent, invokingStateNumber);\n");
grammarBuilder.append(" }\n");
grammarBuilder.append(" @Override public int getAltNumber() { return altNum; }\n");
grammarBuilder.append(" @Override public void setAltNumber(int altNum) { this.altNum = altNum; }\n");
grammarBuilder.append("}\n");
grammarBuilder.append("}\n");
grammarBuilder.append("\n");
grammarBuilder.append("\n");
grammarBuilder.append("s\n");
grammarBuilder.append("@init {\n");
grammarBuilder.append("setBuildParseTree(true);\n");
grammarBuilder.append("}\n");
grammarBuilder.append("@after {\n");
grammarBuilder.append("System.out.println($r.ctx.toStringTree(this));\n");
grammarBuilder.append("}\n");
grammarBuilder.append(" : r=a ;\n");
grammarBuilder.append("\n");
grammarBuilder.append("a : 'f'\n");
grammarBuilder.append(" | 'g'\n");
grammarBuilder.append(" | 'x' b 'z'\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("b : 'e' {} | 'y'\n");
grammarBuilder.append(" ;");
String grammar = grammarBuilder.toString();
String input ="xyz";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "s", input, false);
assertEquals("(a:3 x (b:2 y) z)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testExtraToken() throws Exception {

48
runtime-testsuite/test/org/antlr/v4/test/runtime/javascript/node/TestLeftRecursion.java
View File

@ -1905,6 +1905,54 @@ public class TestLeftRecursion extends BaseTest {
assertEquals("(prog (statement (letterA a)) (statement (letterA a)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_1() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(223);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {console.log($ctx.toStringTree(null, this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1";
String found = execParser("T.g4", grammar, "TParser", "TLexer",
"TListener", "TVisitor",
"s", input, false);
assertEquals("(s (expr (literal - 1)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_2() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(223);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {console.log($ctx.toStringTree(null, this));} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1 + -1";
String found = execParser("T.g4", grammar, "TParser", "TLexer",
"TListener", "TVisitor",
"s", input, false);
assertEquals("(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test

53
runtime-testsuite/test/org/antlr/v4/test/runtime/javascript/node/TestParseTrees.java
View File

@ -1,10 +1,10 @@
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.javascript.node;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
@SuppressWarnings("unused")
public class TestParseTrees extends BaseTest {
@ -58,6 +58,55 @@ public class TestParseTrees extends BaseTest {
assertEquals("(a y)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testAltNum() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(663);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("\n");
grammarBuilder.append("options { contextSuperClass=MyRuleNode; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("@parser::header {\n");
grammarBuilder.append("MyRuleNode = function(parent, invokingState) {\n");
grammarBuilder.append(" antlr4.ParserRuleContext.call(this, parent, invokingState);\n");
grammarBuilder.append("\n");
grammarBuilder.append(" this.altNum = 0;\n");
grammarBuilder.append(" return this;\n");
grammarBuilder.append("};\n");
grammarBuilder.append("\n");
grammarBuilder.append("MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);\n");
grammarBuilder.append("MyRuleNode.prototype.constructor = MyRuleNode;\n");
grammarBuilder.append("MyRuleNode.prototype.getAltNumber = function() { return this.altNum; }\n");
grammarBuilder.append("MyRuleNode.prototype.setAltNumber = function(altNumber) { this.altNum = altNumber; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("}\n");
grammarBuilder.append("\n");
grammarBuilder.append("\n");
grammarBuilder.append("s\n");
grammarBuilder.append("@init {\n");
grammarBuilder.append("this.buildParseTrees = true;\n");
grammarBuilder.append("}\n");
grammarBuilder.append("@after {\n");
grammarBuilder.append("console.log($r.ctx.toStringTree(null, this));\n");
grammarBuilder.append("}\n");
grammarBuilder.append(" : r=a ;\n");
grammarBuilder.append("\n");
grammarBuilder.append("a : 'f'\n");
grammarBuilder.append(" | 'g'\n");
grammarBuilder.append(" | 'x' b 'z'\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("b : 'e' {} | 'y'\n");
grammarBuilder.append(" ;");
String grammar = grammarBuilder.toString();
String input ="xyz";
String found = execParser("T.g4", grammar, "TParser", "TLexer",
"TListener", "TVisitor",
"s", input, false);
assertEquals("(a:3 x (b:2 y) z)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test

54
runtime-testsuite/test/org/antlr/v4/test/runtime/python2/TestLeftRecursion.java
View File

@ -2058,6 +2058,60 @@ public class TestLeftRecursion extends BasePython2Test {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_1() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(216);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {print($ctx.toStringTree(recog=self))} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(s (expr (literal - 1)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_2() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(216);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {print($ctx.toStringTree(recog=self))} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1 + -1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixOpWithActionAndLabel_1() throws Exception {

53
runtime-testsuite/test/org/antlr/v4/test/runtime/python2/TestParseTrees.java
View File

@ -1,9 +1,10 @@
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.python2;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
@SuppressWarnings("unused")
public class TestParseTrees extends BasePython2Test {
@ -64,6 +65,54 @@ public class TestParseTrees extends BasePython2Test {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testAltNum() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(562);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("\n");
grammarBuilder.append("options { contextSuperClass=MyRuleNode; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("@parser::members {\n");
grammarBuilder.append("class MyRuleNode(ParserRuleContext):\n");
grammarBuilder.append(" def __init__(self, parent = None, invokingStateNumber = None ):\n");
grammarBuilder.append(" super(TParser.MyRuleNode, self).__init__(parent, invokingStateNumber)\n");
grammarBuilder.append(" self.altNum = 0;\n");
grammarBuilder.append(" def getAltNumber(self):\n");
grammarBuilder.append(" return self.altNum\n");
grammarBuilder.append(" def setAltNumber(self, altNum):\n");
grammarBuilder.append(" self.altNum = altNum\n");
grammarBuilder.append("}\n");
grammarBuilder.append("\n");
grammarBuilder.append("\n");
grammarBuilder.append("s\n");
grammarBuilder.append("@init {\n");
grammarBuilder.append("self._buildParseTrees = True\n");
grammarBuilder.append("}\n");
grammarBuilder.append("@after {\n");
grammarBuilder.append("print($r.ctx.toStringTree(recog=self))\n");
grammarBuilder.append("}\n");
grammarBuilder.append(" : r=a ;\n");
grammarBuilder.append("\n");
grammarBuilder.append("a : 'f'\n");
grammarBuilder.append(" | 'g'\n");
grammarBuilder.append(" | 'x' b 'z'\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("b : 'e' {} | 'y'\n");
grammarBuilder.append(" ;");
String grammar = grammarBuilder.toString();
String input ="xyz";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(a:3 x (b:2 y) z)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testExtraToken() throws Exception {

4
runtime-testsuite/test/org/antlr/v4/test/runtime/python3/BasePython3Test.java
View File

@ -41,8 +41,8 @@ public abstract class BasePython3Test extends BasePythonTest {
@Override
protected String getPythonExecutable() {
return "python3.4";
}
return "python3.5";
} // force 3.5
@Override
protected void writeLexerTestFile(String lexerName, boolean showDFA) {

54
runtime-testsuite/test/org/antlr/v4/test/runtime/python3/TestLeftRecursion.java
View File

@ -2058,6 +2058,60 @@ public class TestLeftRecursion extends BasePython3Test {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_1() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(216);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {print($ctx.toStringTree(recog=self))} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(s (expr (literal - 1)) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixAndOtherAlt_2() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(216);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("s @after {print($ctx.toStringTree(recog=self))} : expr EOF ; \n");
grammarBuilder.append("expr : literal\n");
grammarBuilder.append(" | op expr\n");
grammarBuilder.append(" | expr op expr\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("literal : '-'? Integer ;\n");
grammarBuilder.append("op : '+' | '-' ;\n");
grammarBuilder.append("Integer : [0-9]+ ;\n");
grammarBuilder.append("WS : (' '|'\\n') -> skip ;");
String grammar = grammarBuilder.toString();
String input ="-1 + -1";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(s (expr (expr (literal - 1)) (op +) (expr (literal - 1))) <EOF>)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testPrefixOpWithActionAndLabel_1() throws Exception {

53
runtime-testsuite/test/org/antlr/v4/test/runtime/python3/TestParseTrees.java
View File

@ -1,9 +1,10 @@
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.python3;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNull;
@SuppressWarnings("unused")
public class TestParseTrees extends BasePython3Test {
@ -64,6 +65,54 @@ public class TestParseTrees extends BasePython3Test {
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testAltNum() throws Exception {
mkdir(tmpdir);
StringBuilder grammarBuilder = new StringBuilder(584);
grammarBuilder.append("grammar T;\n");
grammarBuilder.append("\n");
grammarBuilder.append("options { contextSuperClass=MyRuleNode; }\n");
grammarBuilder.append("\n");
grammarBuilder.append("@parser::members {\n");
grammarBuilder.append("class MyRuleNode(ParserRuleContext):\n");
grammarBuilder.append(" def __init__(self, parent:ParserRuleContext = None, invokingStateNumber:int = None ):\n");
grammarBuilder.append(" super(TParser.MyRuleNode, self).__init__(parent, invokingStateNumber)\n");
grammarBuilder.append(" self.altNum = 0;\n");
grammarBuilder.append(" def getAltNumber(self):\n");
grammarBuilder.append(" return self.altNum\n");
grammarBuilder.append(" def setAltNumber(self, altNum):\n");
grammarBuilder.append(" self.altNum = altNum\n");
grammarBuilder.append("}\n");
grammarBuilder.append("\n");
grammarBuilder.append("\n");
grammarBuilder.append("s\n");
grammarBuilder.append("@init {\n");
grammarBuilder.append("self._buildParseTrees = True\n");
grammarBuilder.append("}\n");
grammarBuilder.append("@after {\n");
grammarBuilder.append("print($r.ctx.toStringTree(recog=self))\n");
grammarBuilder.append("}\n");
grammarBuilder.append(" : r=a ;\n");
grammarBuilder.append("\n");
grammarBuilder.append("a : 'f'\n");
grammarBuilder.append(" | 'g'\n");
grammarBuilder.append(" | 'x' b 'z'\n");
grammarBuilder.append(" ;\n");
grammarBuilder.append("b : 'e' {} | 'y'\n");
grammarBuilder.append(" ;");
String grammar = grammarBuilder.toString();
String input ="xyz";
String found = execParser("T.g4", grammar, "TParser", "TLexer", "TListener", "TVisitor", "s", input, false);
assertEquals("(a:3 x (b:2 y) z)\n", found);
assertNull(this.stderrDuringParse);
}
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
@Test
public void testExtraToken() throws Exception {

1
runtime/CSharp/runtime/CSharp/Antlr4.Runtime/Parser.cs
View File

@ -842,6 +842,7 @@ namespace Antlr4.Runtime
public virtual void EnterOuterAlt(ParserRuleContext localctx, int altNum)
{
localctx.setAltNumber(altNum);
// if we have new localctx, make sure we replace existing ctx
// that is previous child of parse tree
if (_buildParseTrees && _ctx != localctx)

6
runtime/CSharp/runtime/CSharp/Antlr4.Runtime/Properties/AssemblyInfo.cs
View File

@ -67,8 +67,8 @@ using System.Runtime.InteropServices;
// You can specify all the values or you can default the Build and Revision Numbers
// by using the '*' as shown below:
// [assembly: AssemblyVersion("1.0.*")]
[assembly: AssemblyVersion("4.5.2.0")]
[assembly: AssemblyVersion("4.5.3.0")]
#if !COMPACT
[assembly: AssemblyFileVersion("4.5.2.0")]
[assembly: AssemblyInformationalVersion("4.5.2.0")]
[assembly: AssemblyFileVersion("4.5.3.0")]
[assembly: AssemblyInformationalVersion("4.5.3.0")]
#endif

18
runtime/CSharp/runtime/CSharp/Antlr4.Runtime/RuleContext.cs
View File

@ -30,6 +30,7 @@
using System.Collections.Generic;
using System.Text;
using Antlr4.Runtime;
using Antlr4.Runtime.Atn;
using Antlr4.Runtime.Misc;
using Antlr4.Runtime.Sharpen;
using Antlr4.Runtime.Tree;
@ -216,6 +217,23 @@ namespace Antlr4.Runtime
}
}
/* For rule associated with this parse tree internal node, return
* the outer alternative number used to match the input. Default
* implementation does not compute nor store this alt num. Create
* a subclass of ParserRuleContext with backing field and set
* option contextSuperClass.
* to set it.
*/
public virtual int getAltNumber() { return Atn.ATN.InvalidAltNumber; }
/* Set the outer alternative number for this context node. Default
* implementation does nothing to avoid backing field overhead for
* trees that don't need it. Create
* a subclass of ParserRuleContext with backing field and set
* option contextSuperClass.
*/
public virtual void setAltNumber(int altNumber) { }
public virtual IParseTree GetChild(int i)
{
return null;

9
runtime/CSharp/runtime/CSharp/Antlr4.Runtime/Tree/Trees.cs
View File

@ -30,6 +30,7 @@
using System.Collections.Generic;
using System.Text;
using Antlr4.Runtime;
using Antlr4.Runtime.Atn;
using Antlr4.Runtime.Misc;
using Antlr4.Runtime.Sharpen;
using Antlr4.Runtime.Tree;
@ -111,10 +112,14 @@ namespace Antlr4.Runtime.Tree
{
if (ruleNames != null)
{
if (t is IRuleNode)
if (t is RuleContext)
{
int ruleIndex = ((IRuleNode)t).RuleContext.RuleIndex;
int ruleIndex = ((RuleContext)t).RuleIndex;
string ruleName = ruleNames[ruleIndex];
int altNumber = ((RuleContext)t).getAltNumber();
if ( altNumber!=Atn.ATN.InvalidAltNumber ) {
return ruleName+":"+altNumber;
}
return ruleName;
}
else

15
runtime/CSharp/runtime/CSharp/Antlr4.Runtime/Vocabulary.cs
View File

@ -67,6 +67,8 @@ namespace Antlr4.Runtime
[NotNull]
private readonly string[] displayNames;
private readonly int maxTokenType;
/// <summary>
/// Constructs a new instance of
/// <see cref="Vocabulary"/>
@ -126,6 +128,19 @@ namespace Antlr4.Runtime
this.literalNames = literalNames != null ? literalNames : EmptyNames;
this.symbolicNames = symbolicNames != null ? symbolicNames : EmptyNames;
this.displayNames = displayNames != null ? displayNames : EmptyNames;
this.maxTokenType =
System.Math.Max(this.displayNames.Length,
System.Math.Max(this.literalNames.Length, this.symbolicNames.Length)) - 1;
}
/// <summary>
/// Returns the highest token type value. It can be used to iterate from
/// zero to that number, inclusively, thus querying all stored entries.
/// </summary>
public virtual int getMaxTokenType()
{
return maxTokenType;
}
[return: Nullable]

2
runtime/Java/pom.xml
View File

@ -3,7 +3,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
<relativePath>../../pom.xml</relativePath>
</parent>
<artifactId>antlr4-runtime</artifactId>

3
runtime/Java/src/org/antlr/v4/runtime/Parser.java
View File

@ -34,7 +34,6 @@ import org.antlr.v4.runtime.atn.ATNDeserializationOptions;
import org.antlr.v4.runtime.atn.ATNDeserializer;
import org.antlr.v4.runtime.atn.ATNSimulator;
import org.antlr.v4.runtime.atn.ATNState;
import org.antlr.v4.runtime.atn.AmbiguityInfo;
import org.antlr.v4.runtime.atn.ParseInfo;
import org.antlr.v4.runtime.atn.ParserATNSimulator;
import org.antlr.v4.runtime.atn.PredictionMode;
@ -51,7 +50,6 @@ import org.antlr.v4.runtime.tree.pattern.ParseTreePattern;
import org.antlr.v4.runtime.tree.pattern.ParseTreePatternMatcher;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
import java.util.List;
import java.util.Map;
@ -649,6 +647,7 @@ public abstract class Parser extends Recognizer<Token, ParserATNSimulator> {
}
public void enterOuterAlt(ParserRuleContext localctx, int altNum) {
localctx.setAltNumber(altNum);
// if we have new localctx, make sure we replace existing ctx
// that is previous child of parse tree
if ( _buildParseTrees && _ctx != localctx ) {

28
runtime/Java/src/org/antlr/v4/runtime/RuleContext.java
View File

@ -29,20 +29,17 @@
*/
package org.antlr.v4.runtime;
import org.antlr.v4.runtime.atn.ATN;
import org.antlr.v4.runtime.misc.Interval;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeVisitor;
import org.antlr.v4.runtime.tree.RuleNode;
import org.antlr.v4.runtime.tree.Trees;
import javax.print.PrintException;
import javax.swing.*;
import java.io.IOException;
import java.util.Arrays;
import java.util.List;
import java.util.concurrent.Future;
/** /** A rule context is a record of a single rule invocation.
/** A rule context is a record of a single rule invocation.
*
* We form a stack of these context objects using the parent
* pointer. A parent pointer of null indicates that the current
@ -169,6 +166,27 @@ public class RuleContext implements RuleNode {
public int getRuleIndex() { return -1; }
/** For rule associated with this parse tree internal node, return
* the outer alternative number used to match the input. Default
* implementation does not compute nor store this alt num. Create
* a subclass of ParserRuleContext with backing field and set
* option contextSuperClass.
* to set it.
*
* @since 4.5.3
*/
public int getAltNumber() { return ATN.INVALID_ALT_NUMBER; }
/** Set the outer alternative number for this context node. Default
* implementation does nothing to avoid backing field overhead for
* trees that don't need it. Create
* a subclass of ParserRuleContext with backing field and set
* option contextSuperClass.
*
* @since 4.5.3
*/
public void setAltNumber(int altNumber) { }
@Override
public ParseTree getChild(int i) {
return null;

24
runtime/Java/src/org/antlr/v4/runtime/RuleContextWithAltNum.java Normal file
View File

@ -0,0 +1,24 @@
package org.antlr.v4.runtime;
import org.antlr.v4.runtime.atn.ATN;
/** A handy class for use with
*
* options {contextSuperClass=org.antlr.v4.runtime.RuleContextWithAltNum;}
*
* that provides a backing field / impl for the outer alternative number
* matched for an internal parse tree node.
*
* I'm only putting into Java runtime as I'm certain I'm the only one that
* will really every use this.
*/
public class RuleContextWithAltNum extends ParserRuleContext {
public int altNum;
public RuleContextWithAltNum() { altNum = ATN.INVALID_ALT_NUMBER; }
public RuleContextWithAltNum(ParserRuleContext parent, int invokingStateNumber) {
super(parent, invokingStateNumber);
}
@Override public int getAltNumber() { return altNum; }
@Override public void setAltNumber(int altNum) { this.altNum = altNum; }
}

2
runtime/Java/src/org/antlr/v4/runtime/RuntimeMetaData.java
View File

@ -91,7 +91,7 @@ public class RuntimeMetaData {
* omitted.</li>
* </ul>
*/
public static final String VERSION = "4.5.2";
public static final String VERSION = "4.5.3";
/**
* Gets the currently executing version of the ANTLR 4 runtime library.

9
runtime/Java/src/org/antlr/v4/runtime/Vocabulary.java
View File

@ -37,6 +37,13 @@ package org.antlr.v4.runtime;
* @author Sam Harwell
*/
public interface Vocabulary {
/**
* Returns the highest token type value. It can be used to iterate from
* zero to that number, inclusively, thus querying all stored entries.
* @return the highest token type value
*/
int getMaxTokenType();
/**
* Gets the string literal associated with a token type. The string returned
* by this method, when not {@code null}, can be used unaltered in a parser
@ -85,7 +92,7 @@ public interface Vocabulary {
*
* <ul>
* <li>Tokens created by lexer rules.</li>
* <li>Tokens defined in a {@code tokens{}} block in a lexer or parser
* <li>Tokens defined in a <code>tokens{}</code> block in a lexer or parser
* grammar.</li>
* <li>The implicitly defined {@code EOF} token, which has the token type
* {@link Token#EOF}.</li>

11
runtime/Java/src/org/antlr/v4/runtime/VocabularyImpl.java
View File

@ -57,6 +57,8 @@ public class VocabularyImpl implements Vocabulary {
private final String[] displayNames;
private final int maxTokenType;
/**
* Constructs a new instance of {@link VocabularyImpl} from the specified
* literal and symbolic token names.
@ -94,6 +96,10 @@ public class VocabularyImpl implements Vocabulary {
this.literalNames = literalNames != null ? literalNames : EMPTY_NAMES;
this.symbolicNames = symbolicNames != null ? symbolicNames : EMPTY_NAMES;
this.displayNames = displayNames != null ? displayNames : EMPTY_NAMES;
// See note here on -1 part: https://github.com/antlr/antlr4/pull/1146
this.maxTokenType =
Math.max(this.displayNames.length,
Math.max(this.literalNames.length, this.symbolicNames.length)) - 1;
}
/**
@ -143,6 +149,11 @@ public class VocabularyImpl implements Vocabulary {
return new VocabularyImpl(literalNames, symbolicNames, tokenNames);
}
@Override
public int getMaxTokenType() {
return maxTokenType;
}
@Override
public String getLiteralName(int tokenType) {
if (tokenType >= 0 && tokenType < literalNames.length) {

10
runtime/Java/src/org/antlr/v4/runtime/tree/Trees.java
View File

@ -33,7 +33,9 @@ package org.antlr.v4.runtime.tree;
import org.antlr.v4.runtime.CommonToken;
import org.antlr.v4.runtime.Parser;
import org.antlr.v4.runtime.ParserRuleContext;
import org.antlr.v4.runtime.RuleContext;
import org.antlr.v4.runtime.Token;
import org.antlr.v4.runtime.atn.ATN;
import org.antlr.v4.runtime.misc.Interval;
import org.antlr.v4.runtime.misc.Predicate;
import org.antlr.v4.runtime.misc.Utils;
@ -91,9 +93,13 @@ public class Trees {
public static String getNodeText(Tree t, List<String> ruleNames) {
if ( ruleNames!=null ) {
if ( t instanceof RuleNode ) {
int ruleIndex = ((RuleNode)t).getRuleContext().getRuleIndex();
if ( t instanceof RuleContext ) {
int ruleIndex = ((RuleContext)t).getRuleContext().getRuleIndex();
String ruleName = ruleNames.get(ruleIndex);
int altNumber = ((RuleContext) t).getAltNumber();
if ( altNumber!=ATN.INVALID_ALT_NUMBER ) {
return ruleName+":"+altNumber;
}
return ruleName;
}
else if ( t instanceof ErrorNode) {

1
runtime/JavaScript/src/antlr4/Parser.js
View File

@ -451,6 +451,7 @@ Parser.prototype.exitRule = function() {
};
Parser.prototype.enterOuterAlt = function(localctx, altNum) {
localctx.setAltNumber(altNum);
// if we have new localctx, make sure we replace existing ctx
// that is previous child of parse tree
if (this.buildParseTrees && this._ctx !== localctx) {

2
runtime/JavaScript/src/antlr4/Recognizer.js
View File

@ -45,7 +45,7 @@ Recognizer.ruleIndexMapCache = {};
Recognizer.prototype.checkVersion = function(toolVersion) {
var runtimeVersion = "4.5.2";
var runtimeVersion = "4.5.3";
if (runtimeVersion!==toolVersion) {
console.log("ANTLR runtime and generated code versions disagree: "+runtimeVersion+"!="+toolVersion);
}

16
runtime/JavaScript/src/antlr4/RuleContext.js
View File

@ -51,6 +51,7 @@
var RuleNode = require('./tree/Tree').RuleNode;
var INVALID_INTERVAL = require('./tree/Tree').INVALID_INTERVAL;
var INVALID_ALT_NUMBER = require('./atn/ATN').INVALID_ALT_NUMBER;
function RuleContext(parent, invokingState) {
RuleNode.call(this);
@ -113,6 +114,21 @@ RuleContext.prototype.getText = function() {
}
};
// For rule associated with this parse tree internal node, return
// the outer alternative number used to match the input. Default
// implementation does not compute nor store this alt num. Create
// a subclass of ParserRuleContext with backing field and set
// option contextSuperClass.
// to set it.
RuleContext.prototype.getAltNumber = function() { return INVALID_ALT_NUMBER; }
// Set the outer alternative number for this context node. Default
// implementation does nothing to avoid backing field overhead for
// trees that don't need it. Create
// a subclass of ParserRuleContext with backing field and set
// option contextSuperClass.
RuleContext.prototype.setAltNumber = function(altNumber) { }
RuleContext.prototype.getChild = function(i) {
return null;
};

2
runtime/JavaScript/src/antlr4/atn/ATNConfig.js
View File

@ -50,7 +50,7 @@ function checkParams(params, isCfg) {
} else {
var props = {};
props.state = params.state || null;
props.alt = params.alt || null;
props.alt = (params.alt === undefined) ? null : params.alt;
props.context = params.context || null;
props.semanticContext = params.semanticContext || null;
if(isCfg) {

13
runtime/JavaScript/src/antlr4/atn/LexerActionExecutor.js
View File

@ -166,9 +166,18 @@ LexerActionExecutor.prototype.equals = function(other) {
return true;
} else if (!(other instanceof LexerActionExecutor)) {
return false;
} else if (this._hashString != other._hashString) {
return false;
} else if (this.lexerActions.length != other.lexerActions.length) {
return false;
} else {
return this._hashString === other._hashString &&
this.lexerActions === other.lexerActions;
var numActions = this.lexerActions.length
for (var idx = 0; idx < numActions; ++idx) {
if (!this.lexerActions[idx].equals(other.lexerActions[idx])) {
return false;
}
}
return true;
}
};

4
runtime/JavaScript/src/antlr4/atn/SemanticContext.js
View File

@ -301,7 +301,7 @@ AND.prototype.evalPrecedence = function(parser, outerContext) {
}
var result = null;
operands.map(function(o) {
result = result === null ? o : SemanticPredicate.andContext(result, o);
result = result === null ? o : SemanticContext.andContext(result, o);
});
return result;
};
@ -408,7 +408,7 @@ OR.prototype.evalPrecedence = function(parser, outerContext) {
return result;
};
AND.prototype.toString = function() {
OR.prototype.toString = function() {
var s = "";
this.opnds.map(function(o) {
s += "|| " + o.toString();

2
runtime/JavaScript/src/antlr4/package.json
View File

@ -1,6 +1,6 @@
{
"name": "antlr4",
"version": "4.5.2",
"version": "4.5.3",
"description": "JavaScript runtime for ANTLR4",
"main": "src/antlr4/index.js",
"repository": "antlr/antlr4.git",

10
runtime/JavaScript/src/antlr4/tree/Trees.js
View File

@ -34,6 +34,8 @@ var RuleNode = require('./Tree').RuleNode;
var ErrorNode = require('./Tree').ErrorNode;
var TerminalNode = require('./Tree').TerminalNode;
var ParserRuleContext = require('./../ParserRuleContext').ParserRuleContext;
var RuleContext = require('./../RuleContext').RuleContext;
var INVALID_ALT_NUMBER = require('./../atn/ATN').INVALID_ALT_NUMBER;
/** A set of utility routines useful for all kinds of ANTLR trees. */
@ -75,8 +77,12 @@ Trees.getNodeText = function(t, ruleNames, recog) {
ruleNames = recog.ruleNames;
}
if(ruleNames!==null) {
if (t instanceof RuleNode) {
return ruleNames[t.getRuleContext().ruleIndex];
if (t instanceof RuleContext) {
var altNumber = t.getAltNumber();
if ( altNumber!=INVALID_ALT_NUMBER ) {
return ruleNames[t.ruleIndex]+":"+altNumber;
}
return ruleNames[t.ruleIndex];
} else if ( t instanceof ErrorNode) {
return t.toString();
} else if(t instanceof TerminalNode) {

4
runtime/Python2/setup.py
View File

@ -2,12 +2,12 @@ from distutils.core import setup
setup(
name='antlr4-python2-runtime',
version='4.5.2.1',
version='4.5.3',
packages=['antlr4', 'antlr4.atn', 'antlr4.dfa', 'antlr4.tree', 'antlr4.error', 'antlr4.xpath'],
package_dir={'': 'src'},
url='http://www.antlr.org',
license='BSD',
author='Eric Vergnaud, Terence Parr, Sam Harwell',
author_email='eric.vergnaud@wanadoo.fr',
description='ANTLR 4.5.2.1 runtime for Python 2.7.6'
description='ANTLR 4.5.3 runtime for Python 2.7.6'
)

1
runtime/Python2/src/antlr4/Parser.py
View File

@ -382,6 +382,7 @@ class Parser (Recognizer):
self._ctx = self._ctx.parentCtx
def enterOuterAlt(self, localctx, altNum):
localctx.setAltNumber(altNum)
# if we have new localctx, make sure we replace existing ctx
# that is previous child of parse tree
if self.buildParseTrees and self._ctx != localctx:

2
runtime/Python2/src/antlr4/Recognizer.py
View File

@ -55,7 +55,7 @@ class Recognizer(object):
return major, minor
def checkVersion(self, toolVersion):
runtimeVersion = "4.5.2.1"
runtimeVersion = "4.5.3"
rvmajor, rvminor = self.extractVersion(runtimeVersion)
tvmajor, tvminor = self.extractVersion(toolVersion)
if rvmajor!=tvmajor or rvminor!=tvminor:

18
runtime/Python2/src/antlr4/RuleContext.py
View File

@ -52,6 +52,7 @@
from io import StringIO
from antlr4.tree.Tree import RuleNode, INVALID_INTERVAL
from antlr4.tree.Trees import Trees
from antlr4.atn.ATN import ATN
class RuleContext(RuleNode):
@ -109,6 +110,23 @@ class RuleContext(RuleNode):
def getRuleIndex(self):
return -1
# For rule associated with this parse tree internal node, return
# the outer alternative number used to match the input. Default
# implementation does not compute nor store this alt num. Create
# a subclass of ParserRuleContext with backing field and set
# option contextSuperClass.
# to set it.
def getAltNumber(self):
return ATN.INVALID_ALT_NUMBER
# Set the outer alternative number for this context node. Default
# implementation does nothing to avoid backing field overhead for
# trees that don't need it. Create
# a subclass of ParserRuleContext with backing field and set
# option contextSuperClass.
def setAltNumber(self, altNumber):
pass
def getChild(self, i):
return None

6
runtime/Python2/src/antlr4/tree/Trees.py
View File

@ -32,6 +32,8 @@
# A set of utility routines useful for all kinds of ANTLR trees.#
from io import StringIO
import antlr4
from antlr4.atn.ATN import ATN
from antlr4.Token import Token
from antlr4.Utils import escapeWhitespace
from antlr4.tree.Tree import RuleNode, ErrorNode, TerminalNode
@ -65,7 +67,9 @@ class Trees(object):
ruleNames = recog.ruleNames
if ruleNames is not None:
if isinstance(t, RuleNode):
return ruleNames[t.getRuleContext().getRuleIndex()]
if t.getAltNumber()!=ATN.INVALID_ALT_NUMBER:
return ruleNames[t.getRuleIndex()]+":"+str(t.getAltNumber())
return ruleNames[t.getRuleIndex()]
elif isinstance( t, ErrorNode):
return unicode(t)
elif isinstance(t, TerminalNode):

4
runtime/Python3/setup.py
View File

@ -2,12 +2,12 @@ from distutils.core import setup
setup(
name='antlr4-python3-runtime',
version='4.5.2.1',
version='4.5.3',
packages=['antlr4', 'antlr4.atn', 'antlr4.dfa', 'antlr4.tree', 'antlr4.error', 'antlr4.xpath'],
package_dir={'': 'src'},
url='http://www.antlr.org',
license='BSD',
author='Eric Vergnaud, Terence Parr, Sam Harwell',
author_email='eric.vergnaud@wanadoo.fr',
description='ANTLR 4.5.2.1 runtime for Python 3.4.0'
description='ANTLR 4.5.3 runtime for Python 3.4.0'
)

1
runtime/Python3/src/antlr4/Parser.py
View File

@ -389,6 +389,7 @@ class Parser (Recognizer):
self._ctx = self._ctx.parentCtx
def enterOuterAlt(self, localctx:ParserRuleContext, altNum:int):
localctx.setAltNumber(altNum)
# if we have new localctx, make sure we replace existing ctx
# that is previous child of parse tree
if self.buildParseTrees and self._ctx != localctx:

2
runtime/Python3/src/antlr4/Recognizer.py
View File

@ -58,7 +58,7 @@ class Recognizer(object):
return major, minor
def checkVersion(self, toolVersion):
runtimeVersion = "4.5.2.1"
runtimeVersion = "4.5.3"
rvmajor, rvminor = self.extractVersion(runtimeVersion)
tvmajor, tvminor = self.extractVersion(toolVersion)
if rvmajor!=tvmajor or rvminor!=tvminor:

17
runtime/Python3/src/antlr4/RuleContext.py
View File

@ -113,6 +113,23 @@ class RuleContext(RuleNode):
def getRuleIndex(self):
return -1
# For rule associated with this parse tree internal node, return
# the outer alternative number used to match the input. Default
# implementation does not compute nor store this alt num. Create
# a subclass of ParserRuleContext with backing field and set
# option contextSuperClass.
# to set it.
def getAltNumber(self):
return 0 # should use ATN.INVALID_ALT_NUMBER but won't compile
# Set the outer alternative number for this context node. Default
# implementation does nothing to avoid backing field overhead for
# trees that don't need it. Create
# a subclass of ParserRuleContext with backing field and set
# option contextSuperClass.
def setAltNumber(self, altNumber:int):
pass
def getChild(self, i:int):
return None

4
runtime/Python3/src/antlr4/tree/Trees.py
View File

@ -68,7 +68,9 @@ class Trees(object):
ruleNames = recog.ruleNames
if ruleNames is not None:
if isinstance(t, RuleNode):
return ruleNames[t.getRuleContext().getRuleIndex()]
if t.getAltNumber()!=0: # should use ATN.INVALID_ALT_NUMBER but won't compile
return ruleNames[t.getRuleIndex()]+":"+str(t.getAltNumber())
return ruleNames[t.getRuleIndex()]
elif isinstance( t, ErrorNode):
return str(t)
elif isinstance(t, TerminalNode):

2
tool-testsuite/pom.xml
View File

@ -4,7 +4,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
</parent>
<artifactId>antlr4-tool-testsuite</artifactId>
<name>ANTLR 4 Tool Tests</name>

7
tool-testsuite/test/org/antlr/v4/test/tool/InterpreterTreeTextProvider.java
View File

@ -1,10 +1,9 @@
package org.antlr.v4.test.tool;
import org.antlr.v4.gui.TreeTextProvider;
import org.antlr.v4.runtime.tree.ErrorNode;
import org.antlr.v4.runtime.tree.Tree;
import org.antlr.v4.runtime.tree.Trees;
import org.antlr.v4.gui.TreeTextProvider;
import org.antlr.v4.tool.GrammarInterpreterRuleContext;
import java.util.Arrays;
import java.util.List;
@ -17,10 +16,6 @@ public class InterpreterTreeTextProvider implements TreeTextProvider {
public String getText(Tree node) {
if ( node==null ) return "null";
String nodeText = Trees.getNodeText(node, ruleNames);
if ( node instanceof GrammarInterpreterRuleContext) {
GrammarInterpreterRuleContext ctx = (GrammarInterpreterRuleContext) node;
return nodeText+":"+ctx.getOuterAltNum();
}
if ( node instanceof ErrorNode) {
return "<error "+nodeText+">";
}

10
tool-testsuite/test/org/antlr/v4/test/tool/TestActionTranslation.java
View File

@ -31,6 +31,7 @@
package org.antlr.v4.test.tool;
import org.antlr.v4.test.runtime.java.BaseTest;
import org.antlr.v4.tool.Grammar;
import org.junit.Test;
/** */
@ -180,6 +181,15 @@ public class TestActionTranslation extends BaseTest {
testActions(attributeTemplate, "finally", action, expected);
}
@Test public void testEmptyActions() throws Exception {
String gS =
"grammar A;\n"+
"a[] : 'a' ;\n" +
"c : a[] c[] ;\n";
Grammar g = new Grammar(gS);
}
@Test public void testDynamicRuleScopeRefInSubrule() throws Exception {
String action = "$a::n;";
}

2
tool/pom.xml
View File

@ -3,7 +3,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.3-SNAPSHOT</version>
<version>4.5.4-SNAPSHOT</version>
</parent>
<artifactId>antlr4</artifactId>
<name>ANTLR 4 Tool</name>

4
tool/resources/org/antlr/v4/tool/templates/codegen/CSharp/CSharp.stg
View File

@ -30,7 +30,7 @@
// args must be <object-model-object>, <fields-resulting-in-STs>
ParserFile(file, parser, namedActions) ::= <<
ParserFile(file, parser, namedActions, contextSuperClass) ::= <<
<fileHeader(file.grammarFileName, file.ANTLRVersion)>
<if(file.genPackage)>
namespace <file.genPackage> {
@ -839,7 +839,7 @@ CaptureNextTokenType(d) ::= "<d.varName> = TokenStream.La(1);"
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers,
superClass={ParserRuleContext}) ::= <<
public partial class <struct.name> : <superClass><if(interfaces)>, <interfaces; separator=", "><endif> {
public partial class <struct.name> : <if(contextSuperClass)><contextSuperClass><else>ParserRuleContext<endif><if(interfaces)>, <interfaces; separator=", "><endif> {
<attrs:{a | public <a>;}; separator="\n">
<getters:{g | <g>}; separator="\n">
<if(ctorAttrs)>public <struct.name>(ParserRuleContext parent, int invokingState) : base(parent, invokingState) { }<endif>

8
tool/resources/org/antlr/v4/tool/templates/codegen/Java/Java.stg
View File

@ -42,7 +42,7 @@ javaTypeInitMap ::= [
// args must be <object-model-object>, <fields-resulting-in-STs>
ParserFile(file, parser, namedActions) ::= <<
ParserFile(file, parser, namedActions, contextSuperClass) ::= <<
<fileHeader(file.grammarFileName, file.ANTLRVersion)>
<if(file.genPackage)>
package <file.genPackage>;
@ -765,9 +765,9 @@ ListLabelName(label) ::= "<label>"
CaptureNextToken(d) ::= "<d.varName> = _input.LT(1);"
CaptureNextTokenType(d) ::= "<d.varName> = _input.LA(1);"
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers,
superClass={ParserRuleContext}) ::= <<
public static class <struct.name> extends <superClass><if(interfaces)> implements <interfaces; separator=", "><endif> {
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers)
::= <<
public static class <struct.name> extends <if(contextSuperClass)><contextSuperClass><else>ParserRuleContext<endif><if(interfaces)> implements <interfaces; separator=", "><endif> {
<attrs:{a | public <a>;}; separator="\n">
<getters:{g | <g>}; separator="\n">
<if(ctorAttrs)>public <struct.name>(ParserRuleContext parent, int invokingState) { super(parent, invokingState); }<endif>

11
tool/resources/org/antlr/v4/tool/templates/codegen/JavaScript/JavaScript.stg
View File

@ -46,7 +46,7 @@ pythonTypeInitMap ::= [
// args must be <object-model-object>, <fields-resulting-in-STs>
ParserFile(file, parser, namedActions) ::= <<
ParserFile(file, parser, namedActions, contextSuperClass) ::= <<
<fileHeader(file.grammarFileName, file.ANTLRVersion)>
var antlr4 = require('antlr4/index');
<if(file.genListener)>
@ -671,8 +671,7 @@ ListLabelName(label) ::= "<label>"
CaptureNextToken(d) ::= "<d.varName> = self._input.LT(1)"
CaptureNextTokenType(d) ::= "<d.varName> = this._input.LA(1);"
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers,
superClass={antlr4.ParserRuleContext}) ::= <<
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers) ::= <<
function <struct.name>(parser, parent, invokingState<struct.ctorAttrs:{a | , <a.name>}>) {
if(parent===undefined) {
parent = null;
@ -680,7 +679,7 @@ function <struct.name>(parser, parent, invokingState<struct.ctorAttrs:{a | , <a.
if(invokingState===undefined || invokingState===null) {
invokingState = -1;
}
<superClass>.call(this, parent, invokingState);
<if(contextSuperClass)><contextSuperClass><else>antlr4.ParserRuleContext<endif>.call(this, parent, invokingState);
this.parser = parser;
this.ruleIndex = <parser.name>.RULE_<struct.derivedFromName>;
<attrs:{a | <a>}; separator="\n">
@ -688,14 +687,14 @@ function <struct.name>(parser, parent, invokingState<struct.ctorAttrs:{a | , <a.
return this;
}
<struct.name>.prototype = Object.create(<superClass>.prototype);
<struct.name>.prototype = Object.create(<if(contextSuperClass)><contextSuperClass><else>antlr4.ParserRuleContext<endif>.prototype);
<struct.name>.prototype.constructor = <struct.name>;
<getters:{g | <struct.name>.prototype.<g>}; separator="\n\n">
<if(struct.provideCopyFrom)> <! don't need copy unless we have subclasses !>
<struct.name>.prototype.copyFrom = function(ctx) {
<superClass>.prototype.copyFrom.call(this, ctx);
<if(contextSuperClass)><contextSuperClass><else>antlr4.ParserRuleContext<endif>.prototype.copyFrom.call(this, ctx);
<struct.attrs:{a | this.<a.name> = ctx.<a.name>;}; separator="\n">
};
<endif>

7
tool/resources/org/antlr/v4/tool/templates/codegen/Python2/Python2.stg
View File

@ -46,7 +46,7 @@ pythonTypeInitMap ::= [
// args must be <object-model-object>, <fields-resulting-in-STs>
ParserFile(file, parser, namedActions) ::= <<
ParserFile(file, parser, namedActions, contextSuperClass) ::= <<
<fileHeader(file.grammarFileName, file.ANTLRVersion)>
# encoding: utf-8
from __future__ import print_function
@ -630,9 +630,8 @@ ListLabelName(label) ::= "<label>"
CaptureNextToken(d) ::= "<d.varName> = self._input.LT(1)"
CaptureNextTokenType(d) ::= "<d.varName> = self._input.LA(1)"
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers,
superClass={ParserRuleContext}) ::= <<
class <struct.name>(<superClass>):
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers) ::= <<
class <struct.name>(<if(contextSuperClass)><contextSuperClass><else>ParserRuleContext<endif>):
def __init__(self, parser, parent=None, invokingState=-1<struct.ctorAttrs:{a | , <a.name>=None}>):
super(<parser.name>.<struct.name>, self).__init__(parent, invokingState)

7
tool/resources/org/antlr/v4/tool/templates/codegen/Python3/Python3.stg
View File

@ -46,7 +46,7 @@ pythonTypeInitMap ::= [
// args must be <object-model-object>, <fields-resulting-in-STs>
ParserFile(file, parser, namedActions) ::= <<
ParserFile(file, parser, namedActions, contextSuperClass) ::= <<
<fileHeader(file.grammarFileName, file.ANTLRVersion)>
# encoding: utf-8
from antlr4 import *
@ -638,9 +638,8 @@ ListLabelName(label) ::= "<label>"
CaptureNextToken(d) ::= "<d.varName> = self._input.LT(1)"
CaptureNextTokenType(d) ::= "<d.varName> = self._input.LA(1)"
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers,
superClass={ParserRuleContext}) ::= <<
class <struct.name>(<superClass>):
StructDecl(struct,ctorAttrs,attrs,getters,dispatchMethods,interfaces,extensionMembers) ::= <<
class <struct.name>(<if(contextSuperClass)><contextSuperClass><else>ParserRuleContext<endif>):
def __init__(self, parser, parent:ParserRuleContext=None, invokingState:int=-1<struct.ctorAttrs:{a | , <a.name><if(a.type)>:<a.type><endif>=None}>):
super().__init__(parent, invokingState)

2
tool/src/org/antlr/v4/analysis/AnalysisPipeline.java
View File

@ -100,7 +100,7 @@ public class AnalysisPipeline {
}
}
/** Return whether lookahead sets are disjoint; no lookahead => not disjoint */
/** Return whether lookahead sets are disjoint; no lookahead ⇒ not disjoint */
public static boolean disjoint(IntervalSet[] altLook) {
boolean collision = false;
IntervalSet combined = new IntervalSet();

17
tool/src/org/antlr/v4/analysis/LeftRecursiveRuleAnalyzer.java
View File

@ -68,8 +68,7 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
public LinkedHashMap<Integer, LeftRecursiveRuleAltInfo> binaryAlts = new LinkedHashMap<Integer, LeftRecursiveRuleAltInfo>();
public LinkedHashMap<Integer, LeftRecursiveRuleAltInfo> ternaryAlts = new LinkedHashMap<Integer, LeftRecursiveRuleAltInfo>();
public LinkedHashMap<Integer, LeftRecursiveRuleAltInfo> suffixAlts = new LinkedHashMap<Integer, LeftRecursiveRuleAltInfo>();
public List<LeftRecursiveRuleAltInfo> prefixAlts = new ArrayList<LeftRecursiveRuleAltInfo>();
public List<LeftRecursiveRuleAltInfo> otherAlts = new ArrayList<LeftRecursiveRuleAltInfo>();
public List<LeftRecursiveRuleAltInfo> prefixAndOtherAlts = new ArrayList<LeftRecursiveRuleAltInfo>();
/** Pointer to ID node of ^(= ID element) */
public List<Pair<GrammarAST,String>> leftRecursiveRuleRefLabels =
@ -188,7 +187,7 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
LeftRecursiveRuleAltInfo a =
new LeftRecursiveRuleAltInfo(alt, altText, null, altLabel, false, originalAltTree);
a.nextPrec = nextPrec;
prefixAlts.add(a);
prefixAndOtherAlts.add(a);
//System.out.println("prefixAlt " + alt + ": " + altText + ", rewrite=" + rewriteText);
}
@ -222,7 +221,9 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
String altLabel = altTree.altLabel!=null ? altTree.altLabel.getText() : null;
LeftRecursiveRuleAltInfo a =
new LeftRecursiveRuleAltInfo(alt, altText, null, altLabel, false, originalAltTree);
otherAlts.add(a);
// We keep other alts with prefix alts since they are all added to the start of the generated rule, and
// we want to retain any prior ordering between them
prefixAndOtherAlts.add(a);
// System.out.println("otherAlt " + alt + ": " + altText);
}
@ -254,8 +255,7 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
ruleST.add("opAlts", altST);
}
ruleST.add("primaryAlts", prefixAlts);
ruleST.add("primaryAlts", otherAlts);
ruleST.add("primaryAlts", prefixAndOtherAlts);
tool.log("left-recursion", ruleST.render());
@ -330,7 +330,7 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
return lrlabel;
}
/** Strip last 2 tokens if -> label; alter indexes in altAST */
/** Strip last 2 tokens if → label; alter indexes in altAST */
public void stripAltLabel(GrammarAST altAST) {
int start = altAST.getTokenStartIndex();
int stop = altAST.getTokenStopIndex();
@ -439,8 +439,7 @@ public class LeftRecursiveRuleAnalyzer extends LeftRecursiveRuleWalker {
"binaryAlts=" + binaryAlts +
", ternaryAlts=" + ternaryAlts +
", suffixAlts=" + suffixAlts +
", prefixAlts=" + prefixAlts +
", otherAlts=" + otherAlts +
", prefixAndOtherAlts=" +prefixAndOtherAlts+
'}';
}
}

9
tool/src/org/antlr/v4/analysis/LeftRecursiveRuleTransformer.java
View File

@ -168,8 +168,7 @@ public class LeftRecursiveRuleTransformer {
// track recursive alt info for codegen
r.recPrimaryAlts = new ArrayList<LeftRecursiveRuleAltInfo>();
r.recPrimaryAlts.addAll(leftRecursiveRuleWalker.prefixAlts);
r.recPrimaryAlts.addAll(leftRecursiveRuleWalker.otherAlts);
r.recPrimaryAlts.addAll(leftRecursiveRuleWalker.prefixAndOtherAlts);
if (r.recPrimaryAlts.isEmpty()) {
tool.errMgr.grammarError(ErrorType.NO_NON_LR_ALTS, g.fileName, ((GrammarAST)r.ast.getChild(0)).getToken(), r.name);
}
@ -244,9 +243,9 @@ public class LeftRecursiveRuleTransformer {
* (ALT ID))
* (* (BLOCK
* (OPTIONS ...)
* (ALT {7 >= $_p}? '*' (= b e) {$v = $a.v * $b.v;})
* (ALT {6 >= $_p}? '+' (= b e) {$v = $a.v + $b.v;})
* (ALT {3 >= $_p}? '++') (ALT {2 >= $_p}? '--'))))))
* (ALT {7 &gt;= $_p}? '*' (= b e) {$v = $a.v * $b.v;})
* (ALT {6 &gt;= $_p}? '+' (= b e) {$v = $a.v + $b.v;})
* (ALT {3 &gt;= $_p}? '++') (ALT {2 &gt;= $_p}? '--'))))))
* </pre>
*/
public void setAltASTPointers(LeftRecursiveRule r, RuleAST t) {

26
tool/src/org/antlr/v4/automata/ATNFactory.java
View File

@ -93,7 +93,7 @@ public interface ATNFactory {
/** For a non-lexer, just build a simple token reference atom.
* For a lexer, a string is a sequence of char to match. That is,
* "fog" is treated as 'f' 'o' 'g' not as a single transition in
* the DFA. Machine== o-'f'->o-'o'->o-'g'->o and has n+1 states
* the DFA. Machine== o-'f'-&gt;o-'o'-&gt;o-'g'-&gt;o and has n+1 states
* for n characters.
*/
@ -101,16 +101,16 @@ public interface ATNFactory {
/** For reference to rule r, build
*
* o-e->(r) o
* o-e-&gt;(r) o
*
* where (r) is the start of rule r and the trailing o is not linked
* to from rule ref state directly (it's done thru the transition(0)
* RuleClosureTransition.
*
* If the rule r is just a list of tokens, it's block will be just
* a set on an edge o->o->o-set->o->o->o, could inline it rather than doing
* a set on an edge o-&gt;o-&gt;o-set-&gt;o-&gt;o-&gt;o, could inline it rather than doing
* the rule reference, but i'm not doing this yet as I'm not sure
* it would help much in the ATN->DFA construction.
* it would help much in the ATN-&gt;DFA construction.
*
* TODO add to codegen: collapse alt blks that are sets into single matchSet
* @param node
@ -118,7 +118,7 @@ public interface ATNFactory {
Handle ruleRef(GrammarAST node);
/** From an empty alternative build Grip o-e->o */
/** From an empty alternative build Grip o-e-&gt;o */
Handle epsilon(GrammarAST node);
@ -143,13 +143,13 @@ public interface ATNFactory {
/** From A|B|..|Z alternative block build
*
* o->o-A->o->o (last ATNState is blockEndATNState pointed to by all alts)
* o-&gt;o-A-&gt;o-&gt;o (last ATNState is blockEndATNState pointed to by all alts)
* | ^
* o->o-B->o--|
* o-&gt;o-B-&gt;o--|
* | |
* ... |
* | |
* o->o-Z->o--|
* o-&gt;o-Z-&gt;o--|
*
* So every alternative gets begin ATNState connected by epsilon
* and every alt right side points at a block end ATNState. There is a
@ -160,7 +160,7 @@ public interface ATNFactory {
* begin/end.
*
* Special case: if just a list of tokens/chars/sets, then collapse
* to a single edge'd o-set->o graph.
* to a single edge'd o-set-&gt;o graph.
*
* Set alt number (1..n) in the left-Transition ATNState.
*/
@ -171,9 +171,9 @@ public interface ATNFactory {
/** From (A)? build either:
*
* o--A->o
* o--A-&gt;o
* | ^
* o---->|
* o----&gt;|
*
* or, if A is a block, just add an empty alt to the end of the block
*/
@ -184,7 +184,7 @@ public interface ATNFactory {
*
* |---| (Transition 2 from A.right points at alt 1)
* v | (follow of loop is Transition 1)
* o->o-A-o->o
* o-&gt;o-A-o-&gt;o
*
* Meaning that the last ATNState in A points back to A's left Transition ATNState
* and we add a new begin/end ATNState. A can be single alternative or
@ -200,7 +200,7 @@ public interface ATNFactory {
*
* |---|
* v |
* o->o-A-o--o (Transition 2 from block end points at alt 1; follow is Transition 1)
* o-&gt;o-A-o--o (Transition 2 from block end points at alt 1; follow is Transition 1)
* | ^
* o---------| (optional branch is 2nd alt of optional block containing A+)
*

2
tool/src/org/antlr/v4/automata/LexerATNFactory.java
View File

@ -324,7 +324,7 @@ public class LexerATNFactory extends ParserATNFactory {
/** For a lexer, a string is a sequence of char to match. That is,
* "fog" is treated as 'f' 'o' 'g' not as a single transition in
* the DFA. Machine== o-'f'->o-'o'->o-'g'->o and has n+1 states
* the DFA. Machine== o-'f'-&gt;o-'o'-&gt;o-'g'-&gt;o and has n+1 states
* for n characters.
*/
@Override

24
tool/src/org/antlr/v4/automata/ParserATNFactory.java
View File

@ -286,7 +286,7 @@ public class ParserATNFactory implements ATNFactory {
* For reference to rule {@code r}, build
*
* <pre>
* o->(r) o
* o-&gt;(r) o
* </pre>
*
* where {@code (r)} is the start of rule {@code r} and the trailing
@ -391,24 +391,24 @@ public class ParserATNFactory implements ATNFactory {
* From {@code A|B|..|Z} alternative block build
*
* <pre>
* o->o-A->o->o (last ATNState is BlockEndState pointed to by all alts)
* o-&gt;o-A-&gt;o-&gt;o (last ATNState is BlockEndState pointed to by all alts)
* | ^
* |->o-B->o--|
* |-&gt;o-B-&gt;o--|
* | |
* ... |
* | |
* |->o-Z->o--|
* |-&gt;o-Z-&gt;o--|
* </pre>
*
* So start node points at every alternative with epsilon transition and
* every alt right side points at a block end ATNState.
* <p/>
* <p>
* Special case: only one alternative: don't make a block with alt
* begin/end.
* <p/>
* <p>
* Special case: if just a list of tokens/chars/sets, then collapse to a
* single edged o-set->o graph.
* <p/>
* single edged o-set-&gt;o graph.
* <p>
* TODO: Set alt number (1..n) in the states?
*/
@ -506,9 +506,9 @@ public class ParserATNFactory implements ATNFactory {
* From {@code (A)?} build either:
*
* <pre>
* o--A->o
* o--A-&gt;o
* | ^
* o---->|
* o----&gt;|
* </pre>
*
* or, if {@code A} is a block, just add an empty alt to the end of the
@ -535,7 +535,7 @@ public class ParserATNFactory implements ATNFactory {
* <pre>
* |---------|
* v |
* [o-blk-o]->o->o
* [o-blk-o]-&gt;o-&gt;o
* </pre>
*
* We add a decision for loop back node to the existing one at {@code blk}
@ -583,7 +583,7 @@ public class ParserATNFactory implements ATNFactory {
* <pre>
* |-------------|
* v |
* o--[o-blk-o]->o o
* o--[o-blk-o]-&gt;o o
* | ^
* -----------------|
* </pre>

2
tool/src/org/antlr/v4/codegen/ActionTranslator.java
View File

@ -138,7 +138,7 @@ public class ActionTranslator implements ActionSplitterListener {
ActionAST node)
{
String action = tokenWithinAction.getText();
if ( action.charAt(0)=='{' ) {
if ( action!=null && action.length()>0 && action.charAt(0)=='{' ) {
int firstCurly = action.indexOf('{');
int lastCurly = action.lastIndexOf('}');
if ( firstCurly>=0 && lastCurly>=0 ) {

20
tool/src/org/antlr/v4/codegen/DefaultOutputModelFactory.java
View File

@ -30,6 +30,7 @@
package org.antlr.v4.codegen;
import org.antlr.v4.codegen.model.Action;
import org.antlr.v4.codegen.model.CodeBlockForOuterMostAlt;
import org.antlr.v4.codegen.model.OutputModelObject;
import org.antlr.v4.codegen.model.RuleFunction;
@ -38,6 +39,9 @@ import org.antlr.v4.codegen.model.decl.CodeBlock;
import org.antlr.v4.codegen.model.decl.Decl;
import org.antlr.v4.tool.Alternative;
import org.antlr.v4.tool.Grammar;
import org.antlr.v4.tool.Rule;
import org.stringtemplate.v4.ST;
import org.stringtemplate.v4.STGroup;
import java.util.ArrayList;
import java.util.Arrays;
@ -73,6 +77,22 @@ public abstract class DefaultOutputModelFactory extends BlankOutputModelFactory
return controller;
}
@Override
public List<SrcOp> rulePostamble(RuleFunction function, Rule r) {
if ( r.namedActions.containsKey("after") || r.namedActions.containsKey("finally") ) {
// See OutputModelController.buildLeftRecursiveRuleFunction
// and Parser.exitRule for other places which set stop.
CodeGenerator gen = getGenerator();
STGroup codegenTemplates = gen.getTemplates();
ST setStopTokenAST = codegenTemplates.getInstanceOf("recRuleSetStopToken");
Action setStopTokenAction = new Action(this, function.ruleCtx, setStopTokenAST);
List<SrcOp> ops = new ArrayList<SrcOp>(1);
ops.add(setStopTokenAction);
return ops;
}
return super.rulePostamble(function, r);
}
// Convenience methods

2
tool/src/org/antlr/v4/codegen/OutputModelWalker.java
View File

@ -53,7 +53,7 @@ import java.util.Set;
* We identify those nested objects by the list of arguments in the template
* definition. For example, here is the definition of the parser template:
*
* Parser(parser, scopes, funcs) ::= <<...>>
* Parser(parser, scopes, funcs) ::= &lt;&lt;...&gt;&gt;
*
* The first template argument is always the output model object from which
* this walker will create the template. Any other arguments identify

2
tool/src/org/antlr/v4/codegen/Target.java
View File

@ -57,7 +57,7 @@ public abstract class Target {
* predicates and such that may refer to chars that need to be escaped
* when represented as strings. Also, templates need to be escaped so
* that the target language can hold them as a string.
* <p/>
* <p>
* I have defined (via the constructor) the set of typical escapes,
* but your {@link Target} subclass is free to alter the translated chars
* or add more definitions. This is non-static so each target can have

7
tool/src/org/antlr/v4/codegen/model/ParserFile.java
View File

@ -31,6 +31,8 @@
package org.antlr.v4.codegen.model;
import org.antlr.v4.codegen.OutputModelFactory;
import org.antlr.v4.codegen.model.chunk.ActionChunk;
import org.antlr.v4.codegen.model.chunk.ActionText;
import org.antlr.v4.tool.Grammar;
import org.antlr.v4.tool.ast.ActionAST;
@ -42,6 +44,7 @@ public class ParserFile extends OutputFile {
public String genPackage; // from -package cmd-line
@ModelElement public Parser parser;
@ModelElement public Map<String, Action> namedActions;
@ModelElement public ActionChunk contextSuperClass;
public Boolean genListener = false;
public Boolean genVisitor = false;
public String grammarName;
@ -59,5 +62,9 @@ public class ParserFile extends OutputFile {
genListener = g.tool.gen_listener;
genVisitor = g.tool.gen_visitor;
grammarName = g.name;
if (g.getOptionString("contextSuperClass") != null) {
contextSuperClass = new ActionText(null, g.getOptionString("contextSuperClass"));
}
}
}

2
tool/src/org/antlr/v4/codegen/model/decl/AltLabelStructDecl.java
View File

@ -38,7 +38,7 @@ import org.antlr.v4.tool.Rule;
import java.util.ArrayList;
/** A StructDecl to handle a -> label on alt */
/** A StructDecl to handle a -&gt; label on alt */
public class AltLabelStructDecl extends StructDecl {
public int altNum;
public AltLabelStructDecl(OutputModelFactory factory, Rule r,

2
tool/src/org/antlr/v4/codegen/target/CSharpTarget.java
View File

@ -51,7 +51,7 @@ public class CSharpTarget extends Target {
@Override
public String getVersion() {
return "4.5.2"; // crossing fingers that it's close enough.
return "4.5.3";
}
@Override

6
tool/src/org/antlr/v4/codegen/target/JavaScriptTarget.java
View File

@ -74,7 +74,7 @@ public class JavaScriptTarget extends Target {
@Override
public String getVersion() {
return "4.5.2";
return "4.5.3";
}
public Set<String> getBadWords() {
@ -93,11 +93,11 @@ public class JavaScriptTarget extends Target {
/**
* {@inheritDoc}
* <p/>
* <p>
* For Java, this is the translation {@code 'a\n"'} &rarr; {@code "a\n\""}.
* Expect single quotes around the incoming literal. Just flip the quotes
* and replace double quotes with {@code \"}.
* <p/>
* <p>
* Note that we have decided to allow people to use '\"' without penalty, so
* we must build the target string in a loop as {@link String#replace}
* cannot handle both {@code \"} and {@code "} without a lot of messing

6
tool/src/org/antlr/v4/codegen/target/Python2Target.java
View File

@ -50,7 +50,7 @@ public class Python2Target extends Target {
"abs", "all", "any", "apply", "as",
"bin", "bool", "buffer", "bytearray",
"callable", "chr", "classmethod", "coerce", "compile", "complex",
"delattr", "dict", "dir", "divmod",
"del", "delattr", "dict", "dir", "divmod",
"enumerate", "eval", "execfile",
"file", "filter", "float", "format", "frozenset",
"getattr", "globals",
@ -61,7 +61,7 @@ public class Python2Target extends Target {
"memoryview",
"object", "oct", "open", "ord",
"pow", "print", "property",
"range", "raw_input", "reduce", "reload", "repr", "reversed", "round",
"range", "raw_input", "reduce", "reload", "repr", "return", "reversed", "round",
"set", "setattr", "slice", "sorted", "staticmethod", "str", "sum", "super",
"tuple", "type",
"unichr", "unicode",
@ -117,7 +117,7 @@ public class Python2Target extends Target {
@Override
public String getVersion() {
return "4.5.2.1";
return "4.5.3";
}
public Set<String> getBadWords() {

6
tool/src/org/antlr/v4/codegen/target/Python3Target.java
View File

@ -50,7 +50,7 @@ public class Python3Target extends Target {
"abs", "all", "any", "apply", "as",
"bin", "bool", "buffer", "bytearray",
"callable", "chr", "classmethod", "coerce", "compile", "complex",
"delattr", "dict", "dir", "divmod",
"del", "delattr", "dict", "dir", "divmod",
"enumerate", "eval", "execfile",
"file", "filter", "float", "format", "frozenset",
"getattr", "globals",
@ -61,7 +61,7 @@ public class Python3Target extends Target {
"memoryview",
"object", "oct", "open", "ord",
"pow", "print", "property",
"range", "raw_input", "reduce", "reload", "repr", "reversed", "round",
"range", "raw_input", "reduce", "reload", "repr", "return", "reversed", "round",
"set", "setattr", "slice", "sorted", "staticmethod", "str", "sum", "super",
"tuple", "type",
"unichr", "unicode",
@ -119,7 +119,7 @@ public class Python3Target extends Target {
@Override
public String getVersion() {
return "4.5.2.1";
return "4.5.3";
}
/** Avoid grammar symbols in this set to prevent conflicts in gen'd code. */

2
tool/src/org/antlr/v4/misc/Graph.java
View File

@ -81,7 +81,7 @@ public class Graph<T> {
* For sorting, I'm not following convention here since ANTLR
* needs the opposite. Here's what I assume for sorting:
*
* If there exists an edge u -> v then u depends on v and v
* If there exists an edge u -&gt; v then u depends on v and v
* must happen before u.
*
* So if this gives nonreversed postorder traversal, I get the order

15
tool/src/org/antlr/v4/misc/Utils.java
View File

@ -54,21 +54,6 @@ public class Utils {
static Integer[] ints = new Integer[INTEGER_POOL_MAX_VALUE+1];
/** Integer objects are immutable so share all Integers with the
* same value up to some max size. Use an array as a perfect hash.
* Return shared object for 0..INTEGER_POOL_MAX_VALUE or a new
* Integer object with x in it. Java's autoboxing only caches up to 127.
public static Integer integer(int x) {
if ( x<0 || x>INTEGER_POOL_MAX_VALUE ) {
return new Integer(x);
}
if ( ints[x]==null ) {
ints[x] = new Integer(x);
}
return ints[x];
}
*/
public static String stripFileExtension(String name) {
if ( name==null ) return null;
int lastDot = name.lastIndexOf('.');

2
tool/src/org/antlr/v4/parse/GrammarToken.java
View File

@ -35,7 +35,7 @@ import org.antlr.runtime.Token;
import org.antlr.v4.tool.Grammar;
/** A CommonToken that can also track it's original location,
* derived from options on the element ref like BEGIN<line=34,...>.
* derived from options on the element ref like BEGIN&lt;line=34,...&gt;.
*/
public class GrammarToken extends CommonToken {
public Grammar g;

6
tool/src/org/antlr/v4/parse/ScopeParser.java
View File

@ -52,11 +52,15 @@ import java.util.List;
public class ScopeParser {
/** Given an arg or retval scope definition list like
*
* Map<String, String>, int[] j3, char *foo32[3]
* <code>
* Map&lt;String, String&gt;, int[] j3, char *foo32[3]
* </code>
*
* or
*
* <code>
* int i=3, j=a[34]+20
* </code>
*
* convert to an attribute scope.
*/

Some files were not shown because too many files have changed in this diff Show More