forgetest1
/
antlr
forked from jasder/antlr
1
0
Fork 0
Code Issues Pull Requests 1 Releases Wiki Activity

Merge remote-tracking branch 'origin/master' into iterative-tree-walker

This commit is contained in:
Tom Wieczorek 2016-11-25 12:25:27 +01:00
parent cc59a699d4 df40db35a4
commit d1bc0f5ca5
No known key found for this signature in database
GPG Key ID: 97132D3CD100563E
1533 changed files with 244407 additions and 69764 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
.editorconfig Normal file
View File

@ -0,0 +1,7 @@
root = true
[*.{java,stg}]
charset = utf-8
insert_final_newline = true
trim_trailing_whitespace = true
indent_style = tab

2
.gitattributes vendored Normal file
View File

@ -0,0 +1,2 @@
# This rule applies to all files which don't match another line below
* text=auto

10
.github/ISSUE_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,10 @@
Before submitting an issue to ANTLR, please check off these boxes:
- [ ] I am not submitting a question on how to use ANTLR; instead, go to [antlr4-discussion google group](https://groups.google.com/forum/#!forum/antlr-discussion) or ask at [stackoverflow](http://stackoverflow.com/questions/tagged/antlr4)
- [ ] I have done a search of the existing issues to make sure I'm not sending in a duplicate
### Expected behavior
### Actual behavior
### Steps to reproduce the behavior

3
.github/PULL_REQUEST_TEMPLATE.md vendored Normal file
View File

@ -0,0 +1,3 @@
Thank you for proposing a contribution to the ANTLR project. In order to accept changes from the outside world, all contributors most "sign" the [contributors.txt](https://github.com/antlr/antlr4/blob/master/contributors.txt) contributors certificate of origin. It's an unfortunate reality of today's fuzzy and bizarre world of open-source ownership.
Make sure you are already in the contributors.txt file or add a commit to this pull request with the appropriate change. Thanks!

47
.gitignore vendored
View File

@ -1,5 +1,11 @@
# Maven build folders
target/
# ... but not code generation targets
!tool/src/org/antlr/v4/codegen/target/
# Node.js (npm and typings) cached dependencies
node_modules/
typings/
# Ant build folders
build/
@ -10,14 +16,37 @@ user.build.properties
# MacOSX files
.DS_Store
# Python
*.pyc
## Python, selected lines from https://raw.githubusercontent.com/github/gitignore/master/Python.gitignore
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
# CSharp
bin/
obj/
## CSharp and VisualStudio, selected lines from https://raw.githubusercontent.com/github/gitignore/master/VisualStudio.gitignore
# User-specific files
*.suo
*.user
*.userosscache
*.sln.docstates
# User-specific files (MonoDevelop/Xamarin Studio)
*.userprefs
# Build results
[Dd]ebug/
[Dd]ebugPublic/
[Rr]elease/
[Rr]eleases/
x64/
x86/
bld/
[Bb]in/
[Oo]bj/
[Ll]og/
# Visual Studio 2015 cache/options directory
.vs/
# NetBeans user configuration files
nbactions*.xml
/nbproject/private/
@ -57,3 +86,11 @@ bilder.pyc
bild.log
bild_output.txt
runtime/Cpp/demo/generated
xcuserdata
*.jar
.vscode
# VSCode Java plugin temporary files
javac-services.0.log
javac-services.0.log.lck

44
.travis.yml
View File

@ -1,19 +1,31 @@
sudo: true
language: java
matrix:
include:
- os: linux
env: CXX=g++-5
compiler: clang
language: java
jdk: oraclejdk7
- os: osx
compiler: clang
language: java
osx_image: xcode8.1
script:
- mvn install
jdk:
- openjdk6
- oraclejdk7
- oraclejdk8
- if [[ $TRAVIS_OS_NAME == osx ]]; then cd runtime-testsuite; ../.travis/run-tests-macos.sh; fi
- if [[ $TRAVIS_OS_NAME == linux ]]; then cd runtime-testsuite; ../.travis/run-tests-linux.sh; fi
before_install:
- sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
- sudo add-apt-repository ppa:fkrull/deadsnakes -y
- sudo add-apt-repository ppa:rwky/nodejs -y
- sudo apt-get update -qq
- sudo apt-get install -qq python3.5
- sudo apt-get install -qq nodejs
- echo "deb http://download.mono-project.com/repo/debian wheezy/snapshots/3.12.1 main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
- sudo apt-get install -qq mono-complete
- python --version
- python3 --version
- if [[ $TRAVIS_OS_NAME == osx ]]; then ./.travis/before-install-macos.sh; fi
- if [[ $TRAVIS_OS_NAME == linux ]]; then ./.travis/before-install-linux.sh; fi
addons:
apt:
sources:
- ubuntu-toolchain-r-test
- llvm-toolchain-precise-3.7
packages:
- g++-5
- uuid-dev
- clang-3.7

17
.travis/before-install-linux.sh Executable file
View File

@ -0,0 +1,17 @@
#!/bin/bash
set -euo pipefail
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
sudo add-apt-repository ppa:fkrull/deadsnakes -y
sudo add-apt-repository ppa:rwky/nodejs -y
sudo apt-get update -qq
sudo apt-get install -qq python3.5
sudo apt-get install -qq nodejs
echo "deb http://download.mono-project.com/repo/debian wheezy/snapshots/3.12.1 main" | sudo tee /etc/apt/sources.list.d/mono-xamarin.list
sudo apt-get install -qq mono-complete
eval "$(sudo gimme 1.7.3)"
( go version ; go env ) || true
python --version
python3 --version

17
.travis/before-install-macos.sh Executable file
View File

@ -0,0 +1,17 @@
#!/bin/bash
set -euo pipefail
thisdir=$(dirname "$0")
brew update
brew install mono python3 cmake
# Work around apparent rvm bug that is in Travis's Xcode image.
# https://github.com/direnv/direnv/issues/210
# https://github.com/travis-ci/travis-ci/issues/6307
shell_session_update() { :; }
( go version ; go env ) || true
python --version
python3 --version

10
.travis/run-tests-linux.sh Executable file
View File

@ -0,0 +1,10 @@
#!/bin/bash
mvn -Dparallel=methods -DthreadCount=4 -Dtest=java.* test
mvn -Dparallel=methods -DthreadCount=4 -Dtest=csharp.* test
mvn -Dparallel=methods -DthreadCount=4 -Dtest=python2.* test
mvn -Dparallel=methods -DthreadCount=4 -Dtest=python3.* test
mvn -Dparallel=methods -DthreadCount=4 -Dtest=node.* test
mvn -Dparallel=methods -DthreadCount=4 -Dtest=go.* test
mvn -Dtest=cpp.* test # timeout due to no output for 10 min on travis if in parallel
#mvn -Dparallel=methods -DthreadCount=4 -Dtest=swift.* test

4
.travis/run-tests-macos.sh Executable file
View File

@ -0,0 +1,4 @@
#!/bin/bash
# only test swift as we develop on os x so likely well tested and its dog slow on travis
mvn -Dtest=swift.* test

28
README.md
View File

@ -1,5 +1,7 @@
# ANTLR v4
[![Build Status](https://travis-ci.org/antlr/antlr4.png?branch=master)](https://travis-ci.org/antlr/antlr4) [![Java 7+](https://img.shields.io/badge/java-7+-4c7e9f.svg)](http://java.oracle.com) [![License](https://img.shields.io/badge/license-BSD-blue.svg)](https://raw.githubusercontent.com/antlr/antlr4/master/LICENSE.txt)
**ANTLR** (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files. It's widely used to build languages, tools, and frameworks. From a grammar, ANTLR generates a parser that can build parse trees and also generates a listener interface (or visitor) that makes it easy to respond to the recognition of phrases of interest.
*Given day-job constraints, my time working on this project is limited so I'll have to focus first on fixing bugs rather than changing/improving the feature set. Likely I'll do it in bursts every few months. Please do not be offended if your bug or pull request does not yield a response! --parrt*
@ -11,22 +13,29 @@ ANTLR project lead and supreme dictator for life
[University of San Francisco](http://www.usfca.edu/)
* [Sam Harwell](http://tunnelvisionlabs.com/) (Tool co-author, Java and C# target)
* Eric Vergnaud (Javascript, Python2, Python3 targets and significant work on C# target)
* [Peter Boyer](https://github.com/pboyer) (Go target)
* [Mike Lischke](http://www.soft-gems.net/) (C++ completed target)
* Dan McLaughlin (C++ initial target)
* David Sisson (C++ initial target and test)
* [Janyou](https://github.com/janyou) (Swift target)
* [Ewan Mellor](https://github.com/ewanmellor), [Hanzhou Shi](https://github.com/hanjoes) (Swift target merging)
## Useful information
* [Release notes](https://github.com/antlr/antlr4/releases)
* [Getting started with v4](https://raw.githubusercontent.com/antlr/antlr4/master/doc/getting-started.md)
* [Getting started with v4](https://github.com/antlr/antlr4/blob/master/doc/getting-started.md)
* [Official site](http://www.antlr.org/)
* [Documentation](https://raw.githubusercontent.com/antlr/antlr4/master/doc/index.md)
* [FAQ](https://raw.githubusercontent.com/antlr/antlr4/master/doc/faq/index.md)
* [API](http://www.antlr.org/api/Java/index.html)
* [Documentation](https://github.com/antlr/antlr4/blob/master/doc/index.md)
* [FAQ](https://github.com/antlr/antlr4/blob/master/doc/faq/index.md)
* [ANTLR code generation targets](https://github.com/antlr/antlr4/blob/master/doc/targets.md)<br>(Currently: Java, C#, Python2|3, JavaScript, Go, C++, Swift)
* [Java API](http://www.antlr.org/api/Java/index.html)
* [ANTLR v3](http://www.antlr3.org/)
* [v3 to v4 Migration, differences](https://raw.githubusercontent.com/antlr/antlr4/master/doc/faq/general.md)
* [v3 to v4 Migration, differences](https://github.com/antlr/antlr4/blob/master/doc/faq/general.md)
You might also find the following pages useful, particularly if you want to mess around with the various target languages.
* [How to build ANTLR itself](https://raw.githubusercontent.com/antlr/antlr4/master/doc/building-antlr.md)
* [How we create and deploy an ANTLR release](https://raw.githubusercontent.com/antlr/antlr4/master/doc/releasing-antlr.md)
* [How to build ANTLR itself](https://github.com/antlr/antlr4/blob/master/doc/building-antlr.md)
* [How we create and deploy an ANTLR release](https://github.com/antlr/antlr4/blob/master/doc/releasing-antlr.md)
## The Definitive ANTLR 4 Reference
@ -40,8 +49,3 @@ You will find the [Book source code](http://pragprog.com/titles/tpantlr2/source_
[This repository](https://github.com/antlr/grammars-v4) is a collection of grammars without actions where the
root directory name is the all-lowercase name of the language parsed
by the grammar. For example, java, cpp, csharp, c, etc...
Travis Status
---------
<a href="https://travis-ci.org/antlr/antlr4"><img src="https://api.travis-ci.org/antlr/antlr4.png"></a>

78
antlr4-maven-plugin/pom.xml
View File

@ -34,7 +34,7 @@
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.4-SNAPSHOT</version>
<version>4.6-SNAPSHOT</version>
</parent>
<artifactId>antlr4-maven-plugin</artifactId>
<packaging>maven-plugin</packaging>
@ -48,11 +48,16 @@
<!-- Ancilliary information for completeness -->
<inceptionYear>2009</inceptionYear>
<properties>
<mavenVersion>3.3.9</mavenVersion>
<takariLifecycleVersion>1.11.12</takariLifecycleVersion>
</properties>
<!-- ============================================================================= -->
<!-- What are we depedent on for the Mojos to execute? We need the plugin
API itself and of course we need the ANTLR Tool and runtime and any of their
dependencies, which we inherit. The Tool itself provides us with all the
<!-- What are we depedent on for the Mojos to execute? We need the plugin
API itself and of course we need the ANTLR Tool and runtime and any of their
dependencies, which we inherit. The Tool itself provides us with all the
dependencies, so we need only name it here. -->
<dependencies>
@ -63,11 +68,6 @@
<version>3.0.5</version>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-compiler-api</artifactId>
@ -78,8 +78,8 @@
<artifactId>plexus-build-api</artifactId>
<version>0.0.7</version>
</dependency>
<!-- The version of ANTLR tool that this version of the plugin controls.
We have decided that this should be in lockstep with ANTLR itself, other
<!-- The version of ANTLR tool that this version of the plugin controls.
We have decided that this should be in lockstep with ANTLR itself, other
than -1 -2 -3 etc patch releases. -->
<dependency>
<groupId>org.antlr</groupId>
@ -93,21 +93,50 @@
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-plugin-testing-harness</artifactId>
<version>1.1</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven.plugin-tools</groupId>
<artifactId>maven-plugin-annotations</artifactId>
<version>3.2</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.takari.maven.plugins</groupId>
<artifactId>takari-plugin-testing</artifactId>
<version>2.9.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-core</artifactId>
<version>${mavenVersion}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-compat</artifactId>
<version>${mavenVersion}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.plexus</groupId>
<artifactId>plexus-utils</artifactId>
<version>3.0.15</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.2.1</version>
</dependency>
</dependencies>
<build>
<testSourceDirectory>src/test</testSourceDirectory>
<testResources>
<testResource>
<directory>src/test/resources</directory>
</testResource>
</testResources>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@ -132,6 +161,21 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>io.takari.maven.plugins</groupId>
<artifactId>takari-lifecycle-plugin</artifactId>
<version>${takariLifecycleVersion}</version>
<extensions>true</extensions>
<executions>
<execution>
<id>testProperties</id>
<phase>process-test-resources</phase>
<goals>
<goal>testProperties</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>

151
antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/Antlr4Mojo.java
View File

@ -59,6 +59,7 @@ import java.io.OutputStreamWriter;
import java.io.StringWriter;
import java.io.Writer;
import java.net.URI;
import java.nio.charset.Charset;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collections;
@ -95,7 +96,13 @@ public class Antlr4Mojo extends AbstractMojo {
* specify grammar file encoding; e.g., euc-jp
*/
@Parameter(property = "project.build.sourceEncoding")
protected String encoding;
protected String inputEncoding;
/**
* specify output file encoding; defaults to source encoding
*/
@Parameter(property = "project.build.sourceEncoding")
protected String outputEncoding;
/**
* Generate parse tree listener interface and base class.
@ -184,6 +191,12 @@ public class Antlr4Mojo extends AbstractMojo {
@Parameter(defaultValue = "${basedir}/src/main/antlr4/imports")
private File libDirectory;
/**
* The directory where build status information is located.
*/
@Parameter(defaultValue = "${project.build.directory}/maven-status/antlr4", readonly=true)
private File statusDirectory;
@Component
private BuildContext buildContext;
@ -222,6 +235,8 @@ public class Antlr4Mojo extends AbstractMojo {
Log log = getLog();
outputEncoding = validateEncoding(outputEncoding);
if (log.isDebugEnabled()) {
for (String e : excludes) {
log.debug("ANTLR: Exclude: " + e);
@ -249,16 +264,22 @@ public class Antlr4Mojo extends AbstractMojo {
outputDir.mkdirs();
}
GrammarDependencies dependencies = new GrammarDependencies(sourceDirectory, libDirectory, arguments, getDependenciesStatusFile(), getLog());
// Now pick up all the files and process them with the Tool
//
List<List<String>> argumentSets;
Set<File> grammarFiles;
Set<File> importGrammarFiles;
try {
List<String> args = getCommandArguments();
argumentSets = processGrammarFiles(args, sourceDirectory);
} catch (InclusionScanException ie) {
log.error(ie);
throw new MojoExecutionException("Fatal error occured while evaluating the names of the grammar files to analyze", ie);
grammarFiles = getGrammarFiles(sourceDirectory);
importGrammarFiles = getImportFiles(sourceDirectory);
argumentSets = processGrammarFiles(args, grammarFiles, dependencies, sourceDirectory);
} catch (Exception e) {
log.error(e);
throw new MojoExecutionException("Fatal error occured while evaluating the names of the grammar files to analyze", e);
}
log.debug("Output directory base will be " + outputDirectory.getAbsolutePath());
@ -272,6 +293,14 @@ public class Antlr4Mojo extends AbstractMojo {
throw new MojoFailureException("Error creating an instanceof the ANTLR tool.", e);
}
try {
dependencies.analyze(grammarFiles, importGrammarFiles, tool);
} catch (Exception e) {
log.error("Dependency analysis failed, see exception report for details",
e);
throw new MojoFailureException("Dependency analysis failed.", e);
}
// Set working directory for ANTLR to be the base source directory
tool.inputDirectory = sourceDirectory;
@ -288,6 +317,12 @@ public class Antlr4Mojo extends AbstractMojo {
// Tell Maven that there are some new source files underneath the output directory.
addSourceRoot(this.getOutputDirectory());
}
try {
dependencies.save();
} catch (IOException ex) {
log.warn("Could not save grammar dependency status", ex);
}
}
private List<String> getCommandArguments() {
@ -310,9 +345,10 @@ public class Antlr4Mojo extends AbstractMojo {
args.add("-atn");
}
if (encoding != null && !encoding.isEmpty()) {
if ( inputEncoding!=null && !inputEncoding.isEmpty()) {
args.add("-encoding");
args.add(encoding);
outputEncoding = inputEncoding;
args.add(inputEncoding);
}
if (listener) {
@ -355,22 +391,11 @@ public class Antlr4Mojo extends AbstractMojo {
* @param sourceDirectory
* @exception InclusionScanException
*/
private List<List<String>> processGrammarFiles(List<String> args, File sourceDirectory) throws InclusionScanException {
// Which files under the source set should we be looking for as grammar files
SourceMapping mapping = new SuffixMapping("g4", Collections.<String>emptySet());
// What are the sets of includes (defaulted or otherwise).
Set<String> includes = getIncludesPatterns();
// Now, to the excludes, we need to add the imports directory
// as this is autoscanned for imported grammars and so is auto-excluded from the
// set of grammar fields we should be analyzing.
excludes.add("imports/**");
SourceInclusionScanner scan = new SimpleSourceInclusionScanner(includes, excludes);
scan.addSourceMapping(mapping);
Set<File> grammarFiles = scan.getIncludedSources(sourceDirectory, null);
private List<List<String>> processGrammarFiles(
List<String> args,
Set<File> grammarFiles,
GrammarDependencies dependencies,
File sourceDirectory) throws InclusionScanException, IOException {
// We don't want the plugin to run for every grammar, regardless of whether
// it's changed since the last compilation. Check the mtime of the tokens vs
@ -381,7 +406,8 @@ public class Antlr4Mojo extends AbstractMojo {
String tokensFileName = grammarFile.getName().split("\\.")[0] + ".tokens";
File outputFile = new File(outputDirectory, tokensFileName);
if ( (! outputFile.exists()) ||
outputFile.lastModified() < grammarFile.lastModified() ) {
outputFile.lastModified() < grammarFile.lastModified() ||
dependencies.isDependencyChanged(grammarFile)) {
grammarFilesToProcess.add(grammarFile);
}
}
@ -405,7 +431,7 @@ public class Antlr4Mojo extends AbstractMojo {
getLog().debug("Grammar file '" + grammarFile.getPath() + "' detected.");
String relPathBase = findSourceSubdir(sourceDirectory, grammarFile.getPath());
String relPathBase = MojoUtils.findSourceSubdir(sourceDirectory, grammarFile);
String relPath = relPathBase + grammarFile.getName();
getLog().debug(" ... relative path is: " + relPath);
@ -430,6 +456,39 @@ public class Antlr4Mojo extends AbstractMojo {
return result;
}
private Set<File> getImportFiles(File sourceDirectory) throws InclusionScanException {
if (!libDirectory.exists()) return Collections.emptySet();
Set<String> includes = new HashSet<String>();
includes.add("*.g4");
includes.add("*.tokens");
SourceInclusionScanner scan = new SimpleSourceInclusionScanner(includes,
Collections.<String>emptySet());
scan.addSourceMapping(new SuffixMapping("G4", "g4"));
return scan.getIncludedSources(libDirectory, null);
}
private Set<File> getGrammarFiles(File sourceDirectory) throws InclusionScanException
{
// Which files under the source set should we be looking for as grammar files
SourceMapping mapping = new SuffixMapping("g4", Collections.<String>emptySet());
// What are the sets of includes (defaulted or otherwise).
Set<String> includes = getIncludesPatterns();
// Now, to the excludes, we need to add the imports directory
// as this is autoscanned for imported grammars and so is auto-excluded from the
// set of grammar fields we should be analyzing.
excludes.add("imports/**");
SourceInclusionScanner scan = new SimpleSourceInclusionScanner(includes, excludes);
scan.addSourceMapping(mapping);
return scan.getIncludedSources(sourceDirectory, null);
}
private static String getPackageName(String relativeFolderPath) {
if (relativeFolderPath.contains("..")) {
throw new UnsupportedOperationException("Cannot handle relative paths containing '..'");
@ -450,30 +509,14 @@ public class Antlr4Mojo extends AbstractMojo {
return includes;
}
/**
* Given the source directory File object and the full PATH to a grammar,
* produce the path to the named grammar file in relative terms to the
* {@code sourceDirectory}. This will then allow ANTLR to produce output
* relative to the base of the output directory and reflect the input
* organization of the grammar files.
*
* @param sourceDirectory The source directory {@link File} object
* @param grammarFileName The full path to the input grammar file
* @return The path to the grammar file relative to the source directory
*/
private String findSourceSubdir(File sourceDirectory, String grammarFileName) {
String srcPath = sourceDirectory.getPath() + File.separator;
private File getDependenciesStatusFile() {
File statusFile = new File(statusDirectory, "dependencies.ser");
if (!grammarFileName.startsWith(srcPath)) {
throw new IllegalArgumentException("expected " + grammarFileName + " to be prefixed with " + sourceDirectory);
if (!statusFile.getParentFile().exists()) {
statusFile.getParentFile().mkdirs();
}
File unprefixedGrammarFileName = new File(grammarFileName.substring(srcPath.length()));
if (unprefixedGrammarFileName.getParent() == null) {
return "";
}
return unprefixedGrammarFileName.getParent() + File.separator;
return statusFile;
}
private final class CustomTool extends Tool {
@ -515,7 +558,21 @@ public class Antlr4Mojo extends AbstractMojo {
URI relativePath = project.getBasedir().toURI().relativize(outputFile.toURI());
getLog().debug(" Writing file: " + relativePath);
OutputStream outputStream = buildContext.newFileOutputStream(outputFile);
return new BufferedWriter(new OutputStreamWriter(outputStream));
if ( outputEncoding!=null && !outputEncoding.isEmpty()) {
return new BufferedWriter(new OutputStreamWriter(outputStream, outputEncoding));
}
else {
return new BufferedWriter(new OutputStreamWriter(outputStream));
}
}
}
/**
* Validates the given encoding.
*
* @return the validated encoding. If {@code null} was provided, returns the platform default encoding.
*/
private String validateEncoding(String encoding) {
return (encoding == null) ? Charset.defaultCharset().name() : Charset.forName(encoding.trim()).name();
}
}

311
antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/GrammarDependencies.java Normal file
View File

@ -0,0 +1,311 @@
package org.antlr.mojo.antlr4;
import org.antlr.runtime.tree.Tree;
import org.antlr.v4.Tool;
import org.antlr.v4.misc.Graph;
import org.antlr.v4.parse.ANTLRParser;
import org.antlr.v4.tool.ast.GrammarAST;
import org.antlr.v4.tool.ast.GrammarRootAST;
import org.apache.maven.plugin.logging.Log;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.AbstractMap;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.Collection;
import java.util.HashMap;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Set;
class GrammarDependencies {
private final Graph<String> graph = new Graph<String>();
private final File sourceDirectory;
private final File libDirectory;
private final File statusFile;
private final String packageName;
/** Map grammars to their checksum and references. */
private final Map<File, Map.Entry<byte[], Collection<String>>> grammars;
private final Log log;
public GrammarDependencies(File sourceDirectory, File libDirectory,
List<String> arguments, File status, Log log) {
this.log = log;
this.sourceDirectory = sourceDirectory;
this.libDirectory = libDirectory;
this.statusFile = status;
this.grammars = loadStatus(status);
this.packageName = getPackage(arguments);
}
/**
* Determines the package to use.
*
* @param arguments the tool arguments.
*
* @return the package. Returns {@code null} to indicate that no package should be
* used.
*/
private String getPackage(List<String> arguments) {
int index = (arguments != null) ? arguments.indexOf("-package") : -1;
return (index > -1)
? (arguments.get(index + 1).replace('.', File.separatorChar) +
File.separatorChar)
: null;
}
public void save() throws IOException {
if (!grammars.isEmpty()) {
log.debug("Persisting grammars dependency status: " + statusFile);
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream(
statusFile));
try {
out.writeObject(grammars);
} finally {
out.close();
}
}
}
/**
* Performs dependency analysis for the given grammar files.
*
* @param grammarFiles the grammar files.
* @param importGrammarFiles the import grammar files.
* @param tool the tool to use.
*
* @return self-reference.
*/
public GrammarDependencies analyze(Set<File> grammarFiles,
Set<File> importGrammarFiles, Tool tool) throws IOException {
log.debug("Analysing grammar dependencies " + sourceDirectory);
// for dependency analysis we require all grammars
Collection<File> grammarsAndTokens = new HashSet<File>();
grammarsAndTokens.addAll(importGrammarFiles);
grammarsAndTokens.addAll(grammarFiles);
for (File grammarFile : grammarsAndTokens) {
// .tokens files must not be parsed, they can just be referenced
if (!grammarFile.getName().endsWith(".tokens"))
analyse(grammarFile, grammarsAndTokens, tool);
}
for (File grammarFile : grammarFiles) {
Collection<String> usages = findUsages(getRelativePath(grammarFile));
if (!usages.isEmpty()) {
grammars.put(grammarFile,
new AbstractMap.SimpleImmutableEntry<byte[], Collection<String>>(
MojoUtils.checksum(grammarFile), usages));
log.debug(" " + getRelativePath(grammarFile) + " used by " + usages);
}
}
for (File grammarFile : importGrammarFiles) {
// imported files are not allowed to be qualified
Collection<String> usages = findUsages(grammarFile.getName());
if (!usages.isEmpty()) {
grammars.put(grammarFile,
new AbstractMap.SimpleImmutableEntry<byte[], Collection<String>>(
MojoUtils.checksum(grammarFile), usages));
log.debug(" " + grammarFile.getName() + " imported by " + usages);
}
}
return this;
}
/**
* Determines whether a grammar used by the given grammar was modified since the last
* build.
*
* @param grammarFile the grammar.
*
* @return {@code true} if a grammar used by the given grammar has been modified.
*/
public boolean isDependencyChanged(File grammarFile) throws IOException {
String grammarPath = getRelativePath(grammarFile);
for (Map.Entry<File, Map.Entry<byte[], Collection<String>>> e : grammars.entrySet()) {
File depGrammarFile = e.getKey();
byte[] checksum = e.getValue().getKey();
Collection<String> usages = e.getValue().getValue();
if (usages.contains(grammarPath)) {
if (!depGrammarFile.exists() || !Arrays.equals(MojoUtils.checksum(depGrammarFile), checksum)) {
log.debug(" " + grammarPath + ": dependency " +
depGrammarFile.getName() + " changed");
return true;
}
}
}
return false;
}
/**
* Determines the relative target path of the given grammar file.
*
* @param grammarFile the grammar file.
*
* @return the relative path.
*/
private String getRelativePath(File grammarFile) {
// the library directory does not allow sub-directories
if (grammarFile.getPath().startsWith(libDirectory.getPath()))
return grammarFile.getName();
// if a package is given, we have to use it
if (packageName != null)
return packageName + grammarFile.getName();
// otherwise resolve the path relative to the source directory
String path = MojoUtils.findSourceSubdir(sourceDirectory, grammarFile);
return path + grammarFile.getName();
}
/**
* Returns the grammar file names that directly or indirectly use the given grammar.
*
* @param grammarFileName the grammar file name.
*
* @return the grammar file names that use the given grammar file.
*/
private Collection<String> findUsages(String grammarFileName) {
Collection<String> result = new ArrayList<String>();
explore(grammarFileName, result);
return result;
}
private void explore(String grammarName, Collection<String> result) {
for (Graph.Node<String> node : graph.getNode(grammarName).edges) {
result.add(node.payload);
explore(node.payload, result);
}
}
private void analyse(File grammarFile, Collection<File> grammarFiles, Tool tool) {
GrammarRootAST grammar = tool.parseGrammar(grammarFile.getAbsolutePath());
if (grammar == null)
return;
for (GrammarAST importDecl : grammar.getAllChildrenWithType(ANTLRParser.IMPORT)) {
Tree id = importDecl.getFirstChildWithType(ANTLRParser.ID);
// missing id is not valid, but we don't want to prevent the root cause from
// being reported by the ANTLR tool
if (id != null) {
String grammarPath = getRelativePath(grammarFile);
graph.addEdge(id.getText() + ".g4", grammarPath);
}
}
for (GrammarAST options : grammar.getAllChildrenWithType(ANTLRParser.OPTIONS)) {
for (int i = 0, count = options.getChildCount(); i < count; i++) {
Tree option = options.getChild(i);
if (option.getType() == ANTLRParser.ASSIGN) {
String key = option.getChild(0).getText();
String value = option.getChild(1).getText();
if ("tokenVocab".equals(key)) {
String name = stripQuotes(value);
// the grammar name may be qualified, but we resolve the path anyway
String grammarName = stripPath(name);
String grammarPath = MojoUtils.findSourceSubdir(sourceDirectory,
grammarFile);
File depGrammarFile = resolve(grammarName, grammarPath);
// if a package has been given, we use it instead of the file directory path
// (files probably reside in the root directory anyway with such a configuration )
if (packageName != null)
grammarPath = packageName;
graph.addEdge(getRelativePath(depGrammarFile),
grammarPath + grammarFile.getName());
}
}
}
}
}
/**
* Resolves the given grammar name.
*
* @param name the name.
* @param path the relative path.
*
* @return the grammar file.
*/
private File resolve(String name, String path) {
File file = new File(sourceDirectory, path + name + ".g4");
if (file.exists())
return file;
file = new File(libDirectory, name + ".g4");
if (file.exists())
return file;
return new File(libDirectory, name + ".tokens");
}
private Map<File, Map.Entry<byte[], Collection<String>>> loadStatus(File statusFile) {
if (statusFile.exists()) {
log.debug("Load grammars dependency status: " + statusFile);
try {
ObjectInputStream in = new ObjectInputStream(new FileInputStream(
statusFile));
try {
@SuppressWarnings("unchecked")
Map<File, Map.Entry<byte[], Collection<String>>> data =
(Map<File, Map.Entry<byte[], Collection<String>>>)
in.readObject();
return data;
} finally {
in.close();
}
} catch (Exception ex) {
log.warn("Could not load grammar dependency status information", ex);
}
}
return new HashMap<File, Map.Entry<byte[], Collection<String>>>();
}
private String stripPath(String str) {
return str.replaceAll("^.*[/\\\\]", "");
}
private String stripQuotes(String str) {
return str.replaceAll("\\A'|'\\Z", "");
}
}

74
antlr4-maven-plugin/src/main/java/org/antlr/mojo/antlr4/MojoUtils.java Normal file
View File

@ -0,0 +1,74 @@
package org.antlr.mojo.antlr4;
import java.io.File;
import java.io.FileInputStream;
import java.io.InputStream;
import java.io.IOException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
class MojoUtils {
/**
* Creates the MD5 checksum for the given file.
*
* @param file the file.
*
* @return the checksum.
*/
public static byte[] checksum(File file) throws IOException {
try {
InputStream in = new FileInputStream(file);
byte[] buffer = new byte[2048];
MessageDigest complete = MessageDigest.getInstance("MD5");
try {
int n;
do {
n = in.read(buffer);
if (n > 0) {
complete.update(buffer, 0, n);
}
} while (n != -1);
} finally {
in.close();
}
return complete.digest();
} catch (NoSuchAlgorithmException ex) {
throw new IOException("Could not create checksum " + file, ex);
}
}
/**
* Given the source directory File object and the full PATH to a grammar, produce the
* path to the named grammar file in relative terms to the {@code sourceDirectory}.
* This will then allow ANTLR to produce output relative to the base of the output
* directory and reflect the input organization of the grammar files.
*
* @param sourceDirectory The source directory {@link File} object
* @param grammarFileName The full path to the input grammar file
*
* @return The path to the grammar file relative to the source directory
*/
public static String findSourceSubdir(File sourceDirectory, File grammarFile) {
String srcPath = sourceDirectory.getPath() + File.separator;
String path = grammarFile.getPath();
if (!path.startsWith(srcPath)) {
throw new IllegalArgumentException("expected " + path +
" to be prefixed with " + sourceDirectory);
}
File unprefixedGrammarFileName = new File(path.substring(srcPath.length()));
if (unprefixedGrammarFileName.getParent() == null) {
return "";
}
return unprefixedGrammarFileName.getParent() + File.separator;
}
}

356
antlr4-maven-plugin/src/test/java/org/antlr/mojo/antlr4/Antlr4MojoTest.java Normal file
View File

@ -0,0 +1,356 @@
package org.antlr.mojo.antlr4;
import io.takari.maven.testing.TestMavenRuntime;
import io.takari.maven.testing.TestResources;
import org.apache.maven.execution.MavenSession;
import org.apache.maven.plugin.MojoExecution;
import org.apache.maven.plugin.MojoExecutionException;
import org.apache.maven.project.MavenProject;
import org.codehaus.plexus.util.xml.Xpp3Dom;
import static org.junit.Assert.*;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.ExpectedException;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.Arrays;
public class Antlr4MojoTest {
@Rule
public ExpectedException thrown = ExpectedException.none();
@Rule
public final TestResources resources = new TestResources();
@Rule
public final TestMavenRuntime maven = new TestMavenRuntime();
@Test
public void importTokens() throws Exception {
Path baseDir = resources.getBasedir("importTokens").toPath();
Path antlrDir = baseDir.resolve("src/main/antlr4");
Path generatedSources = baseDir.resolve("target/generated-sources/antlr4");
Path genParser = generatedSources.resolve("test/SimpleParser.java");
Path tokens = antlrDir.resolve("imports/SimpleLexer.tokens");
MavenProject project = maven.readMavenProject(baseDir.toFile());
MavenSession session = maven.newMavenSession(project);
MojoExecution exec = maven.newMojoExecution("antlr4");
////////////////////////////////////////////////////////////////////////
// 1st - all grammars have to be processed
////////////////////////////////////////////////////////////////////////
assertFalse(Files.exists(genParser));
maven.executeMojo(session, project, exec);
assertTrue(Files.exists(genParser));
////////////////////////////////////////////////////////////////////////
// 2nd - nothing has been modified, no grammars have to be processed
////////////////////////////////////////////////////////////////////////
{
byte[] sum = checksum(genParser);
maven.executeMojo(session, project, exec);
assertTrue(Arrays.equals(sum, checksum(genParser)));
}
////////////////////////////////////////////////////////////////////////
// 3rd - the imported grammar changed, every dependency has to be processed
////////////////////////////////////////////////////////////////////////
try(Change change = Change.of(tokens, "DOT=4")) {
byte[] sum = checksum(genParser);
maven.executeMojo(session, project, exec);
assertFalse(Arrays.equals(sum, checksum(genParser)));
}
}
@Test
public void importsCustomLayout() throws Exception {
Path baseDir = resources.getBasedir("importsCustom").toPath();
Path antlrDir = baseDir.resolve("src/main/antlr4");
Path generatedSources = baseDir.resolve("src/main/java");
Path genTestLexer = generatedSources.resolve("foo/TestLexer.java");
Path genTestParser = generatedSources.resolve("foo/TestParser.java");
Path genHello = generatedSources.resolve("foo/HelloParser.java");
Path baseGrammar = antlrDir.resolve("imports/TestBaseLexer.g4");
Path lexerGrammar = antlrDir.resolve("TestLexer.g4");
Path parserGrammar = antlrDir.resolve("TestParser.g4");
Xpp3Dom outputDirectory = TestMavenRuntime.newParameter("outputDirectory",
"src/main/java/foo");
Xpp3Dom arguments = new Xpp3Dom("arguments");
arguments.addChild(TestMavenRuntime.newParameter("argument", "-package"));
arguments.addChild(TestMavenRuntime.newParameter("argument", "foo"));
MavenProject project = maven.readMavenProject(baseDir.toFile());
MavenSession session = maven.newMavenSession(project);
MojoExecution exec = maven.newMojoExecution("antlr4", outputDirectory, arguments);
////////////////////////////////////////////////////////////////////////
// 1st - all grammars have to be processed
////////////////////////////////////////////////////////////////////////
assertFalse(Files.exists(genHello));
assertFalse(Files.exists(genTestParser));
assertFalse(Files.exists(genTestLexer));
maven.executeMojo(session, project, exec);
assertTrue(Files.exists(genHello));
assertTrue(Files.exists(genTestParser));
assertTrue(Files.exists(genTestLexer));
////////////////////////////////////////////////////////////////////////
// 2nd - nothing has been modified, no grammars have to be processed
////////////////////////////////////////////////////////////////////////
{
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertTrue(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 3rd - the imported grammar changed, every dependency has to be processed
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(baseGrammar, "DOT: '.' ;")) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 4th - the lexer grammar changed, the parser grammar has to be processed as well
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(lexerGrammar, "fragment DOT : '.';")) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 5th - the parser grammar changed, no other grammars have to be processed
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(parserGrammar, " t : WS* ;")) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
}
@Test
public void importsStandardLayout() throws Exception {
Path baseDir = resources.getBasedir("importsStandard").toPath();
Path antlrDir = baseDir.resolve("src/main/antlr4");
Path generatedSources = baseDir.resolve("target/generated-sources/antlr4");
Path genTestLexer = generatedSources.resolve("test/TestLexer.java");
Path genTestParser = generatedSources.resolve("test/TestParser.java");
Path genHello = generatedSources.resolve("test/HelloParser.java");
Path baseGrammar = antlrDir.resolve("imports/TestBaseLexer.g4");
Path lexerGrammar = antlrDir.resolve("test/TestLexer.g4");
Path parserGrammar = antlrDir.resolve("test/TestParser.g4");
MavenProject project = maven.readMavenProject(baseDir.toFile());
MavenSession session = maven.newMavenSession(project);
MojoExecution exec = maven.newMojoExecution("antlr4");
////////////////////////////////////////////////////////////////////////
// 1st - all grammars have to be processed
////////////////////////////////////////////////////////////////////////
assertFalse(Files.exists(genHello));
assertFalse(Files.exists(genTestParser));
assertFalse(Files.exists(genTestLexer));
maven.executeMojo(session, project, exec);
assertTrue(Files.exists(genHello));
assertTrue(Files.exists(genTestParser));
assertTrue(Files.exists(genTestLexer));
////////////////////////////////////////////////////////////////////////
// 2nd - nothing has been modified, no grammars have to be processed
////////////////////////////////////////////////////////////////////////
{
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertTrue(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 3rd - the imported grammar changed, every dependency has to be processed
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(baseGrammar, "DOT: '.' ;")) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 4th - the lexer grammar changed, the parser grammar has to be processed as well
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(lexerGrammar)) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertFalse(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
////////////////////////////////////////////////////////////////////////
// 5th - the parser grammar changed, no other grammars have to be processed
////////////////////////////////////////////////////////////////////////
// modify the grammar to make checksum comparison detect a change
try(Change change = Change.of(parserGrammar, " t : WS* ;")) {
byte[] testLexerSum = checksum(genTestLexer);
byte[] testParserSum = checksum(genTestParser);
byte[] helloSum = checksum(genHello);
maven.executeMojo(session, project, exec);
assertTrue(Arrays.equals(testLexerSum, checksum(genTestLexer)));
assertFalse(Arrays.equals(testParserSum, checksum(genTestParser)));
assertTrue(Arrays.equals(helloSum, checksum(genHello)));
}
}
@Test
public void processWhenDependencyRemoved() throws Exception {
Path baseDir = resources.getBasedir("dependencyRemoved").toPath();
Path antlrDir = baseDir.resolve("src/main/antlr4");
Path baseGrammar = antlrDir.resolve("imports/HelloBase.g4");
MavenProject project = maven.readMavenProject(baseDir.toFile());
MavenSession session = maven.newMavenSession(project);
MojoExecution exec = maven.newMojoExecution("antlr4");
maven.executeMojo(session, project, exec);
try(Change temp = Change.of(baseGrammar)) {
// if the base grammar no longer exists, processing must be performed
Files.delete(baseGrammar);
thrown.expect(MojoExecutionException.class);
thrown.expectMessage("ANTLR 4 caught 1 build errors.");
maven.executeMojo(session, project, exec);
}
}
private byte[] checksum(Path path) throws IOException {
return MojoUtils.checksum(path.toFile());
}
private static class Change implements AutoCloseable {
final Path file;
final byte[] original;
public Change(Path file, String change) {
this.file = file;
try {
original = Files.readAllBytes(file);
} catch (IOException ex) {
throw new RuntimeException("Could not read file " + file);
}
String text = new String(original, StandardCharsets.UTF_8) + change;
write(file, text.getBytes(StandardCharsets.UTF_8));
}
private void write(Path file, byte[] data) {
try {
Files.write(file, data);
} catch (IOException ex) {
throw new RuntimeException("Could not write file " + file);
}
}
public static Change of(Path file, String change) {
return new Change(file, change);
}
public static Change of(Path file) {
return new Change(file, "\n");
}
@Override
public void close() {
write(file, original);
}
}
}

27
antlr4-maven-plugin/src/test/projects/dependencyRemoved/pom.xml Normal file
View File

@ -0,0 +1,27 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>deps.removed</groupId>
<artifactId>depRemoved</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Test processing after dependency removed</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>antlr4-maven-plugin</artifactId>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</project>

16
antlr4-maven-plugin/src/test/projects/dependencyRemoved/src/main/antlr4/imports/HelloBase.g4 Normal file
View File

@ -0,0 +1,16 @@
lexer grammar TestBaseLexer;
tokens { Name }
// Default "mode": Everything OUTSIDE of a tag
Comment : '<!--' .*? '-->' ;
CDSect : '<![CDATA[' .*? ']]>' ;
fragment
Whitespace : ' ' | '\n' | '\t' | '\r' ;
fragment
Hexdigit : [a-fA-F0-9] ;
fragment
Digit : [0-9] ;

7
antlr4-maven-plugin/src/test/projects/dependencyRemoved/src/main/antlr4/test/Hello.g4 Normal file
View File

@ -0,0 +1,7 @@
grammar Hello;
import HelloBase;
r : 'hello' ID ;
ID : [a-z]+ ;
WS : [ \r\t\n]+ -> skip ;

27
antlr4-maven-plugin/src/test/projects/importTokens/pom.xml Normal file
View File

@ -0,0 +1,27 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>import.tokens</groupId>
<artifactId>importTokens</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Test importing tokens file</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>antlr4-maven-plugin</artifactId>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</project>

3
antlr4-maven-plugin/src/test/projects/importTokens/src/main/antlr4/imports/SimpleLexer.tokens Normal file
View File

@ -0,0 +1,3 @@
ID=1
INT=2
SEMI=3

8
antlr4-maven-plugin/src/test/projects/importTokens/src/main/antlr4/test/SimpleParser.g4 Normal file
View File

@ -0,0 +1,8 @@
parser grammar SimpleParser;
options {
// get token types from SimpleLexer.tokens; don't name it
// SimpleParser.tokens as ANTLR will overwrite!
tokenVocab=SimpleLexer;
}
s : ( ID | INT )* SEMI ;

42
antlr4-maven-plugin/src/test/projects/importsCustom/pom.xml Normal file
View File

@ -0,0 +1,42 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>imports.custom</groupId>
<artifactId>importsCustom</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Test importing, custom layout</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>antlr4-maven-plugin</artifactId>
<configuration>
<outputDirectory>${basedir}/src/main/java/com/foo</outputDirectory>
<arguments>
<argument>-visitor</argument>
<argument>-no-listener</argument>
<argument>-Xlog</argument>
<argument>-package</argument>
<argument>com.foo</argument>
</arguments>
</configuration>
<executions>
<execution>
<goals>
<goal>antlr4</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>

4
antlr4-maven-plugin/src/test/projects/importsCustom/src/main/antlr4/Hello.g4 Normal file
View File

@ -0,0 +1,4 @@
grammar Hello;
r : 'hello' ID ;
ID : [a-z]+ ;
WS : [ \r\t\n]+ -> skip ;

6
antlr4-maven-plugin/src/test/projects/importsCustom/src/main/antlr4/TestLexer.g4 Normal file
View File

@ -0,0 +1,6 @@
lexer grammar TestLexer;
import TestBaseLexer;
WS : Whitespace+ -> skip;
TEXT : ~[<&]+ ; // match any 16 bit char other than < and &

5
antlr4-maven-plugin/src/test/projects/importsCustom/src/main/antlr4/TestParser.g4 Normal file
View File

@ -0,0 +1,5 @@
parser grammar TestParser;
options { tokenVocab=TestLexer; }
document : (Comment | Name) EOF ;

16
antlr4-maven-plugin/src/test/projects/importsCustom/src/main/antlr4/imports/TestBaseLexer.g4 Normal file
View File

@ -0,0 +1,16 @@
lexer grammar TestBaseLexer;
tokens { Name }
// Default "mode": Everything OUTSIDE of a tag
Comment : '<!--' .*? '-->' ;
CDSect : '<![CDATA[' .*? ']]>' ;
fragment
Whitespace : ' ' | '\n' | '\t' | '\r' ;
fragment
Hexdigit : [a-fA-F0-9] ;
fragment
Digit : [0-9] ;

27
antlr4-maven-plugin/src/test/projects/importsStandard/pom.xml Normal file
View File

@ -0,0 +1,27 @@
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>imports.standard</groupId>
<artifactId>importsStandard</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<name>Test importing, standard layout</name>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<artifactId>antlr4-maven-plugin</artifactId>
<configuration>
</configuration>
</plugin>
</plugins>
</build>
</project>

16
antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/imports/TestBaseLexer.g4 Normal file
View File

@ -0,0 +1,16 @@
lexer grammar TestBaseLexer;
tokens { Name }
// Default "mode": Everything OUTSIDE of a tag
Comment : '<!--' .*? '-->' ;
CDSect : '<![CDATA[' .*? ']]>' ;
fragment
Whitespace : ' ' | '\n' | '\t' | '\r' ;
fragment
Hexdigit : [a-fA-F0-9] ;
fragment
Digit : [0-9] ;

4
antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/test/Hello.g4 Normal file
View File

@ -0,0 +1,4 @@
grammar Hello;
r : 'hello' ID ;
ID : [a-z]+ ;
WS : [ \r\t\n]+ -> skip ;

6
antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/test/TestLexer.g4 Normal file
View File

@ -0,0 +1,6 @@
lexer grammar TestLexer;
import TestBaseLexer;
WS : Whitespace+ -> skip;
TEXT : ~[<&]+ ; // match any 16 bit char other than < and &

5
antlr4-maven-plugin/src/test/projects/importsStandard/src/main/antlr4/test/TestParser.g4 Normal file
View File

@ -0,0 +1,5 @@
parser grammar TestParser;
options { tokenVocab=TestLexer; }
document : (Comment | Name) EOF ;

8
appveyor.yml Normal file
View File

@ -0,0 +1,8 @@
version: '4.6-SNAPSHOT+AppVeyor.{build}'
os: Windows Server 2012
build_script:
- mvn -DskipTests install -q --batch-mode
test_script:
- mvn install -q -Dantlr-python2-python="C:\Python27\python.exe" -Dantlr-python3-python="C:\Python35\python.exe" -Dantlr-javascript-nodejs="C:\Program Files (x86)\nodejs\node.exe" --batch-mode
build:
verbosity: minimal

28
contributors.txt
View File

@ -1,4 +1,4 @@
ANTLR Project Contributors Certification of Origin and Rights
ANTLR Project Contributors Certification of Origin and Rights
All contributors to ANTLR v4 must formally agree to abide by this
certificate of origin by signing on the bottom with their github
@ -58,6 +58,7 @@ YYYY/MM/DD, github id, Full name, email
2014/03/18, aphyr, Kyle Kingsbury, aphyr@aphyr.com
2014/06/07, ericvergnaud, Eric Vergnaud, eric.vergnaud@wanadoo.fr
2014/07/04, jimidle, Jim Idle, jimi@Idle.ws
2014/01/01, danmclaughlin, Dan McLaughlin, dan.mclaughlin@gmail.com
2014/09/04. jeduden, Jan-Eric Duden, jeduden@gmail.com
2014/09/27, petrbel, Petr Bělohlávek, antlr@petrbel.cz
2014/10/18, sergiusignacius, Sérgio Silva, serge.a.silva@gmail.com
@ -89,7 +90,32 @@ YYYY/MM/DD, github id, Full name, email
2015/12/23, pboyer, Peter Boyer, peter.b.boyer@gmail.com
2015/12/24, dtymon, David Tymon, david.tymon@gmail.com
2016/02/18, reitzig, Raphael Reitzig, reitzig[at]cs.uni-kl.de
2016/03/10, mike-lischke, Mike Lischke, mike@lischke-online.de
2016/03/27, beardlybread, Bradley Steinbacher, bradley.j.steinbacher@gmail.com
2016/03/29, msteiger, Martin Steiger, antlr@martin-steiger.de
2016/03/28, gagern, Martin von Gagern, gagern@ma.tum.de
2016/07/10, twz123, Tom Wieczorek, tom.wieczorek@zalando.de
2016/07/20, chrisheller, Chris Heller, chris.heller.greyheller@gmail.com
2016/07/20, nburles, Nathan Burles, nburles@gmail.com
2016/07/20, kosl90, Li Liqiang, kos1990l@gmail.com
2016/07/27, timoc, Tim O'Callaghan, timo@linux.com
2016/07/26, nic30, Michal Orsák, michal.o.socials@gmail.com
2016/07/18, willfaught, Will Faught, will.faught@gmail.com
2016/08/08, wjkohnen, Wolfgang Johannes Kohnen, wjkohnen-go-antlr@ko-sys.com
2016/08/11, BurtHarris, Ralph "Burt" Harris, Burt_Harris_antlr4@azxs.33mail.com
2016/08/19, andjo403, Andreas Jonson, andjo403@hotmail.com
2016/09/27, harriman, Kurt Harriman, harriman@acm.org
2016/10/13, cgudrian, Christian Gudrian, christian.gudrian@gmx.de
2016/10/13, nielsbasjes, Niels Basjes, niels@basjes.nl
2016/10/21, FloorGoddijn, Floor Goddijn, floor.goddijn[at]aimms.com
2016/11/01, RYDB3RG, Kai Stammerjohann, RYDB3RG@users.noreply.github.com
2016/11/05, runner-mei, meifakun, runner.mei@gmail.com
2016/11/15, hanjoes, Hanzhou Shi, hanzhou87@gmail.com
2016/11/16, sridharxp, Sridharan S, aurosridhar@gmail.com
2016/11/06, NoodleOfDeath, Thom Morgan, github@bytemeapp.com
2016/11/01, sebkur, Sebastian Kürten, sebastian@topobyte.de
2016/04/13, renatahodovan, Renata Hodovan, reni@inf.u-szeged.hu
2016/11/05, ewanmellor, Ewan Mellor, github@ewanmellor.org
2016/11/06, janyou, Janyou, janyou.antlr@outlook.com
2016/11/20, marcohu, Marco Hunsicker, antlr@hunsicker.de
2016/09/02, lygav, Vladimir (Vladi) Lyga, lyvladi@gmail.com

12
doc/actions.md
View File

@ -21,7 +21,7 @@ decl: type ID ';'
## Token Attributes
All tokens have a collection of predefined, read-only attributes. The attributes include useful token properties such as the token type and text matched for a token. Actions can access these attributes via $ label.attribute where label labels a particular instance of a token reference (a and b in the example below are used in the action code as $a and $b). Often, a particular token is only referenced once in the rule, in which case the token name itself can be used unambiguously in the action code (token INT can be used as $INT in the action). The following example illustrates token attribute expression syntax:
All tokens have a collection of predefined, read-only attributes. The attributes include useful token properties such as the token type and text matched for a token. Actions can access these attributes via `$label.attribute` where label labels a particular instance of a token reference (`a` and `b` in the example below are used in the action code as `$a` and `$b`). Often, a particular token is only referenced once in the rule, in which case the token name itself can be used unambiguously in the action code (token `INT` can be used as `$INT` in the action). The following example illustrates token attribute expression syntax:
```
r : INT {int x = $INT.line;}
@ -34,7 +34,7 @@ The action within the `(...)?` subrule can see the `INT` token matched before it
Because there are two references to the `FLOAT` token, a reference to `$FLOAT` in an action is not unique; you must use labels to specify which token reference you’re interested in.
Token references within different alternatives are unique because only one of them can be matched for any invocation of the rule. For example, in the following rule, actions in both alternatives can reference $ID directly without using a label:
Token references within different alternatives are unique because only one of them can be matched for any invocation of the rule. For example, in the following rule, actions in both alternatives can reference `$ID` directly without using a label:
```
r : ... ID {System.out.println($ID.text);}
@ -72,7 +72,7 @@ Most of the time you access the attributes of the token, but sometimes it is use
## Parser Rule Attributes
ANTLR predefines a number of read-only attributes associated with parser rule references that are available to actions. Actions can access rule attributes only for references that precede the action. The syntax is $ r.attr for rule name r or a label assigned to a rule reference. For example, $expr.text returns the complete text matched by a preceding invocation of rule expr:
ANTLR predefines a number of read-only attributes associated with parser rule references that are available to actions. Actions can access rule attributes only for references that precede the action. The syntax is `$r.attr` for rule name `r` or a label assigned to a rule reference. For example, `$expr.text` returns the complete text matched by a preceding invocation of rule `expr`:
```
returnStat : 'return' expr {System.out.println("matched "+$expr.text);} ;
@ -101,7 +101,7 @@ returnStat : 'return' expr {System.out.println("first token "+$start.getText());
## Dynamically-Scoped Attributes
You can pass information to and from rules using parameters and return values, just like functions in a general-purpose programming language. Programming languages don’t allow functions to access the local variables or parameters of invoking functions, however. For example, the following reference to local variable xfrom a nested method call is illegal in Java:
You can pass information to and from rules using parameters and return values, just like functions in a general-purpose programming language. Programming languages don’t allow functions to access the local variables or parameters of invoking functions, however. For example, the following reference to local variable `x` form a nested method call is illegal in Java:
```java
void f() {
@ -116,7 +116,7 @@ void h() {
}
```
Variable x is available only within the scope of f, which is the text lexically delimited by curly brackets. For this reason, Java is said to use lexical scoping. Lexical scoping is the norm for most programming languages. Languages that allow methods further down in the call chain to access local variables defined earlier are said to use dynamic scoping. The term dynamic refers to the fact that a compiler cannot statically determine the set of visible variables. This is because the set of variables visible to a method changes depending on who calls that method.
Variable `x` is available only within the scope of `f`, which is the text lexically delimited by curly brackets. For this reason, Java is said to use lexical scoping. Lexical scoping is the norm for most programming languages. Languages that allow methods further down in the call chain to access local variables defined earlier are said to use dynamic scoping. The term dynamic refers to the fact that a compiler cannot statically determine the set of visible variables. This is because the set of variables visible to a method changes depending on who calls that method.
It turns out that, in the grammar realm, distant rules sometimes need to communicate with each other, mostly to provide context information to rules matched below in the rule invocation chain. (Naturally, this assumes that you are using actions directly in the grammar instead of the parse-tree listener event mechanism.) ANTLR allows dynamic scoping in that actions can access attributes from invoking rules using syntax `$r::x` where `r` is a rule name and `x` is an attribute within that rule. It is up to the programmer to ensure that `r` is in fact an invoking rule of the current rule. A runtime exception occurs if `r` is not in the current call chain when you access `$r::x`.
@ -176,7 +176,7 @@ $ grun DynScope prog
symbols=[i]
```
There’s an important difference between a simple field declaration in a `@members` action and dynamic scoping. symbols is a local variable and so there is a copy for each invocation of rule `block`. That’s exactly what we want for nested blocks so that we can reuse the same input variable name in an inner block. For example, the following nested code block redefines i in the inner scope. This new definition must hide the definition in the outer scope.
There’s an important difference between a simple field declaration in a `@members` action and dynamic scoping. symbols is a local variable and so there is a copy for each invocation of rule `block`. That’s exactly what we want for nested blocks so that we can reuse the same input variable name in an inner block. For example, the following nested code block redefines `i` in the inner scope. This new definition must hide the definition in the outer scope.
```
{

189
doc/adding-tests.md
View File

@ -1,96 +1,125 @@
# Adding unit tests
## Generating Runtime Tests
## Introduction
Because ANTLR supports multiple target languages, the unit tests are broken into two groups: the unit tests that test the tool itself (in `tool-testsuite`) and the unit tests that test the parser runtimes (in antlr4/runtime-testsuite). To avoid a lot of cut-and-paste, we generate all **runtime** tests from a set of templates using [runtime-testsuite/src/org/antlr/v4/testgen/TestGenerator.java](../runtime-testsuite/src/org/antlr/v4/testgen/TestGenerator.java). The `mvn` command is simple to use:
Because ANTLR supports multiple target languages, the unit tests are broken into two groups: the unit tests that test the tool itself (in `tool-testsuite`) and the unit tests that test the parser runtimes (in `antlr4/runtime-testsuite`). The tool tests are straightforward because they are Java code testing Java code; see the section at the bottom of this file.
```
$ cd ~/antlr/code/antlr4/runtime-testsuite
$ mvn -Pgen generate-test-sources
...
rootDir = /Users/parrt/antlr/code/antlr4/runtime-testsuite
outputDir = /Users/parrt/antlr/code/antlr4/runtime-testsuite/test
templates = /Users/parrt/antlr/code/antlr4/runtime-testsuite/resources/org/antlr/v4/test/runtime/templates
target = ALL
browsers = false
viz = false
The runtime tests must be specified in a generic fashion to work across language targets. Furthermore, we must test the various targets from Java. This usually means Java launching processes to compile, say, C++ and run parsers.
As of 4.6, we use [a Java descriptor object](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/RuntimeTestDescriptor.java) to describe each runtime test. Unit tests are grouped together into categories such as [ParserExecDescriptors](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/ParserExecDescriptors.java), which has multiple nested descriptor objects, one per test. For example, here is the start of that file:
```java
public class ParserExecDescriptors {
public static class APlus extends BaseParserTestDescriptor {
public String input = "a b c";
public String output = "abc\n";
public String errors = "";
public String startRule = "a";
public String grammarName = "T";
/**
grammar T;
a : ID+ {
<writeln("$text")>
};
ID : 'a'..'z'+;
WS : (' '|'\n') -> skip;
*/
@CommentHasStringValue
public String grammar;
}
```
It basically runs the Java program:
The mysterious `@CommentHasStringValue` annotation is a bit of a hack that allows multi-line strings in Java. This kung fu is required so that we can use Java classes rather than StringTemplate group files to specify runtime tests (the legacy system used those and it was hard to get them right). Here are all the [Runtime test descriptors](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors) organized into groups.
The grammars are strings representing StringTemplates (`ST` objects) so `<writeln("$text")>` will get replace when the unit test file is generated (`Test.java`, `Test.cs`, ...). The `writeln` template must be defined per target. Here are all of the
[Target templates for runtime tests](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/resources/org/antlr/v4/test/runtime/templates).
## Running the runtime tests
A single test rig is sufficient to test all targets against all descriptors using the [junit parameterized tests](https://github.com/junit-team/junit4/wiki/parameterized-tests) mechanism. But, that is inconvenient because we often want to test just a single target or perhaps even just a single test within a single group of a single target. I have automatically generated a bunch of
[Target runtime test rigs](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/test/org/antlr/v4/test/runtime) that allow developers such flexibility. For example, here are the Python3 test rigs in intellij:
<img src=images/testrigs.png width=300>
And the result of testing the entire subdirectory:
<img src=images/python3-tests.png width=400>
From `mvn`, on the commandline, you will see:
```bash
$ java org.antlr.v4.testgen.TestGenerator \
-root ~/antlr/code/antlr4/runtime-testsuite \
-outdir ~/antlr/code/antlr4/runtime-testsuite/test \
-templates ~/antlr/code/antlr4/runtime-testsuite/resources/org/antlr/v4/test/runtime/templates
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.antlr.v4.test.runtime.javascript.node.TestCompositeLexers
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.581 sec
Running org.antlr.v4.test.runtime.javascript.node.TestLexerErrors
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.721 sec
Running org.antlr.v4.test.runtime.javascript.node.TestSemPredEvalParser
Tests run: 26, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.084 sec
Running org.antlr.v4.test.runtime.javascript.node.TestSets
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.798 sec
Running org.antlr.v4.test.runtime.javascript.node.TestPerformance
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.505 sec
Running org.antlr.v4.test.runtime.javascript.node.TestSemPredEvalLexer
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.994 sec
Running org.antlr.v4.test.runtime.javascript.node.TestLexerExec
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.433 sec
...
```
## Adding a runtime test
For each target, you will find an `Index.stg` file with a dictionary of all test groups. E.g., `runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/Index.stg` looks like:
To add a new runtime test, first determine which [group of tests](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors) it belongs to. Then, add a new [RuntimeTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/RuntimeTestDescriptor.java) implementation by subclassing one of:
```
TestFolders ::= [
"CompositeLexers": [],
"CompositeParsers": [],
"FullContextParsing": [],
"LeftRecursion": [],
"LexerErrors": [],
"LexerExec": [],
"Listeners": [],
"ParserErrors": [],
"ParserExec": [],
"ParseTrees": [],
"Performance": [],
"SemPredEvalLexer": [],
"SemPredEvalParser": [],
"Sets": []
]
* [BaseParserTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/BaseParserTestDescriptor.java); see example [APlus](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/ParserExecDescriptors.java#L7).
* [BaseDiagnosticParserTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/BaseDiagnosticParserTestDescriptor) if you want to test parser diagnostic output; see [example output](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/FullContextParsingDescriptors.java#L16).
* [BaseCompositeParserTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/BaseCompositeParserTestDescriptor.java); see example [BringInLiteralsFromDelegate](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/CompositeParsersDescriptors.java#L11)
* [BaseLexerTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/BaseLexerTestDescriptor.java); see example [ActionPlacement](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/LexerExecDescriptors.java#L12).
* [BaseCompositeLexerTestDescriptor](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/BaseCompositeLexerTestDescriptor.java); see example [LexerDelegatorInvokesDelegateRule](https://github.com/antlr/antlr4/blob/master/runtime-testsuite/test/org/antlr/v4/test/runtime/descriptors/CompositeLexersDescriptors.java#L11)
Each descriptor object describes the following mandatory elements for the test:
* the test type
* the grammar
* the start rule
* the input text to parse or lex
* the expected output
* the expected errors
Your best bet is to find a similar test in the appropriate group and then copy and paste the descriptor object, creating a new nested class within the test group class. Modify the field definitions to suit your new problem.
If you need to create a whole new group of tests, it requires a new descriptor class; call it `XDescriptors`. Then, in each [target subdirectory](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/test/org/antlr/v4/test/runtime), you need to create a new test rig `TestX.java` file:
```java
package org.antlr.v4.test.runtime.java;
import org.antlr.v4.test.runtime.BaseRuntimeTest;
import org.antlr.v4.test.runtime.RuntimeTestDescriptor;
import org.antlr.v4.test.runtime.descriptors.ListenersDescriptors;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@RunWith(Parameterized.class)
public class TestX extends BaseRuntimeTest {
public TestX(RuntimeTestDescriptor descriptor) {
super(descriptor,new Base<TARGET>Test());
}
@Parameterized.Parameters(name="{0}")
public static RuntimeTestDescriptor[] getAllTestDescriptors() {
return BaseRuntimeTest.getRuntimeTestDescriptors(XDescriptors.class, "<TARGET>");
}
}
```
Then each group has a subdirectory with another index. E.g., `Sets/Index.stg` looks like:
where `<TARGET>` is replaced with Java, Cpp, CSharp, Python2, ... in the various subdirectories.
### Ignoring tests
```
TestTemplates ::= [
"SeqDoesNotBecomeSet": [],
"ParserSet": [],
"ParserNotSet": [],
"ParserNotToken": [],
"ParserNotTokenWithLabel": [],
"RuleAsSet": [],
"NotChar": [],
"OptionalSingleElement": [],
...
```
For every name mentioned, you will find a `.stg` file with the actual test. E.g., `Sets/StarSet.stg`:
```
TestType() ::= "Parser"
Options ::= [
"Debug": false
]
Grammar ::= [
"T": {<grammar("T")>}
]
Input() ::= "abaac"
Rule() ::= "a"
Output() ::= <<
abaac<\n>
>>
Errors() ::= ""
grammar(grammarName) ::= <<
grammar <grammarName>;
a : ('a'|'b')* 'c' {<InputText():writeln()>} ;
>>
```
In order to turn off a test for a particular target, we need to use the `ignore` method. Given a target name, a descriptor object can decide whether to ignore the test. This is not always convenient but it is fully general and works well for the one case we have now where we have to ignore `Visitor` tests in all targets except JavaScript.
### Cross-language actions embedded within grammars
@ -106,14 +135,16 @@ Use instead the language-neutral:
<writeln("$set.stop")>
```
File `runtime-testsuite/resources/org/antlr/v4/test/runtime/java/Java.test.stg` has templates like:
Template file [runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/Java.test.stg](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/Java.test.stg) has templates like:
```
writeln(s) ::= <<System.out.println(<s>);>>
```
that translate generic operations to target-specific language statements or expressions.
## Adding an ANTLR tool unit test
Just go into the appropriate Java test class in dir `antlr4/tool-testsuite/test/org/antlr/v4/test/tool` and add your unit test.
Just go into the appropriate Java test class in dir [antlr4/tool-testsuite/test/org/antlr/v4/test/tool](https://github.com/antlr/antlr4/tree/master/tool-testsuite/test/org/antlr/v4/test/tool) and add your unit test.

194
doc/building-antlr.md
View File

@ -7,20 +7,22 @@ Most programmers do not need the information on this page because they will simp
I will assume that the root directory is `/tmp` for the purposes of explaining how to build ANTLR in this document.
*As of 4.6, ANTLR tool and Java-target runtime requires Java 7.*
# Get the source
The first step is to get the Java source code from the ANTLR 4 repository at github. You can download the repository from github, but the easiest thing to do is simply clone the repository on your local disk:
```bash
$ cd /tmp
/tmp $ git clone git@github.com:antlr/antlr4.git
/tmp $ git clone https://github.com/antlr/antlr4.git
Cloning into 'antlr4'...
remote: Counting objects: 43273, done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 43273 (delta 26), reused 0 (delta 0)
Receiving objects: 100% (43273/43273), 18.76 MiB | 1.60 MiB/s, done.
Resolving deltas: 100% (22419/22419), done.
remote: Counting objects: 61480, done.
remote: Total 61480 (delta 0), reused 0 (delta 0), pack-reused 61480
Receiving objects: 100% (61480/61480), 31.24 MiB | 7.18 MiB/s, done.
Resolving deltas: 100% (32970/32970), done.
Checking connectivity... done.
Checking out files: 100% (1427/1427), done.
```
# Compile
@ -36,34 +38,55 @@ Receiving objects: 100% (59858/59858), 31.10 MiB | 819.00 KiB/s, done.
Resolving deltas: 100% (31898/31898), done.
Checking connectivity... done.
$ cd antlr4
$ mvn compile
..
$ mvn -DskipTests install
...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.447 s]
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 3.113 s]
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 14.408 s]
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 1.276 s]
[INFO] ANTLR 4 Runtime Test Generator ..................... SUCCESS [ 0.773 s]
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 6.920 s]
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.287 s]
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 4.915 s]
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 1.315 s]
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 2.393 s]
[INFO] ANTLR 4 Runtime Test Annotations ................... SUCCESS [ 0.078 s]
[INFO] ANTLR 4 Runtime Test Processors .................... SUCCESS [ 0.019 s]
[INFO] ANTLR 4 Runtime Tests (2nd generation) ............. SUCCESS [ 1.986 s]
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 0.513 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
...
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.005 s
[INFO] Finished at: 2016-11-21T11:42:42-08:00
[INFO] Final Memory: 52M/434M
[INFO] ------------------------------------------------------------------------
```
We do `install` not `compile` as tool tests and such refer to modules that must be pulled from the maven install local cache.
# Installing libs to mvn cache locally
To skip the tests (which require all the target languages be installed) and **install into local repository** `~/.m2/repository/org/antlr`, do this:
```bash
$ mvn install -DskipTests=true # make sure all artifacts are visible on this machine
```
# Testing tool and targets
In order to perform the tests on all target languages, make sure that you have `mono` and `nodejs` installed. For example, on OS X:
In order to perform the tests on all target languages, you need to have the following languages installed:
```bash
$ brew install mono
$ brew install node
```
* `mono` (e.g., `brew install mono`)
* `nodejs`
* Python 2.7
* Python 3.5
* Go
* Swift 3 (via XCode 8.x) tested currently only osx
* clang (for C++ target)
To run the tests and **install into local repository** `~/.m2/repository/org/antlr`, do this:
```bash
$ mvn install
$ mvn install -DskipTests=true # make sure all artifacts are visible on this machine
$ mvn install # now "do it with feeling"
...
-------------------------------------------------------
T E S T S
@ -81,38 +104,123 @@ antlr reports warnings from [-visitor, -Dlanguage=CSharp, -o, /var/folders/s1/h3
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.462 s]
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 9.163 s]
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 3.683 s]
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 1.897 s]
[INFO] ANTLR 4 Runtime Test Generator ..................... SUCCESS [07:11 min]
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 16.694 s]
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.445 s]
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 3.392 s]
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 1.373 s]
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 1.519 s]
[INFO] ANTLR 4 Runtime Test Annotations ................... SUCCESS [ 0.086 s]
[INFO] ANTLR 4 Runtime Test Processors .................... SUCCESS [ 0.014 s]
[INFO] ANTLR 4 Runtime Tests (2nd generation) ............. SUCCESS [06:39 min]
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 6.922 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 07:43 min
...
[INFO] Total time: 06:53 min
[INFO] Finished at: 2016-11-16T15:36:56-08:00
[INFO] Final Memory: 44M/458M
[INFO] ------------------------------------------------------------------------
```
You should see these jars (building 4.5.2-SNAPSHOT):
Note: That is actually result of running the much faster:
`mvn -Dparallel=methods -DthreadCount=4 install`
You should see these jars (when building 4.6-SNAPSHOT):
```bash
/Users/parrt/.m2/repository/org/antlr $ find antlr4* -name '*.jar'
antlr4/4.5/antlr4-4.5.jar
antlr4/4.5.2-SNAPSHOT/antlr4-4.5.2-SNAPSHOT-tests.jar
antlr4/4.5.2-SNAPSHOT/antlr4-4.5.2-SNAPSHOT.jar
antlr4-maven-plugin/4.5/antlr4-maven-plugin-4.5.jar
antlr4-maven-plugin/4.5.2-SNAPSHOT/antlr4-maven-plugin-4.5.2-SNAPSHOT.jar
antlr4-runtime/4.5/antlr4-runtime-4.5.jar
antlr4-runtime/4.5.2-SNAPSHOT/antlr4-runtime-4.5.2-SNAPSHOT.jar
antlr4-runtime-testsuite/4.5.2-SNAPSHOT/antlr4-runtime-testsuite-4.5.2-SNAPSHOT-tests.jar
antlr4-runtime-testsuite/4.5.2-SNAPSHOT/antlr4-runtime-testsuite-4.5.2-SNAPSHOT.jar
antlr4-tool-testsuite/4.5.2-SNAPSHOT/antlr4-tool-testsuite-4.5.2-SNAPSHOT.jar
antlr4-maven-plugin/4.6-SNAPSHOT/antlr4-maven-plugin-4.6-SNAPSHOT.jar
antlr4-runtime-test-annotation-processors/4.6-SNAPSHOT/antlr4-runtime-test-annotation-processors-4.6-SNAPSHOT.jar
antlr4-runtime-test-annotations/4.6-SNAPSHOT/antlr4-runtime-test-annotations-4.6-SNAPSHOT.jar
antlr4-runtime-testsuite/4.6-SNAPSHOT/antlr4-runtime-testsuite-4.6-SNAPSHOT-tests.jar
antlr4-runtime-testsuite/4.6-SNAPSHOT/antlr4-runtime-testsuite-4.6-SNAPSHOT.jar
antlr4-runtime/4.6-SNAPSHOT/antlr4-runtime-4.6-SNAPSHOT.jar
antlr4-tool-testsuite/4.6-SNAPSHOT/antlr4-tool-testsuite-4.6-SNAPSHOT.jar
antlr4/4.6-SNAPSHOT/antlr4-4.6-SNAPSHOT-tests.jar
antlr4/4.6-SNAPSHOT/antlr4-4.6-SNAPSHOT.jar
```
Note that ANTLR is written in itself, which is why maven downloads antlr4-4.5.jar for boostrapping 4.5.2-SNAPSHOT purposes.
Note that ANTLR is written in itself, which is why maven downloads antlr4-4.5.jar for boostrapping 4.6-SNAPSHOT purposes.
To build without running the tests (saves about 8 minutes), do this:
## Running test subsets
*From the `runtime-testsuite` dir*
### Run one test group across targets
```bash
$ cd runtime-testsuite
$ mvn -Dtest=TestParserExec test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.antlr.v4.test.runtime.cpp.TestParserExec
...
Tests run: 32, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.283 sec
Running org.antlr.v4.test.runtime.csharp.TestParserExec
...
```
Or run all lexer related tests:
```
$ mvn -Dtest=Test*Lexer* test
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.antlr.v4.test.runtime.cpp.TestCompositeLexers
...
```
### Run all tests for a single target
```bash
$ mvn -Dtest=java.* test
...
```
Or run all lexer related tests in Java target only:
```bash
$ mvn -Dtest=java.*Lexer* test
...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.antlr.v4.test.runtime.java.TestCompositeLexers
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.277 sec
Running org.antlr.v4.test.runtime.java.TestLexerErrors
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.376 sec
Running org.antlr.v4.test.runtime.java.TestLexerExec
Tests run: 38, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.07 sec
Running org.antlr.v4.test.runtime.java.TestSemPredEvalLexer
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.255 sec
Results :
Tests run: 59, Failures: 0, Errors: 0, Skipped: 0
```
## Testing in parallel
Use this to run tests in parallel:
```bash
$ mvn -Dparallel=methods -DthreadCount=4 test
...
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Concurrency config is parallel='methods', perCoreThreadCount=true, threadCount=4, useUnlimitedThreads=false
...
```
This can be combined with other `-D` above.
## Building without testing
To build without running the tests (saves a lot of time), do this:
```bash
mvn -DskipTests install
@ -122,4 +230,4 @@ mvn -DskipTests install
After download ANTLR source, just "import project from existing sources" and click on the "Maven Projects" tab in right gutter of IDE. It should build stuff in the background automatically and look like:
<img src=images/intellij-maven.png width=200>
<img src=images/intellij-maven.png width=200>

137
doc/cpp-target.md Normal file
View File

@ -0,0 +1,137 @@
# C++
The C++ target supports all platforms that can either run MS Visual Studio 2013 (or newer), XCode 7 (or newer) or CMake (C++11 required). All build tools can either create static or dynamic libraries, both as 64bit or 32bit arch. Additionally, XCode can create an iOS library.
## How to create a C++ lexer or parser?
This is pretty much the same as creating a Java lexer or parser, except you need to specify the language target, for example:
```
$ antlr4 -Dlanguage=Cpp MyGrammar.g4
```
You will see that there are a whole bunch of files generated by this call. If visitor or listener are not suppressed (which is the default) you'll get:
* MyGrammarLexer.h + MyGrammarLexer.cpp
* MyGrammarParser.h + MyGrammarParser.cpp
* MyGrammarVisitor.h + MyGrammarVisitor.cpp
* MyGrammarBaseVisitor.h + MyGrammarBaseVisitor.cpp
* MyGrammarListener.h + MyGrammarListener.cpp
* MyGrammarBaseListener.h + MyGrammarBaseListener.cpp
## Where can I get the runtime?
Once you've generated the lexer and/or parser code, you need to download or build the runtime. Prebuilt C++ runtime binaries for Windows (VS 2013 runtime), OSX and iOS are available on the ANTLR web site:
* http://www.antlr.org
Use CMake to build a Linux library (works also on OSX, however not for the iOS library). Building your own library on OSX or Windows is trivial, however. Just open the VS or XCode project, select target + arch and build it. Should work out of the box without any additional dependency.
## How do I run the generated lexer and/or parser?
Putting it all together to get a working parser is really easy. Look in the [runtime/Cpp/demo](../runtime/Cpp/demo) folder for a simple example. The [README](../runtime/Cpp/demo/README.md) there describes shortly how to build and run the demo on OSX, Windows or Linux.
## How do I create and run a custom listener?
The generation step above created a listener and base listener class for you. The listener class is an abstract interface, which declares enter and exit methods for each of your parser rules. The base listener implements all those abstract methods with an empty body, so you don't have to do it yourself if you just want to implement a single function. Hence use this base listener as the base class for your custom listener:
```c++
#include <iostream>
#include "antlr4-runtime.h"
#include "MyGrammarLexer.h"
#include "MyGrammarParser.h"
#include "MyGrammarBaseListener.h"
using namespace org::antlr::v4::runtime;
class TreeShapeListener : public MyGrammarBaseListener {
public:
void enterKey(Ref<ParserRuleContext> ctx) {
// Do something when entering the key rule.
}
};
int main(int argc, const char* argv[]) {
std::ifstream stream;
stream.open(argv[1]);
ANTLRInputStream input(stream);
MyGrammarLexer lexer(&input);
CommonTokenStream tokens(&lexer);
MyGrammarParser parser(&tokens);
Ref<tree::ParseTree> tree = parser.key();
Ref<TreeShapeListener> listener(new TreeShapeListener());
tree::ParseTreeWalker::DEFAULT->walk(listener, tree);
return 0;
}
```
This example assumes your grammar contains a parser rule named `key` for which the enterKey function was generated. The `Ref<>` template is an alias for `std::shared_ptr<>` to simplify the runtime source code which often makes use of smart pointers.
## Specialities of this ANTLR target
There are a couple of things that only the C++ ANTLR target has to deal with. They are described here.
### Memory Management
Since C++ has no built-in memory management we need to take extra care. For that we rely mostly on smart pointers, which however might cause time penalties or memory side effects (like cyclic references) if not used with care. Currently however the memory household looks very stable. Generally, when you see a raw pointer in code consider this as being managed elsewehere. You should never try to manage such a pointer (delete, assign to smart pointer etc.).
### Unicode Support
Encoding is mostly an input issue, i.e. when the lexer converts text input into lexer tokens. The parser is completely encoding unaware. However, lexer input in the grammar is defined by character ranges with either a single member (e.g. 'a' or [a] or [abc]), an explicit range (e.g. 'a'..'z' or [a-z]), the full Unicode range (for a wildcard) and the full Unicode range minus a sub range (for negated ranges, e.g. ~[a]). The explicit ranges (including single member ranges) are encoded in the serialized ATN by 16bit numbers, hence cannot reach beyond 0xFFFF (the Unicode BMP), while the implicit ranges can include any value (and hence support the full Unicode set, up to 0x10FFFF).
> An interesting side note here is that the Java target fully supports Unicode as well, despite the inherent limitations from the serialized ATN. That's possible because the Java String class represents characters beyond the BMP as surrogate pairs (two 16bit values) and even reads them as 2 separate input characters. To make this work a character range for an identifier in a grammar must include the surrogate pairs area (for a Java parser).
The C++ target however always expects UTF-8 input (either in a string or via a wide stream) which is then converted to UTF-32 (a char32_t array) and fed to the lexer. ANTLR, when parsing your grammar, limits character ranges explicitly to the BMP currently. So, in order to allow specifying the full Unicode set the C++ target uses a little trick: whenever an explicit character range includes the (unused) codepoint 0xFFFF in a grammar it is silently extended to the full Unicode range. It's clear that this is an all-or-nothing solution. You cannot define a subset of Unicode codepoints > 0xFFFF that way. This can only be solved if ANTLR supports larger character intervals.
The differences in handling characters beyond the BMP leads to a difference between Java and C++ lexers: the character offsets may not concur. This is because Java reads two 16bit values per Unicode char (if that falls into the surrogate area) while a C++ parser only reads one 32bit value. That usually doesn't have practical consequences, but might confuse people when comparing token positions.
### Named Actions
In order to help customizing the generated files there are a number of additional socalled **named actions**. These actions are tight to specific areas in the generated code and allow to add custom (target specific) code. All targets support these actions
* @parser::header
* @parser::members
* @lexer::header
* @lexer::members
(and their scopeless alternatives `@header` and `@members`) where header doesn't mean a C/C++ header file, but the top of a code file. The content of the header action appears in all generated files at the first line. So it's good for things like license/copyright information.
The content of a *members* action is placed in the public section of lexer or parser class declarations. Hence it can be used for public variables or predicate functions used in a grammar predicate. Since all targets support *header* + *members* they are the best place for stuff that should be available also in generated files for other languages.
In addition to that the C++ target supports many more such named actions. Unfortunately, it's not possible to define new scopes (e.g. *listener* in addition to *parser*) so they had to be defined as part of the existing scopes (*lexer* or *parser*). The grammar in the demo application contains all of the named actions as well for reference. Here's the list:
* **@lexer::preinclude** - Placed right before the first #include (e.g. good for headers that must appear first, for system headers etc.). Appears in both lexer h and cpp file.
* **@lexer::postinclude** - Placed right after the last #include, but before any class code (e.g. for additional namespaces). Appears in both lexer h and cpp file.
* **@lexer::context** - Placed right before the lexer class declaration. Use for e.g. additional types, aliases, forward declarations and the like. Appears in the lexer h file.
* **@lexer::declarations** - Placed in the private section of the lexer declaration (generated sections in all classes strictly follow the pattern: public, protected, privat, from top to bottom). Use this for private vars etc.
* **@lexer::definitions** - Placed before other implementations in the cpp file (but after *@postinclude*). Use this to implement e.g. private types.
For the parser there are the same actions as shown above for the lexer. In addition to that there are even more actions for visitor and listener classes:
* **@parser::listenerpreinclude**
* **@parser::listenerpostinclude**
* **@parser::listenerdeclarations**
* **@parser::listenermembers**
* **@parser::listenerdefinitions**
*
* **@parser::baselistenerpreinclude**
* **@parser::baselistenerpostinclude**
* **@parser::baselistenerdeclarations**
* **@parser::baselistenermembers**
* **@parser::baselistenerdefinitions**
*
* **@parser::visitorpreinclude**
* **@parser::visitorpostinclude**
* **@parser::visitordeclarations**
* **@parser::visitormembers**
* **@parser::visitordefinitions**
*
* **@parser::basevisitorpreinclude**
* **@parser::basevisitorpostinclude**
* **@parser::basevisitordeclarations**
* **@parser::basevisitormembers**
* **@parser::basevisitordefinitions**
and should be self explanatory now. Note: there is no *context* action for listeners or visitors, simply because they would be even less used than the other actions and there are so many already.

6
doc/creating-a-language-target.md
View File

@ -9,14 +9,14 @@ Creating a new target involves the following key elements:
1. For the tool, create class *X*Target as a subclass of class `Target` in package `org.antlr.v4.codegen.target`. This class describes language specific details about escape characters and strings and so on. There is very little to do here typically.
1. Create *X*.stg in directory tool/resources/org/antlr/v4/tool/templates/codegen/*X*/*X*.stg. This is a [StringTemplate](http://www.stringtemplate.org/) group file (`.stg`) that tells ANTLR how to express all of the parsing elements needed to generate code. You will see templates called `ParserFile`, `Parser`, `Lexer`, `CodeBlockForAlt`, `AltBlock`, etc... Each of these must be described how to build the indicated chunk of code. Your best bet is to find the closest existing target, copy that template file, and tweak to suit.
1. Create a runtime library to support the parsers generated by ANTLR. Under directory runtime/*X*, you are in complete control of the directory structure as dictated by common usage of that target language. For example, Java has: `runtime/Java/lib` and `runtime/Java/src` directories. Under `src`, you will find a directory structure for package `org.antlr.v4.runtime` and below.
1. Create a template file for runtime tests. All you have to do is provide a few simple templates that indicate how to print values and declare variables. Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code in a new target and check the results. All it needs to know is how to generate a test rig (i.e., a `main` program), how to define various class fields, compare members and so on. You must create a *X* directory underneath `runtime-testsuite/resources/org/antlr/v4/test/runtime`. Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
1. Create a template file for runtime tests. All you have to do is provide a few templates that indicate how to print values and declare variables. Our runtime test mechanism in dir `runtime-testsuite` will automatically generate code using these templates for each target and check the test results. It needs to know how to define various class fields, compare members and so on. You must create a *X*.test.stg file underneath [runtime-testsuite/resources/org/antlr/v4/test/runtime](https://github.com/antlr/antlr4/tree/master/runtime-testsuite/resources/org/antlr/v4/test/runtime). Again, your best bet is to copy the templates from the closest language to your target and tweak it to suit.
## Getting started
1. Fork the `antlr/antlr4` repository at github to your own user so that you have repository `username/antlr4`.
2. Clone `username/antlr4`, forked repository, to your local disk. Your remote `origin` will be the forked repository on GitHub. Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`). Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
2. Clone `username/antlr4`, the forked repository, to your local disk. Your remote `origin` will be the forked repository on GitHub. Add a remote `upstream` to the original `antlr/antlr4` repository (URL `https://github.com/antlr/antlr4.git`). Changes that you would like to contribute back to the project are done with [pull requests](https://help.github.com/articles/using-pull-requests/).
3. Try to build it before doing anything
```bash
$ mvn compile
```
That should proceed with success. See [Building ANTLR](building-antlr.md) for more details. (That link does not currently work as I have that documentation in a branch. see https://github.com/parrt/antlr4/blob/move-doc-to-repo/doc/building-antlr.md for now.)
That should proceed with success. See [Building ANTLR](building-antlr.md) for more details.

4
doc/faq/general.md
View File

@ -68,9 +68,9 @@ expr : expr '*' expr
;
```
ANTLR 4 automatically constructs parse trees for you and abstract syntax tree (AST) construction is no longer an option. See also What if I need ASTs not parse trees for a compiler, for example?
ANTLR 4 automatically constructs parse trees for you and abstract syntax tree (AST) construction is no longer an option. See also [What if I need ASTs not parse trees for a compiler, for example?](https://github.com/antlr/antlr4/blob/master/doc/faq/parse-trees.md#what-if-i-need-asts-not-parse-trees-for-a-compiler-for-example).
Another big difference is that we discourage the use of actions directly within the grammar because ANTLR 4 automatically generates [listeners and visitors](https://raw.githubusercontent.com/antlr/antlr4/master/doc/listeners.md) for you to use that trigger method calls when some phrases of interest are recognized during a tree walk after parsing. See also [Parse Tree Matching and XPath](https://raw.githubusercontent.com/antlr/antlr4/master/doc/tree-matching.md).
Another big difference is that we discourage the use of actions directly within the grammar because ANTLR 4 automatically generates [listeners and visitors](https://github.com/antlr/antlr4/blob/master/doc/listeners.md) for you to use that trigger method calls when some phrases of interest are recognized during a tree walk after parsing. See also [Parse Tree Matching and XPath](https://github.com/antlr/antlr4/blob/master/doc/tree-matching.md).
Semantic predicates are still allowed in both the parser and lexer rules as our actions. For efficiency sake keep semantic predicates to the right edge of lexical rules.

6
doc/getting-started.md
View File

@ -29,7 +29,7 @@ It's also a good idea to put this in your `.bash_profile` or whatever your start
3. Create aliases for the ANTLR Tool, and `TestRig`.
```
$ alias antlr4='java -Xmx500M -cp "/usr/local/lib/antlr-4.5-complete.jar:$CLASSPATH" org.antlr.v4.Tool'
$ alias grun='java org.antlr.v4.runtime.misc.TestRig'
$ alias grun='java org.antlr.v4.gui.TestRig'
```
### WINDOWS
@ -51,12 +51,12 @@ SET CLASSPATH=.;C:\Javalib\antlr-4.5-complete.jar;%CLASSPATH%
java org.antlr.v4.Tool %*
```
```
java org.antlr.v4.runtime.misc.TestRig %*
java org.antlr.v4.gui.TestRig %*
```
* Or, use doskey commands:
```
doskey antlr4=java org.antlr.v4.Tool $*
doskey grun =java org.antlr.v4.runtime.misc.TestRig $*
doskey grun =java org.antlr.v4.gui.TestRig $*
```
### Testing the installation

115
doc/go-target.md Normal file
View File

@ -0,0 +1,115 @@
# ANTLR4 Language Target, Runtime for Go
### First steps
#### 1. Install ANTLR4
[The getting started guide](getting-started.md) should get you started.
#### 2. Get the Go ANTLR runtime
Each target language for ANTLR has a runtime package for running parser generated by ANTLR4. The runtime provides a common set of tools for using your parser.
Get the runtime and install it on your GOPATH:
```bash
go get github.com/antlr/antlr4/runtime/Go/antlr
```
#### 3. Set the release tag (optional)
`go get` has no native way to specify a branch or commit. So, when you run it, you'll download the latest commits. This may or may not be your preference.
You'll need to use git to set the release. For example, to set the release tag for release 4.6.0:
```bash
cd $GOPATH/src/github.com/antlr/antlr4 # enter the antlr4 source directory
git checkout tags/4.6.0 # the go runtime was added in release 4.6.0
```
A complete list of releases can be found on [the release page](https://github.com/antlr/antlr4/releases).
#### 4. Generate your parser
You use the ANTLR4 "tool" to generate a parser. These will reference the ANTLR runtime, installed above.
Suppose you're using a UNIX system and have set up an alias for the ANTLR4 tool as described in [the getting started guide](getting-started.md). To generate your go parser, you'll need to invoke:
```bash
antlr4 -Dlanguage=Go MyGrammar.g4
```
For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
### Referencing the Go ANTLR runtime
You can reference the go ANTLR runtime package like this:
```go
import "github.com/antlr/antlr4/runtime/Go/antlr"
```
### Complete example
Suppose you're using the JSON grammar from https://github.com/antlr/grammars-v4/tree/master/json.
Then, invoke `antlr4 -Dlanguage=Go JSON.g4`. The result of this is a collection of .go files in the `parser` directory including:
```
json_parser.go
json_base_listener.go
json_lexer.go
json_listener.go
```
Another common option to the ANTLR tool is `-visitor`, which generates a parse tree visitor, but we won't be doing that here. For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
We'll write a small main func to call the generated parser/lexer (assuming they are separate). This one writes out the encountered `ParseTreeContext`'s. Suppose the gen'ed parser code is in the `parser` directory relative to this code:
```
package main
import (
"github.com/antlr/antlr4/runtime/Go/antlr"
"./parser"
"os"
"fmt"
)
type TreeShapeListener struct {
*parser.BaseJSONListener
}
func NewTreeShapeListener() *TreeShapeListener {
return new(TreeShapeListener)
}
func (this *TreeShapeListener) EnterEveryRule(ctx antlr.ParserRuleContext) {
fmt.Println(ctx.GetText())
}
func main() {
input := antlr.NewFileStream(os.Args[1])
lexer := parser.NewJSONLexer(input)
stream := antlr.NewCommonTokenStream(lexer,0)
p := parser.NewJSONParser(stream)
p.AddErrorListener(antlr.NewDiagnosticErrorListener(true))
p.BuildParseTrees = true
tree := p.Json()
antlr.ParseTreeWalkerDefault.Walk(NewTreeShapeListener(), tree)
}
```
This one expects the input to be passed on the command line:
```
go run test.go input
```
The output is:
```
{"a":1}
{"a":1}
"a":1
1
```

BIN
doc/images/python3-tests.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 228 KiB

BIN
doc/images/testrigs.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 139 KiB

2
doc/index.md
View File

@ -51,6 +51,8 @@ This documentation is a reference and summarizes grammar syntax and the key sema
* [Runtime Libraries and Code Generation Targets](targets.md)
* [Parsing binary streams](parsing-binary-files.md)
* [Parser and lexer interpreters](interpreters.md)
* [Resources](resources.md)

2
doc/javascript-target.md
View File

@ -70,7 +70,7 @@ Right now, there is no npm package available, so you need to register a link ins
$ npm link antlr4
```
This will install antlr4 using the package.son descriptor that comes with the script.
This will install antlr4 using the package.json descriptor that comes with the script.
## How do I run the generated lexer and/or parser?

13
doc/parser-rules.md
View File

@ -380,8 +380,17 @@ The attributes defined within those [...] can be used like any other variable. H
add[int x] returns [int result] : '+=' INT {$result = $x + $INT.int;} ;
```
As with the grammar level, you can specify rule-level named actions. For rules, the valid names are init and after. As the names imply, parsers execute init actions immediately before trying to match the associated rule and execute after actions immediately after matching the rule. ANTLR after actions do not execute as part of the finally code block of the generated rule function. Use the ANTLR finally action to place code in the generated rule function finally code block.
The actions come after any argument, return value, or local attribute definition actions. The row rule preamble from Section 10.2, Accessing Token and Rule Attributes illustrates the syntax nicely:
The args, locals, and return `[...]` are generally in the target language but with some constraints. The `[...]` string is a comma-separated list of declarations either with prefix or postfix type notation or no-type notation. The elements can have initializer such as `[int x = 32, float y]` but don't go too crazy as we are parsing this generic text manually in [ScopeParser](https://github.com/antlr/antlr4/blob/master/tool/src/org/antlr/v4/parse/ScopeParser.java).
* Java, CSharp, C++ use `int x` notation but C++ must use a slightly altered notation for array references, `int[] x`, to fit in the *type* *id* syntax.
* Go and Swift give the type after the variable name, but Swift requires a `:` in between. Go `i int`, Swift `i:int`. For Go target, you must either use `int i` or `i:int`.
* Python and JavaScript don't specify static types so actions are just identifier lists such as `[i,j]`.
Technically any target could use either notation. For examples, see [TestScopeParsing](https://github.com/antlr/antlr4/blob/master/tool-testsuite/test/org/antlr/v4/test/tool/TestScopeParsing.java).
As with the grammar level, you can specify rule-level named actions. For rules, the valid names are `init` and `after`. As the names imply, parsers execute init actions immediately before trying to match the associated rule and execute after actions immediately after matching the rule. ANTLR after actions do not execute as part of the finally code block of the generated rule function. Use the ANTLR finally action to place code in the generated rule function finally code block.
The actions come after any argument, return value, or local attribute definition actions. The `row` rule preamble from Section 10.2, Accessing Token and Rule Attributes illustrates the syntax nicely:
actions/CSV.g4
```

182
doc/parsing-binary-files.md Normal file
View File

@ -0,0 +1,182 @@
# Parsing Binary Files
Parsing binary files is no different than parsing character-based files except that the "characters" are actually bytes not 16-bit unsigned short unicode characters. From a lexer/parser point of view, there is no difference except that the characters are likely not printable. If you want to match a special 2-byte marker 0xCA then 0xFE, the following rule is sufficient.
```
MARKER : '\u00CA' '\u00FE' ;
```
The parser of course would refer to that token like any other token.
Here is a sample grammar for use with the code snippets below.
```
grammar IP;
file : ip+ (MARKER ip)* ;
ip : BYTE '.' BYTE '.' BYTE '.' BYTE ;
MARKER : '\u00CA' '\u00FE' ;
BYTE : '\u0000'..'\u00FF' ;
```
Notice that `BYTE` is using a range operator to match anything between 0 and 255. We can't use character classes like `[a-z]` naturally because we are not parsing character codes. All character specifiers must have `00` as their upper byte. E.g., `\uCAFE` is not a valid character because that 16-bit value will never be created from the input stream (bytes only remember).
If there are actual characters like `$` or `!` encoded as bytes in the binary file, you can refer to them via literals like `'$'` as you normally would. See `'.'` in the grammar.
## Binary streams
There are many targets now so I'm not sure exactly how they process text files but most targets will pull in text per the machine's locale. Much of the time this will mean UTF-8 encoding of text converted to 16-bit Unicode. ANTLR's lexers operate on `int` so we can handle any kind of character you want to send in that fits in `int`.
Once the lexer gets an input stream, it doesn't care whether the characters come from / represent bytes or actual Unicode characters.
Let's get a binary file called `ips` and put it in our resources directory:
```java
public class WriteBinaryFile {
public static final byte[] bytes = {
(byte)172, 0, 0, 1, (byte)0xCA, (byte)0xFE,
(byte)10, 10, 10, 1, (byte)0xCA, (byte)0xFE,
(byte)10, 10, 10, 99
};
public static void main(String[] args) throws IOException {
Files.write(new File("resources/ips").toPath(), bytes);
}
}
```
Now we need to create a stream of bytes satisfactory to ANTLR, which is as simple as:
```java
ANTLRFileStream bytesAsChar = new ANTLRFileStream("resources/ips", "ISO-8859-1");
```
The `ISO-8859-1` encoding is just the 8-bit char encoding for LATIN-1, which effectively tells the stream to treat each byte as a character. That's what we want. Then we have the usual test rig:
```java
ANTLRFileStream bytesAsChar = new ANTLRFileStream("resources/ips", "ISO-8859-1");
IPLexer lexer = new IPLexer(bytesAsChar);
CommonTokenStream tokens = new CommonTokenStream(lexer);
IPParser parser = new IPParser(tokens);
ParseTree tree = parser.file();
IPBaseListener listener = new MyIPListener();
ParseTreeWalker.DEFAULT.walk(listener, tree);
```
Here is the listener:
```java
class MyIPListener extends IPBaseListener {
@Override
public void exitIp(IPParser.IpContext ctx) {
List<TerminalNode> octets = ctx.BYTE();
short[] ip = new short[4];
for (int i = 0; i<octets.size(); i++) {
String oneCharStringHoldingOctet = octets.get(i).getText();
ip[i] = (short)oneCharStringHoldingOctet.charAt(0);
}
System.out.println(Arrays.toString(ip));
}
}
```
We can't just print out the text because we are not reading in text. We need to emit each byte as a decimal value. The output should be the following when you run the test code:
```
[172, 0, 0, 1]
[10, 10, 10, 1]
[10, 10, 10, 99]
```
## Custom stream
If you want to play around with the stream, you can. Here's an example that alters how "text" is computed from the byte stream (which changes how tokens print out their text as well):
```java
/** make a stream treating file as full of single unsigned byte characters */
class BinaryANTLRFileStream extends ANTLRFileStream {
public BinaryANTLRFileStream(String fileName) throws IOException {
super(fileName, "ISO-8859-1");
}
/** Print the decimal value rather than treat as char */
@Override
public String getText(Interval interval) {
StringBuilder buf = new StringBuilder();
int start = interval.a;
int stop = interval.b;
if(stop >= this.n) {
stop = this.n - 1;
}
for (int i = start; i<=stop; i++) {
int v = data[i];
buf.append(v);
}
return buf.toString();
}
}
```
The new test code starts out like this:
```java
ANTLRFileStream bytesAsChar = new BinaryANTLRFileStream("resources/ips");
IPLexer lexer = new IPLexer(bytesAsChar);
...
```
This simplifies our listener then:
```java
class MyIPListenerCustomStream extends IPBaseListener {
@Override
public void exitIp(IPParser.IpContext ctx) {
List<TerminalNode> octets = ctx.BYTE();
System.out.println(octets);
}
}
```
You should get this enhanced output:
```
[172(0xAC), 0(0x0), 0(0x0), 1(0x1)]
[10(0xA), 10(0xA), 10(0xA), 1(0x1)]
[10(0xA), 10(0xA), 10(0xA), 99(0x63)]
```
## Error handling in binary files
Error handling proceeds exactly like any other parser. For example, let's alter the binary file so that it is missing one of the 0's in the first IP address:
```java
public static final byte[] bytes = {
(byte)172, '.', 0, '.', '.', 1, (byte)0xCA, (byte)0xFE, // OOOPS
(byte)10, '.', 10, '.', 10, '.', 1, (byte)0xCA, (byte)0xFE,
(byte)10, '.', 10, '.', 10, '.', 99
};
```
Running the original test case gives us:
```
line 1:4 extraneous input '.' expecting BYTE
line 1:6 mismatched input 'Êþ' expecting '.'
[172, 0, 1, 0]
[10, 10, 10, 1]
[10, 10, 10, 99]
```
That `'Êþ'` is just to the character representation of two bytes 0xCA and 0xFE. Using the enhanced binary stream, we see:
```
line 1:4 extraneous input '46(0x2E)' expecting BYTE
line 1:6 mismatched input '202(0xCA)254(0xFE)' expecting '.'
[172(0xAC), 0(0x0), 1(0x1)]
[10(0xA), 10(0xA), 10(0xA), 1(0x1)]
[10(0xA), 10(0xA), 10(0xA), 99(0x63)]
```

7
doc/python-target.md
View File

@ -32,7 +32,7 @@ We will not document here how to refer to the runtime from your Python project,
## How do I run the generated lexer and/or parser?
Let's suppose that your grammar is named, as above, "MyGrammar". Let's suppose this parser comprises a rule named "StartRule". The tool will have generated for you the following files:
Let's suppose that your grammar is named, as above, "MyGrammar". Let's suppose this parser comprises a rule named "startRule". The tool will have generated for you the following files:
* MyGrammarLexer.py
* MyGrammarParser.py
@ -44,6 +44,7 @@ Let's suppose that your grammar is named, as above, "MyGrammar". Let's suppose t
Now a fully functioning script might look like the following:
```python
import sys
from antlr4 import *
from MyGrammarLexer import MyGrammarLexer
from MyGrammarParser import MyGrammarParser
@ -53,7 +54,7 @@ def main(argv):
lexer = MyGrammarLexer(input)
stream = CommonTokenStream(lexer)
parser = MyGrammarParser(stream)
tree = parser.StartRule()
tree = parser.startRule()
if __name__ == '__main__':
main(sys.argv)
@ -95,7 +96,7 @@ In order to execute this listener, you would simply add the following lines to t
```
...
tree = parser.StartRule() - only repeated here for reference
tree = parser.startRule() - only repeated here for reference
printer = KeyPrinter()
walker = ParseTreeWalker()
walker.walk(printer, tree)

51
doc/releasing-antlr.md
View File

@ -24,11 +24,14 @@ Edit the repository looking for 4.5 or whatever and update it. Bump version in t
* runtime/CSharp/runtime/CSharp/Antlr4.Runtime/Properties/AssemblyInfo.cs
* runtime/JavaScript/src/antlr4/package.json
* runtime/JavaScript/src/antlr4/Recognizer.js
* tool/src/org/antlr/v4/codegen/target/CppTarget.java
* tool/src/org/antlr/v4/codegen/target/CSharpTarget.java
* tool/src/org/antlr/v4/codegen/target/JavaScriptTarget.java
* tool/src/org/antlr/v4/codegen/target/Python2Target.java
* tool/src/org/antlr/v4/codegen/target/Python3Target.java
* runtime/Cpp/VERSION
* runtime/Cpp/RuntimeMetaData.cpp
Here is a simple script to display any line from the critical files with, say, `4.5` in it:
```bash
@ -240,6 +243,52 @@ python setup.py sdist bdist_wininst upload -r pypi
Add links to the artifacts from download.html
### C++
The C++ target is the most complex one, because it addresses multiple platforms, which require individual handling. We have 4 scenarios to cover:
* **Windows**: static and dynamic libraries for the VC++ runtime 2013 or 2015 (corresponding to Visual Studio 2013 or 2015) + header files. All that in 32 and 64bit, debug + release.
* **MacOS**: static and dynamic release libraries + header files.
* **iOS**: no prebuilt binaries, but just a zip of the source, including the XCode project to build everything from source.
* **Linux**: no prebuilt binaries, but just a zip of the source code, including the cmake file to build everything from source there.
In theory we could also create a library for iOS, but that requires to sign it, which depends on an active iOS developer account. So we leave this up to the ANTLR user to build the iOS lib, like we do for Linux builds.
For each platform there's a deployment script which generates zip archives and copies them to the target folder. The Windows deployment script must be run on a machine with VS 2013 + VS 2015 installed. The Mac script must be run on a machine with XCode 7+ installed. The source script can be executed on any Linux or Mac box.
On a Mac (with XCode 7+ installed):
```bash
cd runtime/Cpp
./deploy-macos.sh
```
On any Mac or Linux machine:
```bash
cd runtime/Cpp
./deploy-source.sh
```
On a Windows machine the build scripts checks if VS 2013 and/or VS 2015 are installed and builds binaries for each, if found. This script requires 7z to be installed (http://7-zip.org).
```bash
cd runtime/Cpp
deploy-windows.cmd
```
Move target to website (**_rename to a specific ANTLR version first if needed_**):
```bash
pushd ~/antlr/sites/website-antlr4/download
git add antlr4cpp-runtime-macos.zip
git add antlr4cpp-runtime-windows.zip
git add antlr4cpp-runtime-source.zip
git commit -a -m 'update C++ runtime'
git push origin gh-pages
popd
```
## Update javadoc for runtime and tool
First gen javadoc:

41
doc/runtimetests-overview.md Normal file
View File

@ -0,0 +1,41 @@
# Overview of the ANTLR Runtime Test Suite
An important part of ANTLR4 is its runtime test suite, which consist of 2 subparts:
* Tests for the tool itself
* Tests for the ANTLR runtime
Usually the tests are executed while compiling and installing an ANTLR4 jar from source code. The command for that is simply:
```bash
$ mvn install
```
to be executed in the root of the ANTLR4 repository. More details about this can be found in [Building ANTLR](building-antlr.md).
However, you don't need to run the installation again and again just to run the tests. Instead use
```bash
$ mvn test
```
to only trigger testing. You can find all runtime tests in the [runtime-testsuite/resources/org/antlr/v4/test/runtime](../runtime-testsuite/resources/org/antlr/v4/test/runtime) subfolder (tool tests under [tool-testsuite/test/org/antlr/v4/test/tool](../tool-testsuite/test/org/antlr/v4/test/tool)). The tool tests are just a bunch of Java test cases that test the tool's internal behavior (e.g. for code generation). We focus on the runtime tests here.
The underlying process of running the tests is quite a complicated setup to cater especially for a flexible test specification that can run with different target runtimes. Everything runs in Java, except for the actual target runtime tests. Runtime tests run first, followed by the tool tests. If there was a test failure in the first step, the tool tests are not executed, however. These are the steps involved when running the runtime tests:
* Generate Java JUnit test cases from the test templates, once for each target (C++, C#, Python, Java, Javascript atm.).
* These test cases generate grammar files when executed and run the target specific parser generation step, including compiling a binary, if necessary (e.g. for C++ and C#).
* Finally run the compiled test module using the input specified in the test template. The output (usually a token or parse tree dump) is then compared against the expected output, specified in the test template as well. This also includes any error messages written to the console.
## Generating JUnit Tests
The test specification part makes heavy use of the StringTemplate engine to allow defining target language agnostic tests. For that all tests are described in template (`stg`) files. You can find them in the [templates](../runtime-testsuite/resources/org/antlr/v4/test/runtime/templates) subfolder of the runtime tests folder. Read more about the folder structure in the [adding-tests.md](adding-tests.md) file. As lined out there you have to run
```bash
$ mvn -Pgen generate-test-sources
```
everytime you change any of the test templates or your target language specific template (which is used to translate certain text to your specific language). And a small hint: this command can be executed from the ANTLR source root as well. No need to dig into a subfolder.
## Running the Generated Tests
After generation you can run the tests as written above (`mvn install` or `mvn test`, both recompile ANTLR if necessary), which takes about 40 minutes for the full set (of which 30 mins are alone consumed by the C++ target tests). Which tests actually run is controlled by the [runtime tests pom.xml file](../runtime-testsuite/pom.xml). Look for the `maven-surefire-plugin` plugin entry and especially its includes. If you ever want to run tests only for a specific target, comment out all other `<include>` elements. For a specific test change the wildcard to that specific test name. This is especially helpful when debugging a test (e.g. when it fails) or when creating/changing tests. Additionally, some targets require to install additional dependencies you may not want to add to your box (e.g. mono, python 3.5) just to run e.g. the Java or C++ tests.

113
doc/swift-target.md Normal file
View File

@ -0,0 +1,113 @@
# ANTLR4 Language Target, Runtime for Swift
### Usage
#### 1. Install ANTLR4
[The getting started guide](getting-started.md) should get you started.
#### 2. create a Swift lexer or parser
This is pretty much the same as creating a Java lexer or parser, except you need to specify the language target, for example:
```
$ antlr4 -Dlanguage=Swift MyGrammar.g4
```
For a full list of antlr4 tool options, please visit the [tool documentation page](tool-options.md).
#### 3. Get the Swift ANTLR runtime
You will find Swift runtime (framework project) in
```
antlr4/runtime/Swift
```
#### 4. Example
The example from [here](https://github.com/janyou/Antlr-Swift-Runtime/tree/master/Test)
(1). Hello.g4
```
grammar Hello;
r : 'hello' ID ;
ID : [a-z]+ ;
WS : [ \t\r\n]+ -> skip ;
```
(2). generate lexer/parser/visitor from Hello.g4 file
```
$ antlr4 -Dlanguage=Swift -visitor -o gen Hello.g4
```
in gen folder：
```
Hello.tokens
HelloBaseListener.swift
HelloBaseVisitor.swift
HelloLexer.swift
HelloLexer.tokens
HelloLexerATN.swift
HelloListener.swift
HelloParser.swift
HelloParserATN.swift
HelloVisitor.swift
```
(3). make a custom listener
```
public class HelloWalker: HelloBaseListener{
public override func enterR(_ ctx: HelloParser.RContext) {
print( "enterR: " + ((ctx.ID()?.getText()) ?? ""))
}
public override func exitR(_ ctx: HelloParser.RContext) {
print( "exitR ")
}
}
```
(4). call and run
add TestHello.txt
```
hello world
```
run:
```
import Antlr4
....
do {
let textFileName = "TestHello.txt"
if let textFilePath = Bundle.main.path(forResource: textFileName, ofType: nil) {
let lexer = HelloLexer(ANTLRFileStream(textFilePath))
let tokens = CommonTokenStream(lexer)
let parser = try HelloParser(tokens)
let tree = try parser.r()
let walker = ParseTreeWalker()
try walker.walk(HelloWalker(),tree)
} else {
print("error occur: can not open \(textFileName)")
}
}catch ANTLRException.cannotInvokeStartRule {
print("error occur: CannotInvokeStartRule")
}catch ANTLRException.recognition(let e ) {
print("error occur\(e)")
}catch {
print("error occur")
}
```

11
doc/targets.md
View File

@ -2,16 +2,13 @@
This page lists the available and upcoming ANTLR runtimes. Please note that you won't find here language specific code generators. This is because there is only one tool, written in Java, which is able to generate lexer and parser code for all targets, through command line options. The tool can be invoked from the command line, or any integration plugin to popular IDEs and build systems: Eclipse, IntelliJ, Visual Studio, Maven. So whatever your environment and target is, you should be able to run the tool and produce the code in the targeted language. As of writing, the available targets are the following:
* [Java](java-target.md)<br>
The [ANTLR v4 book](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference) has a decent summary of the runtime library. We have added a useful XPath feature since the book was printed that lets you select bits of parse trees.
<br>[Runtime API](http://www.antlr.org/api/Java/index.html)
<br>See [Getting Started with ANTLR v4](getting-started.md)
* [Java](java-target.md). The [ANTLR v4 book](http://pragprog.com/book/tpantlr2/the-definitive-antlr-4-reference) has a decent summary of the runtime library. We have added a useful XPath feature since the book was printed that lets you select bits of parse trees. See [Runtime API](http://www.antlr.org/api/Java/index.html) and [Getting Started with ANTLR v4](getting-started.md)
* [C#](csharp-target.md)
* [Python](python-target.md) (2 and 3)
* [JavaScript](javascript-target.md)
* Swift (not yet available)
* C++ (not yet available)
* [Go](go-target.md)
* [C++](cpp-target.md)
* [Swift](swift-target.md)
## Target feature parity

21
pom.xml
View File

@ -7,7 +7,7 @@
</parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.5.4-SNAPSHOT</version>
<version>4.6-SNAPSHOT</version>
<packaging>pom</packaging>
<name>ANTLR 4</name>
@ -48,6 +48,12 @@
<role>Developer - JavaScript, C#, Python 2, Python 3</role>
</roles>
</developer>
<developer>
<name>Peter Boyer</name>
<roles>
<role>Developer - Go</role>
</roles>
</developer>
<developer>
<name>Jim Idle</name>
<email>jimi@idle.ws</email>
@ -56,6 +62,12 @@
<role>Developer - Maven Plugin</role>
</roles>
</developer>
<developer>
<name>Mike Lischke</name>
<roles>
<role>Developer - C++ Target</role>
</roles>
</developer>
</developers>
<modules>
@ -63,6 +75,9 @@
<module>tool</module>
<module>antlr4-maven-plugin</module>
<module>tool-testsuite</module>
<module>runtime-testsuite/annotations</module>
<module>runtime-testsuite/processors</module>
<!-- <module>runtime-testsuite-legacy</module> -->
<module>runtime-testsuite</module>
</modules>
@ -70,8 +85,8 @@
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<antlr.testinprocess>true</antlr.testinprocess>
<maven.compiler.source>1.6</maven.compiler.source>
<maven.compiler.target>1.6</maven.compiler.target>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<mailingLists>

114
runtime-testsuite-legacy/pom.xml Normal file
View File

@ -0,0 +1,114 @@
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.antlr</groupId>
<artifactId>antlr4-master</artifactId>
<version>4.6-SNAPSHOT</version>
</parent>
<artifactId>antlr4-runtime-testsuite-legacy</artifactId>
<name>ANTLR 4 Legacy Runtime Test Generator</name>
<description>Legacy collection of tests for ANTLR 4 Runtime libraries.</description>
<prerequisites>
<maven>3.0</maven>
</prerequisites>
<inceptionYear>2009</inceptionYear>
<dependencies>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>ST4</artifactId>
<version>4.0.8</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>org.antlr</groupId>
<artifactId>antlr4-runtime</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.seleniumhq.selenium</groupId>
<artifactId>selenium-java</artifactId>
<version>2.46.0</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.eclipse.jetty</groupId>
<artifactId>jetty-server</artifactId>
<version>8.1.16.v20140903</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<sourceDirectory>src</sourceDirectory>
<!--
<testSourceDirectory>test</testSourceDirectory>
-->
<resources>
<resource>
<directory>resources</directory>
</resource>
<resource>
<directory>../runtime</directory>
</resource>
</resources>
</build>
<profiles>
<profile>
<id>gen</id>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>exec-maven-plugin</artifactId>
<version>1.4.0</version>
<executions>
<execution>
<phase>generate-test-sources</phase>
<goals>
<goal>java</goal>
</goals>
<configuration>
<mainClass>org.antlr.v4.testgen.TestGenerator</mainClass>
<arguments>
<argument>${basedir}/..</argument>
</arguments>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
<profile>
<id>tests</id>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/Test*.java</include>
</includes>
</configuration>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

111
runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/TestInfo.stg Normal file
View File

@ -0,0 +1,111 @@
file(className, descriptors) ::= <<
package org.antlr.v4.test.runtime.xxx.tests.junk;
import org.antlr.v4.runtime.misc.Pair;
import org.antlr.v4.test.runtime.CommentHasStringValue;
import org.antlr.v4.test.runtime.xxx.*;
import org.antlr.v4.test.runtime.xxx.tests.*;
import java.util.ArrayList;
import java.util.List;
public class <className> {
<descriptors; separator="\n\n">
}
>>
rig(targetName, groupName, go, subdir) ::= <<
package org.antlr.v4.test.runtime.xxx.tests.<subdir:{s|<s>.}><targetName; format="lower">;
import org.antlr.v4.test.runtime.xxx.RuntimeTestDescriptor;
import org.antlr.v4.test.runtime.xxx.XXXBaseTest;
import org.antlr.v4.test.runtime.xxx.descriptors.*;
import org.junit.*;
import org.junit.runner.*;
import org.junit.runners.Parameterized;
@RunWith(Parameterized.class)
public class Test<groupName> extends XXXBaseTest {
public Test<groupName>(RuntimeTestDescriptor descriptor) {
super(descriptor,new Base<targetName>Test());
}
<if(go)>
@BeforeClass
public static void groupSetUp() throws Exception { BaseGoTest.groupSetUp(); }
@AfterClass
public static void groupTearDown() throws Exception { BaseGoTest.groupTearDown(); }
<endif>
@Parameterized.Parameters(name="{0}")
public static RuntimeTestDescriptor[] getAllTestDescriptors() {
return XXXBaseTest.getRuntimeTestDescriptors(<groupName>Descriptors.class, "<targetName>");
}
}
>>
descriptor(testName, TestType, Input, LongInput, Errors, LongErrors, Rule, Output, LongOutput,
grammar, slaveGrammars, grammarName, LongAfterGrammar) ::= <<
public static class <testName> extends Base<TestType>TestDescriptor {
<if(LongInput)>
/**
<LongInput>
*/
@CommentHasStringValue
public String input;<\n>
<else>
public String input = "<Input>";
<endif>
<if(LongOutput)>
/**
<LongOutput>
*/
@CommentHasStringValue
public String output;<\n>
<else>
public String output = "<Output>";
<endif>
<if(LongErrors)>
/**
<LongErrors>
*/
@CommentHasStringValue
public String errors;<\n>
<else>
public String errors = "<Errors>";
<endif>
public String startRule = "<Rule>";
public String grammarName = "<grammarName>";
<if(LongAfterGrammar)>
/**
<LongAfterGrammar>
*/
@CommentHasStringValue
public String afterGrammar;
<endif>
/**
<grammar>
*/
@CommentHasStringValue
public String grammar;
<if(slaveGrammars)>
<slaveGrammars:{sg |
/**
<sg.grammar>
*/
@CommentHasStringValue
public String slaveGrammar<sg.name>;
}>
@Override
public List\<Pair\<String, String>\> getSlaveGrammars() {
List\<Pair\<String,String>\> slaves = new ArrayList\<Pair\<String, String>\>();
<slaveGrammars:{sg | slaves.add(new Pair\<String, String>("<sg.name>",slaveGrammar<sg.name>));<\n>}>
return slaves;
\}
<endif>
}
>>

280
runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/cpp/Cpp.test.stg Normal file
View File

@ -0,0 +1,280 @@
writeln(s) ::= "std::cout \<\< <s> \<\< std::endl;"
write(s) ::= "std::cout \<\< <s>;"
writeList(s) ::= << std::cout \<\< <s; separator=" \<\< "> \<\< std::endl;>>
False() ::= "false"
True() ::= "true"
Not(v) ::= "!<v>"
Assert(s) ::= ""
Cast(t,v) ::= "dynamic_cast\<<t> *>(<v>)" // Should actually use a more specific name. We may have to use other casts as well.
Append(a,b) ::= "<a> + <b>->toString()"
Concat(a,b) ::= "<a><b>"
DeclareLocal(s,v) ::= "<s> = <v>"
AssertIsList(v) ::= "assert(<v>.size() >= 0);" // Use a method that exists only on a list (vector actually).
AssignLocal(s,v) ::= "<s> = <v>;"
InitIntMember(n,v) ::= "int <n> = <v>;"
InitBooleanMember(n,v) ::= "bool <n> = <v>;"
GetMember(n) ::= "<n>"
SetMember(n,v) ::= "<n> = <v>;"
AddMember(n,v) ::= "<n> += <v>;"
PlusMember(v,n) ::= "<v> + <n>"
MemberEquals(n,v) ::= "<n> == <v>"
ModMemberEquals(n,m,v) ::= "<n> % <m> == <v>"
ModMemberNotEquals(n,m,v) ::= "<n> % <m> != <v>"
DumpDFA() ::= "dumpDFA();"
Pass() ::= "/* do nothing */"
StringList() ::= "std::vector\<std::string>"
BuildParseTrees() ::= "setBuildParseTree(true);"
BailErrorStrategy() ::= "_errHandler = std::make_shared\<BailErrorStrategy>();"
ToStringTree(s) ::= "<s>->toStringTree(this)"
Column() ::= "getCharPositionInLine()"
Text() ::= "getText()"
ValEquals(a,b) ::= "<a> == <b>"
TextEquals(a) ::= "getText() == \"<a>\""
PlusText(a) ::="\"<a>\" + getText()"
InputText() ::= "_input->getText()"
LTEquals(i, v) ::= "_input->LT(<i>)->getText() == <v>"
LANotEquals(i, v) ::= "_input->LA(<i>) != <v>"
TokenStartColumnEquals(i) ::= "tokenStartCharPositionInLine == <i>"
ImportListener(X) ::= ""
GetExpectedTokenNames() ::= "getExpectedTokens().toString(_tokenNames)"
RuleInvocationStack() ::= "Arrays::listToString(getRuleInvocationStack(), \", \")"
LL_EXACT_AMBIG_DETECTION() ::= <<getInterpreter\<atn::ParserATNSimulator>()->setPredictionMode(atn::PredictionMode::LL_EXACT_AMBIG_DETECTION);>>
ParserToken(parser, token) ::= <%<parser>::<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
bool Property() {
return true;
}
}
>>
ParserPropertyCall(p, call) ::= "<call>"
PositionAdjustingLexer() ::= <<
protected:
class PositionAdjustingLexerATNSimulator : public atn::LexerATNSimulator {
public:
PositionAdjustingLexerATNSimulator(Lexer *recog, const atn::ATN &atn, std::vector\<dfa::DFA> &decisionToDFA,
atn::PredictionContextCache &sharedContextCache)
: atn::LexerATNSimulator(recog, atn, decisionToDFA, sharedContextCache) {
}
void resetAcceptPosition(CharStream *input, int index, int line, int charPositionInLine) {
input->seek(index);
_line = line;
_charPositionInLine = charPositionInLine;
consume(input);
}
};
public:
virtual std::unique_ptr\<Token> nextToken() override {
if (dynamic_cast\<PositionAdjustingLexerATNSimulator *>(_interpreter) == nullptr) {
delete _interpreter;
_interpreter = new PositionAdjustingLexerATNSimulator(this, _atn, _decisionToDFA, _sharedContextCache);
}
return Lexer::nextToken();
}
virtual Token* emit() override {
switch (type) {
case TOKENS:
handleAcceptPositionForKeyword("tokens");
break;
case LABEL:
handleAcceptPositionForIdentifier();
break;
default:
break;
}
return Lexer::emit();
}
private:
bool handleAcceptPositionForIdentifier() {
std::string tokenText = getText();
int identifierLength = 0;
while (identifierLength \< tokenText.length() && isIdentifierChar(tokenText[identifierLength])) {
identifierLength++;
}
if (getInputStream()->index() > tokenStartCharIndex + identifierLength) {
int offset = identifierLength - 1;
getInterpreter\<PositionAdjustingLexerATNSimulator>()->resetAcceptPosition(getInputStream(),
tokenStartCharIndex + offset, tokenStartLine, tokenStartCharPositionInLine + offset);
return true;
}
return false;
}
bool handleAcceptPositionForKeyword(const std::string &keyword) {
if (getInputStream()->index() > tokenStartCharIndex + keyword.length()) {
long offset = keyword.size() - 1;
getInterpreter\<PositionAdjustingLexerATNSimulator>()->resetAcceptPosition(getInputStream(),
tokenStartCharIndex + offset, tokenStartLine, tokenStartCharPositionInLine + offset);
return true;
}
return false;
}
static bool isIdentifierChar(char c) {
return std::isalnum(c) || c == '_';
}
public:
>>
BasicListener(X) ::= <<
@parser::definitions {
#include "TBaseListener.h"
class LeafListener : public TBaseListener {
public:
virtual void visitTerminal(tree::TerminalNode *node) override {
std::cout \<\< node->getSymbol()->getText() \<\< std::endl;
}
};
}
>>
WalkListener(s) ::= <<
LeafListener listener;
tree::ParseTreeWalker::DEFAULT.walk(&listener, <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
class MyRuleNode : public ParserRuleContext {
public:
size_t altNum;
MyRuleNode(ParserRuleContext *parent, int invokingStateNumber)
: ParserRuleContext(parent, invokingStateNumber) {
}
virtual size_t getAltNumber() const override { return altNum; }
virtual void setAltNumber(size_t altNum) override { this->altNum = altNum; }
};
}
>>
TokenGetterListener(X) ::= <<
@parser::definitions {
#include "TBaseListener.h"
class LeafListener : public TBaseListener {
public:
virtual void exitA(TParser::AContext *ctx) override {
if (ctx->children.size() == 2)
std::cout \<\< ctx->INT(0)->getSymbol()->getText() \<\< " " \<\< ctx->INT(1)->getSymbol()->getText()
\<\< " " \<\< Arrays::toString(ctx->INT()) \<\< std::endl;
else
std::cout \<\< ctx->ID()->getSymbol()->toString() \<\< std::endl;
}
};
}
>>
RuleGetterListener(X) ::= <<
@parser::definitions {
#include "TBaseListener.h"
class LeafListener : public TBaseListener {
public:
virtual void exitA(TParser::AContext *ctx) override {
if (ctx->children.size() == 2) {
std::cout \<\< ctx->b(0)->start->getText() \<\< " " \<\< ctx->b(1)->start->getText() \<\< " " \<\< ctx->b()[0]->start->getText() \<\< std::endl;
} else {
std::cout \<\< ctx->b(0)->start->getText() \<\< std::endl;
}
}
};
}
>>
LRListener(X) ::= <<
@parser::definitions {
#include "TBaseListener.h"
class LeafListener : public TBaseListener {
public:
virtual void exitE(TParser::EContext *ctx) override {
if (ctx->children.size() == 3) {
std::cout \<\< ctx->e(0)->start->getText() \<\< " " \<\< ctx->e(1)->start->getText() \<\< " " \<\< ctx->e()[0]->start->getText() \<\< std::endl;
} else {
std::cout \<\< ctx->INT()->getSymbol()->getText() \<\< std::endl;
}
}
};
}
>>
LRWithLabelsListener(X) ::= <<
@parser::definitions {
#include "TBaseListener.h"
class LeafListener : public TBaseListener {
public:
virtual void exitCall(TParser::CallContext *ctx) override {
std::cout \<\< ctx->e()->start->getText() \<\< " " \<\< ctx->eList()->toString() \<\< std::endl;
}
virtual void exitInt(TParser::IntContext *ctx) override {
std::cout \<\< ctx->INT()->getSymbol()->getText() \<\< std::endl;
}
};
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
void foo() {
SContext *s;
std::vector\<AContext *> a = s->a();
std::vector\<BContext *> b = s->b();
}
>>
Declare_foo() ::= <<void foo() {
std::cout \<\< "foo";
}
>>
Invoke_foo() ::= "foo();"
Declare_pred() ::= <<
bool pred(bool v) {
std::cout \<\< "eval=" \<\< std::boolalpha \<\< v \<\< std::endl;
return v;
}
>>
Invoke_pred(v) ::= <<pred(<v>)>>
ContextRuleFunction(ctx, rule) ::= "<ctx>-><rule>"
StringType() ::= "std::string"
ContextMember(ctx, subctx, member) ::= "<ctx>-><subctx>-><member>"

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/Antlr4.Test.mono.csproj → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/csharp/Antlr4.Test.mono.csproj
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/Antlr4.Test.vs2013.csproj → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/csharp/Antlr4.Test.vs2013.csproj
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/App.config → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/csharp/App.config
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/AssemblyInfo.cs → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/csharp/AssemblyInfo.cs
View File

186
runtime-testsuite/resources/org/antlr/v4/test/runtime/csharp/CSharp.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/csharp/CSharp.test.stg
View File

@ -1,155 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.csharp;
import org.junit.Test;
import org.junit.Ignore;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer", "<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser", "<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<Console.WriteLine(<s>);>>
write(s) ::= <<Console.Write(<s>);>>
writeList(s) ::= <<Console.WriteLine(<s; separator="+">);>>
False() ::= "false"
@ -227,6 +78,12 @@ RuleInvocationStack() ::= "GetRuleInvocationStackAsString()"
LL_EXACT_AMBIG_DETECTION() ::= <<Interpreter.PredictionMode = PredictionMode.LlExactAmbigDetection;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
bool Property() {
@ -235,6 +92,8 @@ bool Property() {
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
public override IToken NextToken() {
@ -315,11 +174,13 @@ public class PositionAdjustingLexerATNSimulator : LexerATNSimulator {
>>
BasicListener(X) ::= <<
@parser::members {
public class LeafListener : TBaseListener {
public override void VisitTerminal(ITerminalNode node) {
Console.WriteLine(node.Symbol.Text);
}
}
}
>>
WalkListener(s) ::= <<
@ -341,6 +202,7 @@ public class MyRuleNode : ParserRuleContext {
>>
TokenGetterListener(X) ::= <<
@parser::members {
public class LeafListener : TBaseListener {
public override void ExitA(TParser.AContext ctx) {
if (ctx.ChildCount==2)
@ -359,9 +221,11 @@ public class LeafListener : TBaseListener {
Console.WriteLine(ctx.ID().Symbol);
}
}
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
public class LeafListener : TBaseListener {
public override void ExitA(TParser.AContext ctx) {
if (ctx.ChildCount==2) {
@ -371,10 +235,12 @@ public class LeafListener : TBaseListener {
Console.WriteLine(ctx.b(0).Start.Text);
}
}
}
>>
LRListener(X) ::= <<
@parser::members {
public class LeafListener : TBaseListener {
public override void ExitE(TParser.EContext ctx) {
if (ctx.ChildCount==3) {
@ -384,9 +250,11 @@ public class LeafListener : TBaseListener {
Console.WriteLine(ctx.INT().Symbol.Text);
}
}
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
public class LeafListener : TBaseListener {
public override void ExitCall(TParser.CallContext ctx) {
Console.Write("{0} {1}",ctx.e().Start.Text,ctx.eList());
@ -395,8 +263,17 @@ public class LeafListener : TBaseListener {
Console.WriteLine(ctx.INT().Symbol.Text);
}
}
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
void foo() {
SContext s = null;
@ -416,8 +293,7 @@ Declare_pred() ::= <<bool pred(bool v) {
>>
Invoke_pred(v) ::= <<this.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

338
runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/go/Go.test.stg Normal file
View File

@ -0,0 +1,338 @@
writeln(s) ::= <<fmt.Println(<s>)>>
write(s) ::= <<fmt.Print(<s>)>>
writeList(s) ::= <<fmt.Print(<s; separator="+">);>>
False() ::= "false"
True() ::= "true"
Not(v) ::= "!<v>"
Assert(s) ::= ""
Cast(t,v) ::= "(<v>)"
Append(a,b) ::= "<a> + fmt.Sprint(<b>)"
Concat(a,b) ::= "<a><b>"
DeclareLocal(s, v) ::= "var <s> = <v>"
AssertIsList(v) ::= ""
AssignLocal(s, v) ::= "<s> = <v>;"
InitIntMember(n, v) ::= <%var <n> int = <v>; var _ int = <n>; %>
InitBooleanMember(n, v) ::= <%var <n> bool = <v>; var _ bool = <n>; %>
GetMember(n) ::= <%<n>%>
SetMember(n, v) ::= <%<n> = <v>;%>
AddMember(n, v) ::= <%<n> += <v>;%>
PlusMember(v, n) ::= <%<v> + fmt.Sprint(<n>)%>
MemberEquals(n, v) ::= <%<n> == <v>%>
ModMemberEquals(n, m, v) ::= <%<n> % <m> == <v>%>
ModMemberNotEquals(n, m, v) ::= <%<n> % <m> != <v>%>
DumpDFA() ::= "p.DumpDFA()"
Pass() ::= ""
StringList() ::= "[]string"
BuildParseTrees() ::= "p.BuildParseTrees = true"
BailErrorStrategy() ::= <%p.SetErrorHandler(antlr.NewBailErrorStrategy())%>
ToStringTree(s) ::= <%<s>.ToStringTree(nil, p)%>
Column() ::= "p.GetCharPositionInLine()"
Text() ::= "l.GetText()"
ValEquals(a, b) ::= <%<a> == <b>%>
TextEquals(a) ::= <%p.GetText() == "<a>"%>
PlusText(a) ::= <%"<a>" + l.GetText()%>
InputText() ::= "p.GetTokenStream().GetAllText()"
LTEquals(i, v) ::= <%p.GetTokenStream().LT(<i>).GetText() == <v>%>
LANotEquals(i, v) ::= <%p.GetTokenStream().LA(<i>) != <v>%>
TokenStartColumnEquals(i) ::= <%p.TokenStartColumn == <i>%>
ImportListener(X) ::= ""
GetExpectedTokenNames() ::= "p.GetExpectedTokens().StringVerbose(p.GetTokenNames(), nil, false)"
RuleInvocationStack() ::= "antlr.PrintArrayJavaStyle(p.GetRuleInvocationStack(nil))"
LL_EXACT_AMBIG_DETECTION() ::= <<p.Interpreter.SetPredictionMode(antlr.PredictionModeLLExactAmbigDetection);>>
ParserToken(parser, token) ::= <%<parser><token>%>
Production(p) ::= <%<p; format="cap">%>
Result(r) ::= <%Get<r; format="cap">()%>
ParserPropertyMember() ::= <<
@parser::members {
func (p *TParser) Property() bool {
return true
}
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
func (p *PositionAdjustingLexer) NextToken() antlr.Token {
if _, ok := p.Interpreter.(*PositionAdjustingLexerATNSimulator); !ok {
p.Interpreter = NewPositionAdjustingLexerATNSimulator(p, lexerAtn, p.Interpreter.DecisionToDFA(), p.Interpreter.SharedContextCache())
p.Virt = p
}
return p.BaseLexer.NextToken()
}
func (p *PositionAdjustingLexer) Emit() antlr.Token {
switch p.GetType() {
case PositionAdjustingLexerTOKENS:
p.HandleAcceptPositionForKeyword("tokens")
case PositionAdjustingLexerLABEL:
p.HandleAcceptPositionForIdentifier()
}
return p.BaseLexer.Emit()
}
func isIdentifierChar(c rune) bool {
return unicode.IsLetter(c) || unicode.IsDigit(c) || c == '_'
}
func (p *PositionAdjustingLexer) HandleAcceptPositionForIdentifier() bool {
var tokenText = p.GetText()
var identifierLength int
for identifierLength \< len(tokenText) && isIdentifierChar([]rune(tokenText)[identifierLength]) {
identifierLength++
}
if p.GetInputStream().Index() \<= p.TokenStartCharIndex + identifierLength {
return false
}
var offset = identifierLength - 1
p.GetInterpreter().(*PositionAdjustingLexerATNSimulator).ResetAcceptPosition(p.GetInputStream(), p.TokenStartCharIndex + offset, p.TokenStartLine, p.TokenStartColumn + offset)
return true
}
func (p *PositionAdjustingLexer) HandleAcceptPositionForKeyword(keyword string) bool {
if p.GetInputStream().Index() \<= p.TokenStartCharIndex + len(keyword) {
return false
}
var offset = len(keyword) - 1
p.GetInterpreter().(*PositionAdjustingLexerATNSimulator).ResetAcceptPosition(p.GetInputStream(), p.TokenStartCharIndex + offset, p.TokenStartLine, p.TokenStartColumn + offset)
return true
}
type PositionAdjustingLexerATNSimulator struct {
*antlr.LexerATNSimulator
}
func NewPositionAdjustingLexerATNSimulator(recog antlr.Lexer, atn *antlr.ATN, decisionToDFA []*antlr.DFA, sharedContextCache *antlr.PredictionContextCache) *PositionAdjustingLexerATNSimulator {
return &PositionAdjustingLexerATNSimulator{
LexerATNSimulator: antlr.NewLexerATNSimulator(recog, atn, decisionToDFA, sharedContextCache),
}
}
func (p *PositionAdjustingLexerATNSimulator) ResetAcceptPosition(input antlr.CharStream, index, line, charPositionInLine int) {
input.Seek(index)
p.Line = line
p.CharPositionInLine = charPositionInLine
p.Consume(input)
}
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::members {
type MyRuleNode struct {
*antlr.BaseParserRuleContext
altNum int
}
func NewMyRuleNode(parent antlr.ParserRuleContext, invokingStateNumber int) *MyRuleNode {
return &MyRuleNode{
BaseParserRuleContext : antlr.NewBaseParserRuleContext(parent, invokingStateNumber),
}
}
func (m *MyRuleNode) GetAltNumber() int {
return m.altNum
}
func (m *MyRuleNode) SetAltNumber(altNum int) {
m.altNum = altNum
}
}
>>
BasicListener(notused) ::= <<
@parser::members {
type LeafListener struct {
*BaseTListener
}
func NewLeafListener() *LeafListener {
return &LeafListener{BaseTListener: &BaseTListener{}}
}
func (*LeafListener) VisitTerminal(node antlr.TerminalNode) {
fmt.Println(node.GetSymbol().GetText())
}
}
>>
WalkListener(s) ::= <<
var walker = antlr.NewParseTreeWalker()
walker.Walk(NewLeafListener(), <s>)
>>
TokenGetterListener(notused) ::= <<
@parser::members {
type LeafListener struct {
*BaseTListener
}
func NewLeafListener() *LeafListener {
return &LeafListener{BaseTListener: &BaseTListener{}}
}
func (*LeafListener) ExitA(ctx *AContext) {
if ctx.GetChildCount() == 2 {
fmt.Printf("%s %s %s", ctx.INT(0).GetSymbol().GetText(), ctx.INT(1).GetSymbol().GetText(), antlr.PrintArrayJavaStyle(antlr.TerminalNodeToStringArray(ctx.AllINT())))
} else {
fmt.Println(ctx.ID().GetSymbol())
}
}
}
>>
RuleGetterListener(notused) ::= <<
@parser::members {
type LeafListener struct {
*BaseTListener
}
func NewLeafListener() *LeafListener {
return &LeafListener{BaseTListener: &BaseTListener{}}
}
func (*LeafListener) ExitA(ctx *AContext) {
if ctx.GetChildCount() == 2 {
fmt.Printf("%s %s %s", ctx.B(0).GetStart().GetText(), ctx.B(1).GetStart().GetText(), ctx.AllB()[0].GetStart().GetText())
} else {
fmt.Println(ctx.B(0).GetStart().GetText())
}
}
}
>>
LRListener(notused) ::= <<
@parser::members {
type LeafListener struct {
*BaseTListener
}
func NewLeafListener() *LeafListener {
return &LeafListener{BaseTListener: &BaseTListener{}}
}
func (*LeafListener) ExitE(ctx *EContext) {
if ctx.GetChildCount() == 3 {
fmt.Printf("%s %s %s\n", ctx.E(0).GetStart().GetText(), ctx.E(1).GetStart().GetText(), ctx.AllE()[0].GetStart().GetText())
} else {
fmt.Println(ctx.INT().GetSymbol().GetText())
}
}
}
>>
LRWithLabelsListener(notused) ::= <<
@parser::members {
type LeafListener struct {
*BaseTListener
}
func NewLeafListener() *LeafListener {
return &LeafListener{BaseTListener: &BaseTListener{}}
}
func (*LeafListener) ExitCall(ctx *CallContext) {
fmt.Printf("%s %s", ctx.E().GetStart().GetText(), ctx.EList().String(nil, nil))
}
func (*LeafListener) ExitInt(ctx *IntContext) {
fmt.Println(ctx.INT().GetSymbol().GetText())
}
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
func foo() {
// TODO
// var s SContext
// var a = s.A()
// var b = s.B()
}
>>
Declare_foo() ::= <<
func foo() {
fmt.Println("foo")
}
>>
Invoke_foo() ::= "foo()"
Declare_pred() ::= <<
func pred(v bool) bool {
fmt.Println("eval=" + fmt.Sprint(v))
return v
}
>>
Invoke_pred(v) ::= <<pred(<v>)>>
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member; format={cap}>"

192
runtime-testsuite/resources/org/antlr/v4/test/runtime/java/Java.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/java/Java.test.stg
View File

@ -1,158 +1,6 @@
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.java;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<if(test.AfterGrammar)>
<test.AfterGrammar>
<endif>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.AfterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer", "<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser", "<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
<testAnnotations(test)>
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Timeout"))>
(timeout = <test.Options.("Timeout")>)
<endif>
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<System.out.println(<s>);>>
write(s) ::= <<System.out.print(<s>);>>
writeList(s) ::= <<System.out.println(<s; separator="+">);>>
False() ::= "false"
@ -230,6 +78,12 @@ RuleInvocationStack() ::= "getRuleInvocationStack()"
LL_EXACT_AMBIG_DETECTION() ::= <<_interp.setPredictionMode(PredictionMode.LL_EXACT_AMBIG_DETECTION);>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
boolean Property() {
@ -238,6 +92,8 @@ boolean Property() {
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
@Override
@ -323,11 +179,13 @@ protected static class PositionAdjustingLexerATNSimulator extends LexerATNSimula
>>
BasicListener(X) ::= <<
@parser::members {
public static class LeafListener extends TBaseListener {
public void visitTerminal(TerminalNode node) {
System.out.println(node.getSymbol().getText());
}
}
}
>>
WalkListener(s) ::= <<
@ -349,6 +207,7 @@ public static class MyRuleNode extends ParserRuleContext {
>>
TokenGetterListener(X) ::= <<
@parser::members {
public static class LeafListener extends TBaseListener {
public void exitA(TParser.AContext ctx) {
if (ctx.getChildCount()==2)
@ -358,9 +217,11 @@ public static class LeafListener extends TBaseListener {
System.out.println(ctx.ID().getSymbol());
}
}
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
public static class LeafListener extends TBaseListener {
public void exitA(TParser.AContext ctx) {
if (ctx.getChildCount()==2) {
@ -370,10 +231,12 @@ public static class LeafListener extends TBaseListener {
System.out.println(ctx.b(0).start.getText());
}
}
}
>>
LRListener(X) ::= <<
@parser::members {
public static class LeafListener extends TBaseListener {
public void exitE(TParser.EContext ctx) {
if (ctx.getChildCount()==3) {
@ -383,9 +246,11 @@ public static class LeafListener extends TBaseListener {
System.out.println(ctx.INT().getSymbol().getText());
}
}
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
public static class LeafListener extends TBaseListener {
public void exitCall(TParser.CallContext ctx) {
System.out.printf("%s %s",ctx.e().start.getText(),ctx.eList());
@ -394,8 +259,17 @@ public static class LeafListener extends TBaseListener {
System.out.println(ctx.INT().getSymbol().getText());
}
}
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
void foo() {
SContext s = null;
@ -418,11 +292,7 @@ Declare_pred() ::= <<boolean pred(boolean v) {
Invoke_pred(v) ::= <<this.pred(<v>)>>
IgnoredTests ::= [
default: false
]
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

169
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/chrome/Chrome.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/javascript/chrome/Chrome.test.stg
View File

@ -1,158 +1,3 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.javascript.chrome;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer",
"<grammar>Listener", "<grammar>Visitor",
"<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser",
"<test.grammar.grammarName>Listener", "<test.grammar.grammarName>Visitor",
"<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<document.getElementById('output').value += <s> + '\\n';>>
write(s) ::= <<document.getElementById('output').value += <s>;>>
@ -233,6 +78,12 @@ RuleInvocationStack() ::= "antlr4.Utils.arrayToString(this.getRuleInvocationStac
LL_EXACT_AMBIG_DETECTION() ::= <<this._interp.predictionMode = antlr4.atn.PredictionMode.LL_EXACT_AMBIG_DETECTION;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
this.Property = function() {
@ -428,7 +279,7 @@ Declare_pred() ::= <<this.pred = function(v) {
Invoke_pred(v) ::= <<this.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"
ParserPropertyCall(p, call) ::= "<p>.<call>"

294
runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/javascript/explorer/Explorer.test.stg Normal file
View File

@ -0,0 +1,294 @@
writeln(s) ::= <<document.getElementById('output').value += <s> + '\\n';>>
write(s) ::= <<document.getElementById('output').value += <s>;>>
writeList(s) ::= <<document.getElementById('output').value += <s; separator="+">;>>
False() ::= "false"
True() ::= "true"
Not(v) ::= "!<v>"
Assert(s) ::= ""
Cast(t,v) ::= "<v>"
Append(a,b) ::= "<a> + <b>"
Concat(a,b) ::= "<a><b>"
DeclareLocal(s,v) ::= "var <s> = <v>;"
AssertIsList(v) ::= <<if ( !(v instanceof Array) ) {throw "value is not an array";}>>
AssignLocal(s,v) ::= "<s> = <v>;"
InitIntMember(n,v) ::= <%this.<n> = <v>;%>
InitBooleanMember(n,v) ::= <%this.<n> = <v>;%>
GetMember(n) ::= <%this.<n>%>
SetMember(n,v) ::= <%this.<n> = <v>;%>
AddMember(n,v) ::= <%this.<n> += <v>;%>
PlusMember(v,n) ::= <%<v> + this.<n>%>
MemberEquals(n,v) ::= <%this.<n> === <v>%>
ModMemberEquals(n,m,v) ::= <%this.<n> % <m> === <v>%>
ModMemberNotEquals(n,m,v) ::= <%this.<n> % <m> != <v>%>
DumpDFA() ::= "this.dumpDFA();"
Pass() ::= ""
StringList() ::= "list"
BuildParseTrees() ::= "this.buildParseTrees = true;"
BailErrorStrategy() ::= <%this._errHandler = new antlr4.error.BailErrorStrategy();%>
ToStringTree(s) ::= <%<s>.toStringTree(null, this)%>
Column() ::= "this.column"
Text() ::= "this.text"
ValEquals(a,b) ::= <%<a>===<b>%>
TextEquals(a) ::= <%this.text==="<a>"%>
PlusText(a) ::= <%"<a>" + this.text%>
InputText() ::= "this._input.getText()"
LTEquals(i, v) ::= <%this._input.LT(<i>).text===<v>%>
LANotEquals(i, v) ::= <%this._input.LA(<i>)!=<v>%>
TokenStartColumnEquals(i) ::= <%this._tokenStartColumn===<i>%>
ImportListener(X) ::= <<
@parser::header {
var <X>Listener = require('./<X>Listener').<X>Listener;
}
>>
GetExpectedTokenNames() ::= "this.getExpectedTokens().toString(this.literalNames)"
RuleInvocationStack() ::= "antlr4.Utils.arrayToString(this.getRuleInvocationStack())"
LL_EXACT_AMBIG_DETECTION() ::= <<this._interp.predictionMode = antlr4.atn.PredictionMode.LL_EXACT_AMBIG_DETECTION;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
this.Property = function() {
return true;
}
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
PositionAdjustingLexer.prototype.resetAcceptPosition = function(index, line, column) {
this._input.seek(index);
this.line = line;
this.column = column;
this._interp.consume(this._input);
};
PositionAdjustingLexer.prototype.nextToken = function() {
if (!("resetAcceptPosition" in this._interp)) {
var lexer = this;
this._interp.resetAcceptPosition = function(index, line, column) { lexer.resetAcceptPosition(index, line, column); };
}
return antlr4.Lexer.prototype.nextToken.call(this);
};
PositionAdjustingLexer.prototype.emit = function() {
switch(this._type) {
case PositionAdjustingLexer.TOKENS:
this.handleAcceptPositionForKeyword("tokens");
break;
case PositionAdjustingLexer.LABEL:
this.handleAcceptPositionForIdentifier();
break;
}
return antlr4.Lexer.prototype.emit.call(this);
};
PositionAdjustingLexer.prototype.handleAcceptPositionForIdentifier = function() {
var tokenText = this.text;
var identifierLength = 0;
while (identifierLength \< tokenText.length &&
PositionAdjustingLexer.isIdentifierChar(tokenText[identifierLength])
) {
identifierLength += 1;
}
if (this._input.index > this._tokenStartCharIndex + identifierLength) {
var offset = identifierLength - 1;
this._interp.resetAcceptPosition(this._tokenStartCharIndex + offset,
this._tokenStartLine, this._tokenStartColumn + offset);
return true;
} else {
return false;
}
};
PositionAdjustingLexer.prototype.handleAcceptPositionForKeyword = function(keyword) {
if (this._input.index > this._tokenStartCharIndex + keyword.length) {
var offset = keyword.length - 1;
this._interp.resetAcceptPosition(this._tokenStartCharIndex + offset,
this._tokenStartLine, this._tokenStartColumn + offset);
return true;
} else {
return false;
}
};
PositionAdjustingLexer.isIdentifierChar = function(c) {
return c.match(/^[0-9a-zA-Z_]+$/);
}
>>
BasicListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.visitTerminal = function(node) {
document.getElementById('output').value += node.symbol.text + '\\n';
};
return this;
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
WalkListener(s) ::= <<
var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
this.altNum = 0;
return this;
};
MyRuleNode.prototype = Object.create(antlr4.ParserRuleContext.prototype);
MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
if(ctx.getChildCount()===2) {
str = ctx.INT(0).symbol.text + ' ' + ctx.INT(1).symbol.text + ' ' + antlr4.Utils.arrayToString(ctx.INT());
} else {
str = ctx.ID().symbol.toString();
}
document.getElementById('output').value += str + '\\n';
};
return this;
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
if(ctx.getChildCount()===2) {
str = ctx.b(0).start.text + ' ' + ctx.b(1).start.text + ' ' + ctx.b()[0].start.text;
} else {
str = ctx.b(0).start.text;
}
document.getElementById('output').value += str + '\\n';
};
return this;
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitE = function(ctx) {
var str;
if(ctx.getChildCount()===3) {
str = ctx.e(0).start.text + ' ' + ctx.e(1).start.text + ' ' + ctx.e()[0].start.text;
} else {
str = ctx.INT().symbol.text;
}
document.getElementById('output').value += str + '\\n';
};
return this;
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitCall = function(ctx) {
var str = ctx.e().start.text + ' ' + ctx.eList();
document.getElementById('output').value += str + '\\n';
};
this.exitInt = function(ctx) {
var str = ctx.INT().symbol.text;
document.getElementById('output').value += str + '\\n';
};
return this;
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
DeclareContextListGettersFunction() ::= <<
function foo() {
var s = new SContext();
var a = s.a();
var b = s.b();
};
>>
Declare_foo() ::= "this.foo = function() {document.getElementById('output').value += 'foo' + '\\n';};"
Invoke_foo() ::= "this.foo();"
Declare_pred() ::= <<this.pred = function(v) {
document.getElementById('output').value += 'eval=' + v.toString() + '\\n';
return v;
};
>>
Invoke_pred(v) ::= <<this.pred(<v>)>>
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

196
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/firefox/Firefox.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/javascript/firefox/Firefox.test.stg
View File

@ -1,161 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.javascript.firefox;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer",
"<grammar>Listener", "<grammar>Visitor",
"<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser",
"<test.grammar.grammarName>Listener", "<test.grammar.grammarName>Visitor",
"<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<document.getElementById('output').value += <s> + '\\n';>>
write(s) ::= <<document.getElementById('output').value += <s>;>>
writeList(s) ::= <<document.getElementById('output').value += <s; separator="+">;>>
False() ::= "false"
@ -227,7 +72,11 @@ LANotEquals(i, v) ::= <%this._input.LA(<i>)!=<v>%>
TokenStartColumnEquals(i) ::= <%this._tokenStartColumn===<i>%>
ImportListener(X) ::= <<var <X>Listener = require('./<X>Listener').<X>Listener;>>
ImportListener(X) ::= <<
@parser::header {
var <X>Listener = require('./<X>Listener').<X>Listener;
}
>>
GetExpectedTokenNames() ::= "this.getExpectedTokens().toString(this.literalNames)"
@ -235,6 +84,12 @@ RuleInvocationStack() ::= "antlr4.Utils.arrayToString(this.getRuleInvocationStac
LL_EXACT_AMBIG_DETECTION() ::= <<this._interp.predictionMode = antlr4.atn.PredictionMode.LL_EXACT_AMBIG_DETECTION;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
this.Property = function() {
@ -243,6 +98,8 @@ this.Property = function() {
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
PositionAdjustingLexer.prototype.resetAcceptPosition = function(index, line, column) {
@ -308,6 +165,7 @@ PositionAdjustingLexer.isIdentifierChar = function(c) {
>>
BasicListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.visitTerminal = function(node) {
document.getElementById('output').value += node.symbol.text + '\\n';
@ -316,7 +174,7 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
WalkListener(s) ::= <<
@ -325,7 +183,6 @@ walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
@ -339,6 +196,7 @@ MyRuleNode.prototype.constructor = MyRuleNode;
>>
TokenGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -353,10 +211,11 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -371,11 +230,12 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitE = function(ctx) {
var str;
@ -390,10 +250,11 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitCall = function(ctx) {
var str = ctx.e().start.text + ' ' + ctx.eList();
@ -407,7 +268,7 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
DeclareContextListGettersFunction() ::= <<
@ -429,8 +290,7 @@ Declare_pred() ::= <<this.pred = function(v) {
>>
Invoke_pred(v) ::= <<this.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

279
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/node/Node.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/javascript/node/Node.test.stg
View File

@ -1,161 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.javascript.node;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer",
"<grammar>Listener", "<grammar>Visitor",
"<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser",
"<test.grammar.grammarName>Listener", "<test.grammar.grammarName>Visitor",
"<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<console.log(<s>);>>
write(s) ::= <<process.stdout.write(<s>);>>
writeList(s) ::= <<console.log(<s; separator="+">);>>
False() ::= "false"
@ -225,7 +70,13 @@ LANotEquals(i, v) ::= <%this._input.LA(<i>)!=<v>%>
TokenStartColumnEquals(i) ::= <%this._tokenStartColumn===<i>%>
ImportListener(X) ::= <<var <X>Listener = require('./<X>Listener').<X>Listener;>>
ImportListener(X) ::= <<
@parser::header {
var <X>Listener = require('./<X>Listener').<X>Listener;
}
>>
ImportVisitor(X) ::= <<var <X>Visitor = require('./<X>Visitor').<X>Visitor;>>
GetExpectedTokenNames() ::= "this.getExpectedTokens().toString(this.literalNames)"
@ -233,6 +84,12 @@ RuleInvocationStack() ::= "antlr4.Utils.arrayToString(this.getRuleInvocationStac
LL_EXACT_AMBIG_DETECTION() ::= <<this._interp.predictionMode = antlr4.atn.PredictionMode.LL_EXACT_AMBIG_DETECTION;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
this.Property = function() {
@ -241,6 +98,8 @@ this.Property = function() {
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
PositionAdjustingLexer.prototype.resetAcceptPosition = function(index, line, column) {
@ -306,6 +165,7 @@ PositionAdjustingLexer.isIdentifierChar = function(c) {
>>
BasicListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.visitTerminal = function(node) {
console.log(node.symbol.text);
@ -314,7 +174,7 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
WalkListener(s) ::= <<
@ -322,8 +182,25 @@ var walker = new antlr4.tree.ParseTreeWalker();
walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
BasicVisitor(X) ::= <<
this.LeafVisitor = function() {
this.visitTerminal = function(node) {
return node.symbol.text;
};
return this;
};
this.LeafVisitor.prototype = Object.create(<X>Visitor.prototype);
this.LeafVisitor.prototype.constructor = this.LeafVisitor;
>>
WalkVisitor(s) ::= <<
var visitor = new this.LeafVisitor();
console.log(<s>.accept(visitor));
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
@ -341,6 +218,7 @@ MyRuleNode.prototype.setAltNumber = function(altNumber) { this.altNum = altNumbe
>>
TokenGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -355,10 +233,29 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
TokenGetterVisitor(X) ::= <<
this.LeafVisitor = function() {
this.visitA = function(ctx) {
var str;
if(ctx.getChildCount()===2) {
str = ctx.INT(0).symbol.text + ' ' + ctx.INT(1).symbol.text + ' ' + antlr4.Utils.arrayToString(ctx.INT());
} else {
str = ctx.ID().symbol.toString();
}
return this.visitChildren(ctx) + str;
};
return this;
};
this.LeafVisitor.prototype = Object.create(<X>Visitor.prototype);
this.LeafVisitor.prototype.constructor = this.LeafVisitor;
>>
RuleGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -373,11 +270,30 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
RuleGetterVisitor(X) ::= <<
this.LeafVisitor = function() {
this.visitA = function(ctx) {
var str;
if(ctx.getChildCount()===2) {
str = ctx.b(0).start.text + ' ' + ctx.b(1).start.text + ' ' + ctx.b()[0].start.text;
} else {
str = ctx.b(0).start.text;
}
return this.visitChildren(ctx) + str;
};
return this;
};
this.LeafVisitor.prototype = Object.create(<X>Visitor.prototype);
this.LeafVisitor.prototype.constructor = this.LeafVisitor;
>>
LRListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitE = function(ctx) {
var str;
@ -392,10 +308,29 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRVisitor(X) ::= <<
this.LeafVisitor = function() {
this.visitE = function(ctx) {
var str;
if(ctx.getChildCount()===3) {
str = ctx.e(0).start.text + ' ' + ctx.e(1).start.text + ' ' + ctx.e()[0].start.text;
} else {
str = ctx.INT().symbol.text;
}
return this.visitChildren(ctx) + str;
};
return this;
};
this.LeafVisitor.prototype = Object.create(<X>Visitor.prototype);
this.LeafVisitor.prototype.constructor = this.LeafVisitor;
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitCall = function(ctx) {
var str = ctx.e().start.text + ' ' + ctx.eList();
@ -409,6 +344,23 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRWithLabelsVisitor(X) ::= <<
this.LeafVisitor = function() {
this.visitCall = function(ctx) {
var str = ctx.e().start.text + ' ' + ctx.eList();
return this.visitChildren(ctx) + str;
};
this.visitInt = function(ctx) {
var str = ctx.INT().symbol.text;
return this.visitChildren(ctx) + str;
};
return this;
};
this.LeafVisitor.prototype = Object.create(<X>Visitor.prototype);
this.LeafVisitor.prototype.constructor = this.LeafVisitor;
>>
@ -431,8 +383,7 @@ Declare_pred() ::= <<this.pred = function(v) {
>>
Invoke_pred(v) ::= <<this.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

197
runtime-testsuite/resources/org/antlr/v4/test/runtime/javascript/explorer/Explorer.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/javascript/safari/Safari.test.stg
View File

@ -1,161 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.javascript.explorer;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BaseTest {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}<\n>
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.afterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar>Parser", "<grammar>Lexer",
"<grammar>Listener", "<grammar>Visitor",
"<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser",
"<test.grammar.grammarName>Listener", "<test.grammar.grammarName>Visitor",
"<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<document.getElementById('output').value += <s> + '\\n';>>
write(s) ::= <<document.getElementById('output').value += <s>;>>
writeList(s) ::= <<document.getElementById('output').value += <s; separator="+">;>>
False() ::= "false"
@ -225,7 +70,11 @@ LANotEquals(i, v) ::= <%this._input.LA(<i>)!=<v>%>
TokenStartColumnEquals(i) ::= <%this._tokenStartColumn===<i>%>
ImportListener(X) ::= <<var <X>Listener = require('./<X>Listener').<X>Listener;>>
ImportListener(X) ::= <<
@parser::header {
var <X>Listener = require('./<X>Listener').<X>Listener;
}
>>
GetExpectedTokenNames() ::= "this.getExpectedTokens().toString(this.literalNames)"
@ -233,6 +82,12 @@ RuleInvocationStack() ::= "antlr4.Utils.arrayToString(this.getRuleInvocationStac
LL_EXACT_AMBIG_DETECTION() ::= <<this._interp.predictionMode = antlr4.atn.PredictionMode.LL_EXACT_AMBIG_DETECTION;>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
this.Property = function() {
@ -241,6 +96,8 @@ this.Property = function() {
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
PositionAdjustingLexer.prototype.resetAcceptPosition = function(index, line, column) {
@ -306,6 +163,7 @@ PositionAdjustingLexer.isIdentifierChar = function(c) {
>>
BasicListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.visitTerminal = function(node) {
document.getElementById('output').value += node.symbol.text + '\\n';
@ -314,7 +172,7 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
WalkListener(s) ::= <<
@ -323,7 +181,6 @@ walker.walk(new this.LeafListener(), <s>);
>>
TreeNodeWithAltNumField(X) ::= <<
@parser::header {
MyRuleNode = function(parent, invokingState) {
antlr4.ParserRuleContext.call(this, parent, invokingState);
@ -336,7 +193,9 @@ MyRuleNode.prototype.constructor = MyRuleNode;
}
>>
TokenGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -351,10 +210,11 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitA = function(ctx) {
var str;
@ -369,11 +229,12 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitE = function(ctx) {
var str;
@ -388,10 +249,11 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
this.LeafListener = function() {
this.exitCall = function(ctx) {
var str = ctx.e().start.text + ' ' + ctx.eList();
@ -405,7 +267,7 @@ this.LeafListener = function() {
};
this.LeafListener.prototype = Object.create(<X>Listener.prototype);
this.LeafListener.prototype.constructor = this.LeafListener;
}
>>
DeclareContextListGettersFunction() ::= <<
@ -427,8 +289,7 @@ Declare_pred() ::= <<this.pred = function(v) {
>>
Invoke_pred(v) ::= <<this.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

201
runtime-testsuite/resources/org/antlr/v4/test/runtime/python2/Python2.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/python2/Python2.test.stg
View File

@ -1,165 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.python2;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BasePython2Test {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<if(test.AfterGrammar)>
<test.AfterGrammar>
<endif>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.AfterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar><if(!test.slaveIsLexer)>Parser<endif>", "<if(test.slaveIsLexer)><first(test.slaveGrammars).grammarName><else><grammar>Lexer<endif>", "<grammar>Listener", "<grammar>Visitor", "<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser", "<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<print(<s>)>>
write(s) ::= <<print(<s>,end='')>>
writeList(s) ::= <<print(<s: {v | str(<v>)}; separator="+">)>>
False() ::= "False"
@ -237,6 +78,12 @@ RuleInvocationStack() ::= "str_list(self.getRuleInvocationStack())"
LL_EXACT_AMBIG_DETECTION() ::= <<self._interp.predictionMode = PredictionMode.LL_EXACT_AMBIG_DETECTION>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
def Property(self):
@ -245,6 +92,8 @@ def Property(self):
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
def resetAcceptPosition(self, index, line, column):
@ -296,6 +145,7 @@ def isIdentifierChar(c):
>>
BasicListener(X) ::= <<
@parser::members {
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener
else:
@ -304,7 +154,7 @@ else:
class LeafListener(TListener):
def visitTerminal(self, node):
print(node.symbol.text)
}
>>
WalkListener(s) ::= <<
@ -326,6 +176,7 @@ class MyRuleNode(ParserRuleContext):
>>
TokenGetterListener(X) ::= <<
@parser::members {
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener
else:
@ -337,10 +188,11 @@ class LeafListener(TListener):
print(ctx.INT(0).symbol.text + ' ' + ctx.INT(1).symbol.text + ' ' + str_list(ctx.INT()))
else:
print(str(ctx.ID().symbol))
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener
else:
@ -352,11 +204,12 @@ class LeafListener(TListener):
print(ctx.b(0).start.text + ' ' + ctx.b(1).start.text + ' ' + ctx.b()[0].start.text)
else:
print(ctx.b(0).start.text)
}
>>
LRListener(X) ::= <<
@parser::members {
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener
else:
@ -368,10 +221,11 @@ class LeafListener(TListener):
print(ctx.e(0).start.text + ' ' + ctx.e(1).start.text + ' ' + ctx.e()[0].start.text)
else:
print(ctx.INT().symbol.text)
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
if __name__ is not None and "." in __name__:
from .<X>Listener import <X>Listener
else:
@ -382,9 +236,17 @@ class LeafListener(TListener):
print(ctx.e().start.text + ' ' + str(ctx.eList()))
def exitInt(self, ctx):
print(ctx.INT().symbol.text)
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
def foo():
s = SContext()
@ -405,8 +267,7 @@ Declare_pred() ::= <<def pred(self, v):
>>
Invoke_pred(v) ::= <<self.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

206
runtime-testsuite/resources/org/antlr/v4/test/runtime/python3/Python3.test.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/python3/Python3.test.stg
View File

@ -1,165 +1,6 @@
IgnoredTests ::= [
default: false
]
TestFile(file) ::= <<
/* This file is generated by TestGenerator, any edits will be overwritten by the next generation. */
package org.antlr.v4.test.runtime.python3;
import org.junit.Ignore;
import org.junit.Test;
import static org.junit.Assert.*;
<if(file.Options.("ImportErrorQueue"))>
import org.antlr.v4.test.runtime.java.ErrorQueue;
<endif>
<if(file.Options.("ImportGrammar"))>
import org.antlr.v4.tool.Grammar;
<endif>
@SuppressWarnings("unused")
public class Test<file.name> extends BasePython3Test {
<file.tests:{test | <test>}; separator="\n", wrap, anchor>
}
>>
LexerTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<if(test.AfterGrammar)>
<test.AfterGrammar>
<endif>
String input =<writeStringLiteral(test.Input)>;
String found = execLexer("<grammar>.g4", grammar, "<grammar><if(test.Options.("CombinedGrammar"))>Lexer<endif>", input, <writeBoolean(test.Options.("ShowDFA"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeLexerTestMethod(test) ::= <<
<LexerTestMethod(test)>
>>
ParserTestMethod(test) ::= <<
/* This file and method are generated by TestGenerator, any edits will be overwritten by the next generation. */
<testAnnotations(test)>
public void test<test.name>() throws Exception {
mkdir(tmpdir);
<test.SlaveGrammars:{grammar |
String slave_<grammar> =<writeStringLiteral(test.SlaveGrammars.(grammar))>;
<if(test.Options.("SlaveIsLexer"))>
rawGenerateAndBuildRecognizer("<grammar>.g4", slave_<grammar>, null, "<grammar>");
<else>
writeFile(tmpdir, "<grammar>.g4", slave_<grammar>);
<endif>
}; separator="\n">
<test.Grammar:{grammar |
<buildStringLiteral(test.Grammar.(grammar), "grammar")>
<test.AfterGrammar>
String input =<writeStringLiteral(test.Input)>;
String found = execParser("<grammar>.g4", grammar, "<grammar><if(!test.slaveIsLexer)>Parser<endif>", "<if(test.slaveIsLexer)><first(test.slaveGrammars).grammarName><else><grammar>Lexer<endif>", "<grammar>Listener", "<grammar>Visitor", "<test.Rule>", input, <writeBoolean(test.Options.("Debug"))>);
assertEquals(<writeStringLiteral(test.Output)>, found);
<if(!isEmpty.(test.Errors))>
assertEquals(<writeStringLiteral(test.Errors)>, this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}>
}
>>
CompositeParserTestMethod(test) ::= <<
<ParserTestMethod(test)>
>>
AbstractParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
String test<test.name>(String input) throws Exception {
String grammar = <test.grammar.lines:{ line | "<line>};separator="\\n\" +\n", wrap, anchor>";
return execParser("<test.grammar.grammarName>.g4", grammar, "<test.grammar.grammarName>Parser", "<test.grammar.grammarName>Lexer", "<test.startRule>", input, <test.debug>);
}
>>
ConcreteParserTestMethod(test) ::= <<
/* this file and method are generated, any edit will be overwritten by the next generation */
@Test
public void test<test.name>() throws Exception {
String found = test<test.baseName>("<test.input>");
assertEquals("<test.expectedOutput>", found);
<if(test.expectedErrors)>
assertEquals("<test.expectedErrors>", this.stderrDuringParse);
<else>
assertNull(this.stderrDuringParse);
<endif>
}
>>
testAnnotations(test) ::= <%
@Test
<if(test.Options.("Ignore"))>
<\n>@Ignore(<writeStringLiteral(test.Options.("Ignore"))>)
<elseif(IgnoredTests.(({<file.name>.<test.name>})))>
<\n>@Ignore(<writeStringLiteral(IgnoredTests.(({<file.name>.<test.name>})))>)
<endif>
%>
buildStringLiteral(text, variable) ::= <<
StringBuilder <variable>Builder = new StringBuilder(<strlen.(text)>);
<lines.(text):{line|<variable>Builder.append("<escape.(line)>");}; separator="\n">
String <variable> = <variable>Builder.toString();
>>
writeStringLiteral(text) ::= <%
<if(isEmpty.(text))>
""
<else>
<writeLines(lines.(text))>
<endif>
%>
writeLines(textLines) ::= <%
<if(rest(textLines))>
<textLines:{line|
<\n> "<escape.(line)>}; separator="\" +">"
<else>
"<escape.(first(textLines))>"
<endif>
%>
string(text) ::= <<
"<escape.(text)>"
>>
writeBoolean(o) ::= "<if(o && !isEmpty.(o))>true<else>false<endif>"
writeln(s) ::= <<print(<s>)>>
write(s) ::= <<print(<s>,end='')>>
writeList(s) ::= <<print(<s: {v | str(<v>)}; separator="+">)>>
False() ::= "False"
@ -229,8 +70,11 @@ LANotEquals(i, v) ::= <%self._input.LA(<i>)!=<v>%>
TokenStartColumnEquals(i) ::= <%self._tokenStartColumn==<i>%>
ImportListener(X) ::= <<class MockListener:
ImportListener(X) ::= <<
@parser::header {
class MockListener:
pass
}
>>
GetExpectedTokenNames() ::= "self.getExpectedTokens().toString(self.literalNames, self.symbolicNames)"
@ -239,6 +83,12 @@ RuleInvocationStack() ::= "str_list(self.getRuleInvocationStack())"
LL_EXACT_AMBIG_DETECTION() ::= <<self._interp.predictionMode = PredictionMode.LL_EXACT_AMBIG_DETECTION>>
ParserToken(parser, token) ::= <%<parser>.<token>%>
Production(p) ::= <%<p>%>
Result(r) ::= <%<r>%>
ParserPropertyMember() ::= <<
@members {
def Property(self):
@ -247,6 +97,8 @@ def Property(self):
}
>>
ParserPropertyCall(p, call) ::= "<p>.<call>"
PositionAdjustingLexer() ::= <<
def resetAcceptPosition(self, index, line, column):
@ -298,10 +150,11 @@ def isIdentifierChar(c):
>>
BasicListener(X) ::= <<
@parser::members {
class LeafListener(MockListener):
def visitTerminal(self, node):
print(node.symbol.text)
}
>>
WalkListener(s) ::= <<
@ -328,45 +181,57 @@ class MyRuleNode(ParserRuleContext):
>>
TokenGetterListener(X) ::= <<
@parser::members {
class LeafListener(MockListener):
def exitA(self, ctx):
if ctx.getChildCount()==2:
print(ctx.INT(0).symbol.text + ' ' + ctx.INT(1).symbol.text + ' ' + str_list(ctx.INT()))
else:
print(str(ctx.ID().symbol))
}
>>
RuleGetterListener(X) ::= <<
@parser::members {
class LeafListener(MockListener):
def exitA(self, ctx):
if ctx.getChildCount()==2:
print(ctx.b(0).start.text + ' ' + ctx.b(1).start.text + ' ' + ctx.b()[0].start.text)
else:
print(ctx.b(0).start.text)
}
>>
LRListener(X) ::= <<
@parser::members {
class LeafListener(MockListener):
def exitE(self, ctx):
if ctx.getChildCount()==3:
print(ctx.e(0).start.text + ' ' + ctx.e(1).start.text + ' ' + ctx.e()[0].start.text)
else:
print(ctx.INT().symbol.text)
}
>>
LRWithLabelsListener(X) ::= <<
@parser::members {
class LeafListener(MockListener):
def exitCall(self, ctx):
print(ctx.e().start.text + ' ' + str(ctx.eList()))
def exitInt(self, ctx):
print(ctx.INT().symbol.text)
}
>>
ImportVisitor(X) ::= ""
BasicVisitor(X) ::= ""
WalkVisitor(s) ::= ""
LRWithLabelsVisitor(X) ::= ""
RuleGetterVisitor(X) ::= ""
LRVisitor(x) ::= ""
TokenGetterVisitor(x) ::= ""
DeclareContextListGettersFunction() ::= <<
def foo():
s = SContext()
@ -387,8 +252,7 @@ Declare_pred() ::= <<def pred(self, v):
>>
Invoke_pred(v) ::= <<self.pred(<v>)>>
isEmpty ::= [
"": true,
default: false
]
ParserTokenType(t) ::= "Parser.<t>"
ContextRuleFunction(ctx, rule) ::= "<ctx>.<rule>"
StringType() ::= "String"
ContextMember(ctx, subctx, member) ::= "<ctx>.<subctx>.<member>"

2
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeLexers/LexerDelegatorInvokesDelegateRule.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeLexers/LexerDelegatorInvokesDelegateRule.stg
View File

@ -22,7 +22,7 @@ Errors() ::= ""
masterGrammar(grammarName, slaveGrammarName) ::= <<
lexer grammar <grammarName>;
import <slaveGrammarName>;
import S;
B : 'b';
WS : (' '|'\n') -> skip ;
>>

2
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeLexers/LexerDelegatorRuleOverridesDelegate.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeLexers/LexerDelegatorRuleOverridesDelegate.stg
View File

@ -20,7 +20,7 @@ Errors() ::= ""
masterGrammar(grammarName, slaveGrammarName) ::= <<
lexer grammar <grammarName>;
import <slaveGrammarName>;
import S;
A : 'a' B {<writeln("\"M.A\"")>} ;
WS : (' '|'\n') -> skip ;
>>

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/BringInLiteralsFromDelegate.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/BringInLiteralsFromDelegate.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/CombinedImportsCombined.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/CombinedImportsCombined.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatesSeeSameTokenType.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatesSeeSameTokenType.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorAccessesDelegateMembers.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorAccessesDelegateMembers.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRule.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRule.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRuleWithArgs.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRuleWithArgs.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRuleWithReturnStruct.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesDelegateRuleWithReturnStruct.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesFirstVersionOfDelegateRule.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorInvokesFirstVersionOfDelegateRule.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesDelegate.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesDelegate.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesDelegates.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesDelegates.stg
View File

4
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesLookaheadInDelegate.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/DelegatorRuleOverridesLookaheadInDelegate.stg
View File

@ -33,6 +33,6 @@ slaveGrammar(grammarName) ::= <<
parser grammar S;
type_ : 'int' ;
decl : type_ ID ';'
| type_ ID init ';' {<write("\"JavaDecl: \" + $text")>};
init : '=' INT;
| type_ ID init_ ';' {<write("\"JavaDecl: \" + $text")>};
init_ : '=' INT;
>>

2
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportLexerWithOnlyFragmentRules.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportLexerWithOnlyFragmentRules.stg
View File

@ -25,7 +25,7 @@ Errors() ::= ""
masterGrammar(grammarName, slaveGrammarName) ::= <<
grammar <grammarName>;
import <slaveGrammarName; separator=", ">;
import Unicode;
program : 'test' 'test';

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportedGrammarWithEmptyOptions.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportedGrammarWithEmptyOptions.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportedRuleWithAction.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/ImportedRuleWithAction.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/KeywordVSIDOrder.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/CompositeParsers/KeywordVSIDOrder.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/AmbigYieldsCtxSensitiveDFA.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/AmbigYieldsCtxSensitiveDFA.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/AmbiguityNoLoop.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/AmbiguityNoLoop.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFATwoDiffInput.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFATwoDiffInput.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA_1.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA_1.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA_2.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/CtxSensitiveDFA_2.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity_1.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity_1.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity_2.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/ExprAmbiguity_2.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse_1.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse_1.stg
View File

0
runtime-testsuite/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse_2.stg → runtime-testsuite-legacy/resources/org/antlr/v4/test/runtime/templates/FullContextParsing/FullContextIF_THEN_ELSEParse_2.stg
View File

Some files were not shown because too many files have changed in this diff Show More