diff --git a/.gitignore b/.gitignore index 2068ad7b6..70169b06c 100644 --- a/.gitignore +++ b/.gitignore @@ -1,18 +1,16 @@ # Maven build folders -/target/ -/tool/target/ -/runtime/Java/target/ -/antlr4-maven-plugin/target/ +target/ # Ant build folders -/build/ -/dist/ +build/ +dist/ +lib/ +user.build.properties # NetBeans user configuration files -/nbactions.xml -/tool/nbactions.xml -/runtime/Java/nbactions.xml -/antlr4-maven-plugin/nbactions.xml +nbactions*.xml +/nbproject/private/ +*/nbproject/private/ # IntelliJ projects *.iml @@ -20,5 +18,13 @@ *.iws .idea/ +# Eclipse projects +.classpath +.project +.settings/ + # Profiler results *.hprof + +# Playground +/tool/playground/ diff --git a/CHANGES.txt b/CHANGES.txt index 233e78cd4..6edc3aeb3 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -1,5 +1,297 @@ ANTLR v4 Honey Badger +August 31, 2013 + +* Lots of little fixes thanks to Coverity Scan + +August 7, 2013 + +* [BREAKING CHANGE] Altered left-recursion elimination to be simpler. Now, + we use the following patterns: + + * Binary expressions are expressions which contain a recursive invocation of + the rule as the first and last element of the alternative. + + * Suffix expressions contain a recursive invocation of the rule as the first + element of the alternative, but not as the last element. + + * Prefix expressions contain a recursive invocation of the rule as the last + element of the alternative, but not as the first element. + +There is no such thing as a "ternary" expression--they are just binary +expressions in disguise. + +The right associativity specifiers no longer on the individual tokens because +it's done on alternative basis anyway. The option is now on the individual +alternative; e.g., + + e : e '*' e + | e '+' e + | e '?' e ':' e + | e '=' e + | INT + ; + +If your language uses a right-associative ternary operator, you will need +to update your grammar to include on the alternative operator. + +This also fixes #245 and fixes #268: + +https://github.com/antlr/antlr4/issues/245 +https://github.com/antlr/antlr4/issues/268 + +To smooth the transition, is still allowed on token references +but it is ignored. + +June 30, 2013 -- 4.1 release + +June 24, 2013 + +* Resize ANTLRInputStream.data after reading a file with fewer characters than + bytes +* Fix ATN created for non-greedy optional block with multiple alternatives +* Support Unicode escape sequences with indirection in JavaUnicodeInputStream + (fixes #287) +* Remove the ParserRuleContext.altNum field (fixes #288) +* PredictionContext no longer implements Iterable +* PredictionContext no longer implements Comparable +* Add the EPSILON_CLOSURE error and EPSILON_OPTIONAL warning +* Optimized usage of closureBusy set (fixes #282) + +June 9, 2013 + +* Add regression test for #239 (already passes) + +June 8, 2013 + +* Support list labels on a set of tokens (fixes #270) +* Fix associativity of XOR in Java LR grammar (fixes #280) + +June 1, 2013 + +* DiagnosticErrorListener includes rule names for each decision in its reports +* Document ANTLRErrorListener and DiagnosticErrorListener (fixes #265) +* Support '\uFFFF' (fixes #267) +* Optimize serialized ATN + +May 26, 2013 + +* Report errors that occur while lexing a grammar (fixes #262) +* Improved error message for unterminated string literals (fixes #243) + +May 24, 2013 + +* Significantly improve performance of JavaUnicodeInputStream.LA(1) + +May 20, 2013 + +* Generate Javadoc for generated visitor and listener interfaces and classes +* Fix unit tests + +May 18, 2013 + +* Group terminals in Java grammars so ATN can collapse sets +* Improved Java 7 support in Java grammars (numeric literals) +* Updated error listener interfaces +* Support detailed statistics in TestPerformance + +May 17, 2013 + +* Add JavaUnicodeInputStream to handle Unicode escapes in Java code +* Proper Unicode identifier handling in Java grammars +* Report file names with lexer errors in TestPerformance + +May 14, 2013 + +* Use a called rule stack to prevent stack overflow in LL1Analyzer +* Use 0-based indexing for several arrays in the tool +* Code simplification, assertions, documentation + +May 13, 2013 + +* Unit test updates to ensure exceptions are not hidden + +May 12, 2013 + +* Updates to TestPerformance + +May 5, 2013 + +* Updated several classes to use MurmurHash 3 hashing + +May 1, 2013 + +* Added parse tree JTree to TreeViewer (Bart Kiers) + +April 30, 2013 + +* Updated TestPerformance to support parallelization across passes + +April 24, 2013 + +* Remove unused stub class ParserATNPathFinder +* Remove ParserInterpreter.predictATN +* Remove DFA.getATNStatesAlongPath +* Encapsulate implementation methods in LexerATNSimulator and ParserATNSimulator +* Updated documentation +* Simplify creation of new DFA edges +* Fix handling of previously cached error edges +* Fix DFA created during forced-SLL parsing (PredictionMode.SLL) +* Extract methods ParserATNSimulator.getExistingTargetState and + ParserATNSimulator.computeTargetState. + +April 22, 2013 + +* Lazy initialization of ParserATNSimulator.mergeCache +* Improved hash code for DFAState +* Improved hash code with caching for ATNConfigSet +* Add new configuration parameters to TestPerformance +* Update Java LR and Java Std to support Java 7 syntax + +April 21, 2013 + +* Add new configuration parameters to TestPerformance + +April 18, 2013 + +* Must check rule transition follow states before eliminating states in + the ATN (fixes #224) +* Simplify ParserATNSimulator and improve performance by combining execDFA and + execATN and using DFA edges even after edge computation is required + +April 15, 2013 + +* Fix code in TestPerformance that clears the DFA + +April 12, 2013 + +* Improved initialization and concurrency control in DFA updates +* Fix EOF handling in edge case (fixes #218) + +April 4, 2013 + +* Improved testing of error reporting +* Fix NPE revealed by updated testing method +* Strict handling of redefined rules - prevents code generation (fixes #210) +* Updated documentation in Tool + +March 27, 2013 + +* Avoid creating empty action methods in lexer (fixes #202) +* Split serialized ATN when it exceeds Java's 65535 byte limit (fixes #76) +* Fix incorrect reports of label type conflicts across separated labeled outer + alternatives (fixes #195) +* Update Maven plugin site documentation + +March 26, 2013 + +* Fix bugs with the closureBusy set in ParserATNSimulator.closure +* Fix handling of empty options{} block (fixes #194) +* Add error 149 INVALID_LEXER_COMMAND (fixes #190) +* Add error 150 MISSING_LEXER_COMMAND_ARGUMENT +* Add error 151 UNWANTED_LEXER_COMMAND_ARGUMENT +* Updated documentation in the Parser and RecognitionException classes +* Refactored and extensively documented the ANTLRErrorStrategy interface and + DefaultErrorStrategy default implementation +* Track the number of syntax errors in Parser.notifyErrorListeners instead of in + the error strategy +* Move primary implementation of getExpectedTokens to ATN, fixes #191 +* Updated ATN documentation +* Use UUID instead of incremented integer for serialized ATN versioning + +March 7, 2013 + +* Added export to PNG feature to the parse tree viewer + +March 6, 2013 + +* Allow direct calls to left-recursive rules (fixes #161) +* Change error type 146 (EPSILON_TOKEN) to a warning (fixes #180) +* Specify locale for all format operations (fixes #158) +* Fix generation of invalid Unicode escape sequences in Java code (fixes #164) +* Do not require escape for $ in action when not followed by an ID start char + (fixes #176) + +February 23, 2013 + +* Refactoring Target-related classes to improve support for additional language + targets + +February 22, 2013 + +* Do not allow raw newline characters in literals +* Pair and Triple are immutable; Triple is not a Pair + +February 5, 2013 + +* Fix IntervalSet.add when multiple merges are required (fixes #153) + +January 29, 2013 + +* don't call process() if args aren't specified (Dave Parfitt) + +January 21, 2013 -- Release 4.0 + +* Updated PredictionContext Javadocs +* Updated Maven site documentation +* Minor tweaks in Java.stg + +January 15, 2013 + +* Tweak error messages +* (Tool) Make TokenVocabParser fields `protected final` +* Fix generated escape sequences for literals containing backslashes + +January 14, 2013 + +* Relax parser in favor of errors during semantic analysis +* Add error 145: lexer mode must contain at least one non-fragment rule +* Add error 146: non-fragment lexer rule can match the empty string + +January 11, 2013 + +* Updated error 72, 76; added 73-74 and 136-143: detailed errors about name + conflicts +* Report exact location for parameter/retval/local name conflicts +* Add error 144: multi-character literals are not allowed in lexer sets +* Error 134 now only applies to rule references in lexer sets +* Updated error messages (cleanup) +* Reduce size of _serializedATN by adding 2 to each element: new representation + avoids embedded values 0 and 0xFFFF which are common and have multi-byte + representations in Java's modified UTF-8 + +January 10, 2013 + +* Add error 135: cannot assign a value to list label: $label + (fixes antlr/antlr4#128) + +January 2, 2013 + +* Fix EOF handling (antlr/antlr4#110) +* Remove TREE_PARSER reference +* Additional validation checks in ATN deserialization +* Fix potential NPE in parser predicate evaluation +* Fix termination condition detection in full-context parsing + +January 1, 2013 + +* Updated documentation +* Minor code cleanup +* Added the `-XdbgSTWait` command line option for the Tool +* Removed method override since bug was fixed in V3 runtime + +December 31, 2012 + +* I altered Target.getTargetStringLiteralFromANTLRStringLiteral() so that + it converts \uXXXX in an ANTLR string to \\uXXXX, thus, avoiding Java's + conversion to a single character before compilation. + +December 16, 2012 + +* Encapsulate some fields in ANTLRMessage +* Remove ErrorType.INVALID +* Update error/warning messages, show all v3 compatibility messages + December 12, 2012 * Use arrays instead of HashSet to save memory in SemanticContext.AND/OR diff --git a/LICENSE.txt b/LICENSE.txt index 47a177082..49a546774 100644 --- a/LICENSE.txt +++ b/LICENSE.txt @@ -1,5 +1,5 @@ [The "BSD license"] -Copyright (c) 2012 Terence Parr, Sam Harwell +Copyright (c) 2013 Terence Parr, Sam Harwell All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/README.txt b/README.txt index 8d5a5822b..8aac63ffd 100644 --- a/README.txt +++ b/README.txt @@ -1,64 +1,92 @@ ANTLR v4 -Terence Parr, parrt at cs usfca edu +Terence Parr, parrt@cs.usfca.edu ANTLR project lead and supreme dictator for life University of San Francisco INTRODUCTION -Hi and welcome to the Honey Badger 4.0 release of ANTLR! +Hi and welcome to the Honey Badger 4.1 release of ANTLR! INSTALLATION -$ cd /usr/local/lib -$ curl -O --silent http://www.antlr.org/download/antlr-4.0-complete.jar +UNIX -Or just download from http://www.antlr.org/download/antlr-4.0-complete.jar -and put it somewhere rational for your operating system. +0. Install Java (version 1.6 or higher) -You can either add to your CLASSPATH: +1. Download -$ export CLASSPATH=".:/usr/local/lib/antlr-4.0-complete.jar:$CLASSPATH" + $ cd /usr/local/lib + $ curl -O http://www.antlr4.org/download/antlr-4.1-complete.jar -and launch org.antlr.v4.Tool directly: + Or just download in browser using URL: + + http://www.antlr4.org/download/antlr-4.1-complete.jar + + and put it somewhere rational like /usr/local/lib. + +2. Add antlr-4.1-complete.jar to your CLASSPATH: + + $ export CLASSPATH=".:/usr/local/lib/antlr-4.1-complete.jar:$CLASSPATH" + + Is also a good idea to put this in your .bash_profile or whatever your + startup script is. + +3. Create aliases for the ANTLR Tool, and TestRig. + + $ alias antlr4='java -jar /usr/local/lib/antlr-4.1-complete.jar' + $ alias grun='java org.antlr.v4.runtime.misc.TestRig' + +WINDOWS (Thanks to Graham Wideman) + +0. Install Java (version 1.6 or higher) + +1. Download http://antlr.org/download/antlr-4.1-complete.jar + Save to your directory for 3rd party Java libraries, say C:\Javalib + +2. Add antlr-4.1-complete.jar to CLASSPATH, either: + + * Permanently: Using System Properties dialog > Environment variables > + Create or append to CLASSPATH variable + + * Temporarily, at command line: + SET CLASSPATH=C:\Javalib\antlr-4.1-complete.jar;%CLASSPATH% + +3. Create short convenient commands for the ANTLR Tool, and TestRig, + using batch files or doskey commands: + + * Batch files (in directory in system PATH) + + antlr4.bat: java org.antlr.v4.Tool %* + run.bat: java org.antlr.v4.runtime.misc.TestRig %* + + * Or, use doskey commands: + + doskey antlr4=java org.antlr.v4.Tool $* + doskey grun =java org.antlr.v4.runtime.misc.TestRig $* + +TESTING INSTALLATION + +Either launch org.antlr.v4.Tool directly: $ java org.antlr.v4.Tool -ANTLR Parser Generator Version 4.0 +ANTLR Parser Generator Version 4.1 -o ___ specify output directory where all output is generated -lib ___ specify location of .tokens files ... or use -jar option on java: -$ java -jar /usr/local/lib/antlr-4.0-complete.jar -ANTLR Parser Generator Version 4.0 +$ java -jar /usr/local/lib/antlr-4.1-complete.jar +ANTLR Parser Generator Version 4.1 -o ___ specify output directory where all output is generated -lib ___ specify location of .tokens files ... -You can make a script, /usr/local/bin/antlr4: - -#!/bin/sh -java -cp "/usr/local/lib/antlr4-complete.jar:$CLASSPATH" org.antlr.v4.Tool $* - -On Windows, you can do something like this (assuming you put the -jar in C:\libraries) for antlr4.bat: - -java -cp C:\libraries\antlr-4.0-complete.jar;%CLASSPATH% org.antlr.v4.Tool %* - -You can also use an alias - -$ alias antlr4='java -jar /usr/local/lib/antlr-4.0-complete.jar' - -Either way, say just antlr4 to run ANTLR now. - -The TestRig class is very useful for testing your grammars: - -$ alias grun='java org.antlr.v4.runtime.misc.TestRig' EXAMPLE -In /tmp/Hello.g4, paste this: +In a temporary directory, put the following grammar inside file Hello.g4: // Define a grammar called Hello // match keyword hello followed by an identifier @@ -87,3 +115,7 @@ the parse tree in LISP notation. BOOK SOURCE CODE http://pragprog.com/titles/tpantlr2/source_code + +GRAMMARS + +https://github.com/antlr/grammars-v4 diff --git a/antlr4-maven-plugin/pom.xml b/antlr4-maven-plugin/pom.xml index a74429bbd..3e518518a 100644 --- a/antlr4-maven-plugin/pom.xml +++ b/antlr4-maven-plugin/pom.xml @@ -41,7 +41,7 @@ org.antlr antlr4-master - 4.0-SNAPSHOT + 4.1.1-SNAPSHOT antlr4-maven-plugin @@ -49,6 +49,7 @@ ANTLR 4 Maven plugin Maven plugin for ANTLR 4 grammars + http://www.antlr.org 3.0 @@ -75,7 +76,7 @@ org.apache.maven maven-plugin-api - 3.0.4 + 3.0.5 compile @@ -88,7 +89,7 @@ org.codehaus.plexus plexus-compiler-api - 2.0 + 2.2 - - - - - - - - - - - - - - - - - - - - + + + + + - - - + + + + + + + + + + + + + + + + + + + + - - parse grammars - - - - - - - - - - - - - - + + + - + + + + + + + + + - codegen grammars - - - - - - - - - - + + + + + + + + + + + + + + + + + + + + + - + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + - + - - - - - - - - - + + + + + + + - + @@ -163,14 +182,7 @@ build.sysclasspath=ignore - + diff --git a/contributors.txt b/contributors.txt index 38c18dcbb..91c887522 100644 --- a/contributors.txt +++ b/contributors.txt @@ -52,3 +52,6 @@ YYYY/MM/DD, github id, Full name, email 2012/09/18, sharwell, Sam Harwell, sam@tunnelvisionlabs.com 2012/10/10, stephengaito, Stephen Gaito, stephen@percepitsys.co.uk 2012/11/23, maguro, Alan Cabrera, adc@toolazydogs.com +2013/01/29, metadave, Dave Parfitt, diparfitt@gmail.com +2013/03/06, bkiers, Bart Kiers, bkiers@gmail.com +2013/08/20, cayhorstmann, Cay Horstmann, cay@horstmann.com diff --git a/pom.xml b/pom.xml index d4b191fcf..ed26a8501 100644 --- a/pom.xml +++ b/pom.xml @@ -10,12 +10,17 @@ 4.0.0 org.antlr antlr4-master - 4.0-SNAPSHOT + 4.1.1-SNAPSHOT pom ANTLR 4 ANTLR 4 Master Build POM http://www.antlr.org + 1992 + + ANTLR + http://www.antlr.org + @@ -43,6 +48,15 @@ + + Jim Idle + jimi@idle.ws + http://www.linkedin.com/in/jimidle + + Developer - Maven Plugin + + + @@ -60,25 +74,65 @@ ${java6.home}/lib/rt.jar ${bootclasspath.java6} ${bootclasspath.java6} + true + + + antlr-discussion + https://groups.google.com/forum/?fromgroups#!forum/antlr-discussion + + + GitHub Issues https://github.com/antlr/antlr4/issues - git://github.com/antlr/antlr4.git + https://github.com/antlr/antlr4/tree/master scm:git:git://github.com/antlr/antlr4.git scm:git:git@github.com:antlr/antlr4.git + + + sonatype-oss-release + + + + org.apache.maven.plugins + maven-compiler-plugin + + + default-compile + + + ${bootclasspath.compile} + + + + + default-testCompile + + + ${bootclasspath.testCompile} + + + + + + + + + + org.apache.maven.plugins maven-compiler-plugin - 3.0 + 3.1 src true @@ -93,7 +147,6 @@ 1.6 -Xlint:-serial - ${bootclasspath.compile} @@ -105,7 +158,6 @@ 1.6 -Xlint:-serial - ${bootclasspath.testCompile} @@ -113,37 +165,6 @@ - - org.apache.maven.plugins - maven-source-plugin - 2.2.1 - - - attach-sources - - jar - - - - - - - org.apache.maven.plugins - maven-javadoc-plugin - 2.9 - - - attach-javadocs - - jar - - - true - - - - - org.apache.maven.plugins maven-jar-plugin @@ -157,19 +178,34 @@ + + org.apache.maven.plugins + maven-surefire-plugin + 2.15 + + + + org.apache.maven.plugins + maven-source-plugin + + 2.2.1 + + + + org.apache.maven.plugins + maven-javadoc-plugin + + 2.9 + + true + + + org.apache.maven.plugins maven-gpg-plugin + 1.4 - - - sign-artifacts - verify - - sign - - - diff --git a/runtime/Java/doxyfile b/runtime/Java/doxyfile index c66ff9966..8949f2f79 100644 --- a/runtime/Java/doxyfile +++ b/runtime/Java/doxyfile @@ -6,7 +6,7 @@ DOXYFILE_ENCODING = UTF-8 PROJECT_NAME = "ANTLR v4 API" PROJECT_NUMBER = 4.0 -OUTPUT_DIRECTORY = api +OUTPUT_DIRECTORY = api/Java CREATE_SUBDIRS = NO OUTPUT_LANGUAGE = English BRIEF_MEMBER_DESC = YES @@ -33,7 +33,7 @@ MULTILINE_CPP_IS_BRIEF = NO DETAILS_AT_TOP = NO INHERIT_DOCS = YES SEPARATE_MEMBER_PAGES = NO -TAB_SIZE = 8 +TAB_SIZE = 4 ALIASES = OPTIMIZE_OUTPUT_FOR_C = NO OPTIMIZE_OUTPUT_JAVA = YES @@ -113,7 +113,7 @@ VERBATIM_HEADERS = YES #--------------------------------------------------------------------------- # configuration options related to the alphabetical class index #--------------------------------------------------------------------------- -ALPHABETICAL_INDEX = NO +ALPHABETICAL_INDEX = YES COLS_IN_ALPHA_INDEX = 5 IGNORE_PREFIX = #--------------------------------------------------------------------------- @@ -126,6 +126,7 @@ HTML_HEADER = HTML_FOOTER = HTML_STYLESHEET = HTML_ALIGN_MEMBERS = YES +HTML_DYNAMIC_SECTIONS = YES GENERATE_HTMLHELP = NO CHM_FILE = HHC_LOCATION = @@ -134,8 +135,8 @@ BINARY_TOC = NO TOC_EXPAND = NO DISABLE_INDEX = NO ENUM_VALUES_PER_LINE = 4 -GENERATE_TREEVIEW = NO -TREEVIEW_WIDTH = 250 +GENERATE_TREEVIEW = YES +TREEVIEW_WIDTH = 210 #--------------------------------------------------------------------------- # configuration options related to the LaTeX output #--------------------------------------------------------------------------- @@ -214,9 +215,9 @@ MSCGEN_PATH = /Applications/Doxygen.app/Contents/Resources/ HIDE_UNDOC_RELATIONS = YES HAVE_DOT = YES CLASS_GRAPH = YES -COLLABORATION_GRAPH = YES -GROUP_GRAPHS = YES -UML_LOOK = YES +COLLABORATION_GRAPH = NO +GROUP_GRAPHS = NO +UML_LOOK = NO TEMPLATE_RELATIONS = NO INCLUDE_GRAPH = YES INCLUDED_BY_GRAPH = YES @@ -235,4 +236,4 @@ DOT_CLEANUP = YES #--------------------------------------------------------------------------- # Configuration::additions related to the search engine #--------------------------------------------------------------------------- -SEARCHENGINE = NO +SEARCHENGINE = YES diff --git a/runtime/Java/nb-configuration.xml b/runtime/Java/nb-configuration.xml index 2d74a8819..8ecf24501 100644 --- a/runtime/Java/nb-configuration.xml +++ b/runtime/Java/nb-configuration.xml @@ -6,6 +6,10 @@ The configuration is intended to be shared among all the users of project and therefore it is assumed to be part of version control checkout. Without this configuration present, some functionality in the IDE may be limited or fail altogether. --> + + rule's + validator +