- Fixed an endless recursion in Any, caused by the removal of one of the (apparently) unneeded copy constructors. As it turned out both are required. That leads to a warning in VS about a duplicate copy c-tor, which had to be suppressed therefore.
- Raised the warning level to W4 in both VS 2013 and VS 2015 and fixed all warnings resulting from that.
The translation from Java generics to templates in C++ lead to the need of virtual template functions, which is not supported by C++. Instead we use now the Any class for results of visits and no longer need templates for that part.
- Only require JRE
- Support out of tree build from antlr repostitory
- Support Superproject build with ExternalAntlr4Cpp cmake module
ExternalAntlr4Cpp module has quickstart documentation for people to be
able to start working quicly with antlr4cpp from the base demo sources
see source file for example.
Document the minimum cmake version needed to build C++ target (if compiling with cmake). Also, fix a missing space in cmake command between directory path and defining where the ANTLR .jar is located.
The root cause was that ATNConfigSet was not using he required custom hashing strategy for ParserATNSimulator.
The commit includes a number of additional fixes, related to code that was never executed before due to the root cause.
A similar issue is also likely to exist in the JavaScript runtime, I'll fix it later.
- The previous approach to load and convert UTF-8 data via a stream didn't work well, so I replaced that with a simple load-to-buffer + convert buffer from UTF-8 to UTF-32.
- Removed deleted Token.cpp file from XCode project.
- Created deployment script for Windows + updated doc/releasing-antlr.md.
- Created projects for both VS2013 and VS2015 to be used by the deployment script.
- Fixed trouble with a bug in VS2015 where std::codecvt_utf8<char32_t> is not properly supported.
- Fixed a few #include paths + a number of warnings.
- Settled on a final library name scheme: base part is "libantlr4-runtime" on MacOS + Linux. The extension determines the type (.a static lib, .dylib dynamic lib in MacOS, .so dynamic lib in Linux). No more mention of target language (cpp) or type (static) in the lib name. On Windows we omit the lib prefix, so the name becomes: antlr4-runtime.dll + antlr4-runtime.lib. We may later want to add version information there, but doing that automatically is difficult.
- Updated XCode project and CMakeLists.txt file for the new naming scheme.
- Added deployment scripts for source code (for Linux + iOS) and MacOS.
- Added C++ section in docs/releasing-antlr.md.
No need to use shared_ptr for management. Listeners are, like the other main classes (parser, lexer, input stream etc.) provided by the application and hence managed there.
In order to lower the overhead when passing around Token instances via smart pointers and because the ownership is clear (token streams own them), these instances can be passed as raw pointers.
No need for the ConfigLookup(Impl) class as we only need a way to determine if a config has been added to a set or not. This can be done much simpler by using individual hash codes produced by either ATNConfigSet or OrderedATNConfigSet (idea taken from C# runtime).
ATNConfigSet instance follow a clear ownership (all owned by DFAState instances, which stay alive all time). Hence we can use a unique_ptr for memory management instead and pass raw pointers around where needed.
All shared_ptr<> now use const& for function parameters to avoid constant copies + locks. Ownership and lifetime control is still ensured by the owning containers. Code templates have been updated as well.
- Changed namespace chain (org::antlr::v4::runtime) to just antlr4 in all files.
- Fixed runtime tests for that.
- Added conversion of the xpath code, which compiles now (no tests, tho, as there are runtime tests for it).
- Removed TestRig stuff. That doesn't work in C++.
- A few remaining changes had to be done for the C++ runtime tests which now completely pass.
- Added a runtime testing overview document.
- Added a description of C++ target.
- Updated the ANTLR release document.
- Allow loading new input instead of setting up a new input stream.
- Made is<> template function inline, for hopefully faster execution.
- A few other fixes in stream + lexer classes.
- Reverted another rename from context to _localctx in the Cpp.stg file as this is used for certain actions.
That seemed to be a find/replace accident. Apparently this change turns
the tests ...
* testTokenMismatch
* testTokenMismatch2
* testNoViableAltAvoidance
* testNoViableAlt
* testExprAmbiguity_2
... to green. Although I am not sure about testExprAmbiguity_2 as that
one doesn't have mis[Mm]atched in its output AFAICS.
- The ATN simular generated lots of duplicate DFAStates that consumed a lot of memory, due to missing hashing/== functionality. By using an own hasher/comparer for the unordered_map in the DFA this problem could be solved. Memory consumption is now 100% stable on a low level (e.g. 115MB for a fairly complex parser) and doesn't increase anymore after the warmup phase.
- Fixed also a number of things reported by the clang static analyzer.
- Fixed a bug in Parser.cpp where the stop token in the current context would be set twice overriding a previously set value.
- Optimized token range check via ATN::nextTokens, by not duplicating the stored interval set for a given state.
- Slightly improved IntervalSet::contains() by moving some tests out of the loop and not using intermediate variables.
- Reverted the change from RecognitionException to exception_ptr (and removed the make_exception_ptr() call). This is not needed and only leads to object slicing. Instead we check directly the given exception object for the individual handling in reportError() and use std::current_exception() to get the exception pointer for our contexts.
- Removed precompiled header stuff from cmake file for the same reasons as I did in the XCode project (Visual Studio still pending).
In order to ease including the antlr runtime in other projects the include structure has been changed:
- Removed precompiled header usage (only OSX, VS + cmake still need updates).
- Created umbrella header antlr4-runtime.h that contains everything needed in a target application.
- Changed all includes to use relative paths, so it is enough to add the src folder to the header search path in an application.
- Also fixed a smaller issue in the C++ template wrt the serialized ATN storage for large grammars.
- The upper char limit in Lexer.h was wrong. Now correctly set to 10FFFF.
- The lexer ATN simulator now uses lower + upper limit of char32_t instead of hardcoded values.
- Added a little hack to Interval, where a range ending with 0xFFFF will automatically be extended to 0x10FFFF. This is necessary until ANTLR generates full Unicode intervals. This hack allows to include Unicode chars beyond the BMP in char classes in a lexer.
- Fixed an error display issue in Lexer.
- In order to support UTF-8 the input streams now support loading data from UTF-8 strings and convert them internally to UTF32.
- Additionally, all the toString() functions that (unnecessarily) used wstring are now on string as well. Only some corner cases remain where we still have std::wstring (ATNSerializer).
- The transition classes use size_t instead of int now for vocabulary matching.
- The max char value in the lexer has been increased to 1FFFE, to allow matching the full Unicode range. This is also used in the Interval class to e.g. to negate sets.
- Renamed Strings.h to StringUtils.h, to avoid a conflict with an OSX header (used with Obj-C compilation, e.g. in a project that uses the runtime).
- Also the ArrayPredictionContext has parent references (like the SingletonPredictionContext) which need to be strong refs or we may lose some of the parent contexts if they are not held somewhere else.
- Don't use WCHAR_MIN as lower bounds for char input checks, it's not 0 as you would expect but -2G, making so EOF succeed even though it should fail this check.
- Don't resize the parents array when merging parents + return states in PredictionContext or we will try to access parents outside of the available range.
- Use an unordered set when merging parents in PredictionContext, so that the normal equality pattern kicks in when comparing contexts.
- Some parameters in AbstractParseTreeVisitor where wrongly outcommented where only the param name should.
C++ template:
- No longer include the DEFAULT_MODE in the generated lexer (it's defined elsewhere).
- Corrected formatting and finished some reference rules that were not done yet.
- Reversed the meaning of grammar sections members + declarations to maintain the same meaning for members between C++ and Java target. Now members are placed in the public section of a class, while declarations use the private section. This change helps to minimize language specific parts in grammar actions.
- Removed deleted cpp files from XCode project.
- Cpp.stg:
- Renamed all occurences of "result" back to "_localctx" as they appear in the Java.stg file. While the name "result" better fits the purpose the rename increases differences between targets, hence it was taken back, so we can use the same actions in all targets.
- TokenPropertyRef_text is now complete.
Some warnings in generated files cannot be fixed in a general way because usage of parameters depends on the grammar, hence we suppress unused-parameter warnings in the grammar (for lexer and parser files).
- @parser::context or @lexer::context are now also accepted for code that should be placed directly before the class declaration (e.g. additional types, like enums etc.)
- Reverted the removal of explicit EOF handling. Thought we can just live with the EOF macro, but that doesn't work out, so we go with the same approach as the ANTLR3 C target: #undef EOF and use EOF member constants as in the original Java code.
- Fixed a crash when trying to create a hash from a null parent in PredictionContext.cpp.
- Generated token and rule enums are now placed in the lexer/parser classes which allows to use them without qualfication within those classes, making so actions in a grammar more language independent. Outside code still has to use e.g. TParser::ID to access them.
- Made some lambda capture lists more explicit. Need to test yet if we can just use a default capture instead.
- Compiling with cmake brought up quite a number of new warnings. Some of them have to be disabled yet (dollar in identifier, four char constant, overloaded virtual). The others have all been fixed.
- Updated the README to include build instruction.
termination condition.
The PredictionMode::hasSLLConflictTerminatingPrediction method aims to
create ATNConfig objects from another ATNConfig and SemanticContext
objects. In case of the Python targets, the initialization
happened without keyword arguments. Since the called __init__
method had default values set for all the parameters, the parameter
substitution worked by indices. As a consequence, the first ATNConfig
parameter was wrongly interpreted as an ATNState and the SemanticContext as
an alternative. The patch fixes this by adding the missing keywords.
C++ target:
- More sections are now supported: pre + post include, declarations, definitions (in addition to header and members).
- Added specific variants of these sections for (base)listener + (base)visitor files (baselistenerpreinclude etc.).
Had to add named sections to VisitorFile.java + ListenerFile.java.
Also added the new namedActions parameter to all target stg files where needed.
The runtime folder now contains the individual project files for each platform (and associated files/folders). The actual source code moved down one level to folder src.
The default install name is /usr/libs/antlrcpp.dylib which makes running the demo from command line not working (location and install name of the dylib differ). Since the probability that the lib is placed in an app bundle is much higher than that the lib is being installed in the system, the default has been changed to just the dylib name. This can be changed to anything else if needed in a concrete project.
- Updated XCode project (removed obsolete cpp files refs).
- Set channel datatype in lexer + tokens to size_t.
- Removed one unit test that no longer works, now that we require all objects in a vector given to murmur hash code computation to support the hashCode() function.
Removed a few unneeded cpp files on the way.
Note: building as DLL produces many of the well known compiler warnings C4251 (type needs to have a DLL interface ..) because the runtime uses many STL classes in exported classes. I tried exporting all those types via explicit template instantion, but once I hit unordered_map I gave up. Just crazy what you have to export just to make this map export properly (and I already had like 50 exports already in place). So at the end I disabled this warning on project level. So make sure you build this DLL and all binaries using it with exact the same compiler and linker settings (same C++ runtime etc.).
Nathan Burles
Mike Lischke
Mike Lischke
Tim O'Callaghan
Will Faught
Nic30
Eric Vergnaud
Chris Heller
nuuman
Peter Boyer
Wolfgang Johannes Kohnen
Marcin Szalowicz
Renata Hodovan