We can certainly revert to simple locks, but then have to do synchronization differently than the Java target does, which is not in the scope of the current work.
- Finished adding all new ATN related classes (e.g. profiling infos, lexer actions).
- Introduced a global Ref<X> alias for std::shared_ptr<X>. The previous approach (defining a type individually for each class led to cyclic #includes).
- Converted all ::Ref typedefs and uses to the new Ref template (+ a number of std::shared_ptr<> occurances). Work in progress still, all those occurences will be converted for 100% consistency.
- Removed rules no longer used in the Cpp.stg.
- Removed CppTarget.java in old place.
- Converted + added most new LexerAction* classes to XCode project.
- Added changes to a number of classes.
- Class ATN is currently leaking states (delete disabled in d-tor) until I have found a solution for the failing return value optimization from ATNDeserializer::deserialize().
- Added member init in BlockEndState.
Originally the serialized ATN was coded as a wide string literal with embeded Unicode escapes (where necessary). This might fail on Windows however, as VS is very strict when checking Unicode code points and rejects compilation if it finds undefined code points. Hence this generation has been changed to hex numbers instead of that string literal.
- Solved all compilation issues. Updated the antlr4cpp library project. At the moment we only build a static lib. Need to add exports for a DLL. Since we want this library to be compatible with VS 2013 still, we cannot use std::rethrow_with_nested(), hence we do a simple unnested throw in such cases. Starting with VS 2015 this works fully then.
- Added demo application.
- Added parser generation script.
- The needed ANTLR jar is provided now, so it's not needed to build it yourself.
- The generate.sh script has been updated to use the new jar.
- Small update of the readme too.
- All allocations are now checked for proper deallocation.
- Ran LLVM analyzer over the runtime but it found mostly valid stuff and did not find non-freed allocations I left undeleted by intention. So it's not worth much.
- Added move and copy assignment operator overloading, as well as a copy c-tor to ATN class to avoid a copy (and to be able to free content properly) after deserialization.
- Some clean up.
- Removed ultra simple test grammar + parser. No longer needed.
- Removed long list of keywords from (regular) test grammar.
- Fixed a number of toString() methods to get better debug output.
- Moved Ref typedefs from Declarations.h to the individual classes as defining them on the forward declarations totally confuses the XCode debugger.
- Removed reference to the owning ATN in an ATNState. We cannot guarantee to have the correct address there due to the way the states are created. The reference is not needed anyway.
- ATNDeserializationOptions now has verifyATN set by default (as in the Java target).
- Had to add a workaround for a weird situation: static initialization in ATNDeseralizer stopped working for no apparent reason. Need to investigate this.
- Added a few support methods to the CPPUtils, mostly to ease debug output creation.
- Added console listener by default to the listeners list (as done in the Java target).
- Fixed translation mistakes in the CommonTokenStream class.
- Fixed some memory leaks and exception handling bugs.
- Removed a few unused classes.
- More raw pointers to smart pointers conversion: RuleContext, ParserRuleContext, ParseTree, Token, ParseTreeWalker, Tree...
- BitSet is now used directly instead of all those dynamic allocations and is a derived class instead of a composite.
- Replced ATNState equals with == operator overload.
- Correct wrong iterator over ATNConfigsets.
- Added utilitiy function that mimics Java's generic toString().
- Exceptions are now consistently thrown by value and captured by reference. C++11 exception_ptr and nested_exception are used when exception references are neeeded or when implementing the equivalent of Java's nesting.
- The is<> helper didn't handle properly (const) references, which is now explicitly handled. Added new unit tests for that.
- Fixed a number of places where a catch all was used to implement a "finally" (which hides exceptions).
- Changed exceptions to hold (temporary) raw pointers instead of shared pointers, as otherwise it is tried to free wrapped pointers which might just be references to static objects. Might later be updated again when we continue with removing raw pointers.
- Some smaller fixes.
- The generated simple parser now runs through without any error (yet, it doesn't do anything useful).
- ANTLR C++ target template:
- Added getListener and genVisitor bool members to ANTLR's LexerFile + ParserFile classes, so can use them in the template.
- Made addition of listener #include dependent on the new genListener member, which allows to run parser generation without listeners/visitors.
- Added an even simpler grammar to ease debugging while getting the lib into a working state.
- Added helper template is<> to ease frequent type checks (for value types, ref types and shared_ptr). Added some unit tests for that as well.
- Changed the MurmurHash::hashCode() function to take shared_ptr as this is the only variant we need. Had to change the MurmurHash unit tests for that.
- Removed conflicting IntStream::_EOF (and other variants). We use the C runtime EOF value instead.
- Changed all references to semantic contexts, prediction context and the prediction context cache to use shared_ptr<>. Created *Ref typedefs to simplify usage.
- Adjusted the C++ string templates for that.
- Fixed a number of memory leaks + some cleanup.
- Reworked the exception hierarchy to conform with the Java hierarchy (where we mimic that). Ultimative base class is std::exception, which uses std::string (char* actually) for messages, so all exceptions use std::string for that as well. Consider that as first step to rework the entire lib to use std::string instead of std::wstring (with utf-8 for full Unicode support).
- Removed ASSERTException + TODOException and fixed the places where they were used.
- Removed ANTLRException, which was only an intermediate layer without an equivalent on Java side.
- Replaced some equals() calls by == (with defined operator overloading).
- Enhanced Arrays::equals() to ensure it compiles only if the actual types being compared support the != operator (both value + reference types).
- Made the Recognizer class template free by using plain polymorphism. Some adjustments were need also in the Cpp template to support that. Could convert the .inl file to .cpp then.
- Added IntervalSet unit tests. Fixed a few bugs found by that.
- Enhanced the demo grammar so that we use as many as possible template rules from Cpp.stg. Still not fully done it seems.
- Fixed bugs in size determination for arrays (vectors now).
- Simplified PredictionContext and SemanticContext (one template parameter less).
- Removed no longer used Utils.h/cpp. Fixed CPPUtils.h/cpp.
- Extended LexerXXCommand template rules to take a new grammar parameter (code gen has been updated) as we need this context in the Cpp target. This change requires to update all existing templates! Cannot do here as this is an old revision.
- Some cleanup.
While testing Interval() and Interval::of() I found that the latter is twice as slow as the normal object creation. Seems caching single element intervals doesn't have the same impact as in Java (quite the opposite), so I removed Interval::off and the interval cache.
The MurmurHash implementation was actually for a 32bit platform, so I added a 64 bit version too (stripped down from 128 bit MurmurHash3). Tests cannot directly check the correctness of the algorithm, but duplicate checks over 300K hashs (for short input, which is more prone to duplicates than longer input) showed there are no duplicates. So I take it that the code is good.
Fixed a hash creation bug in PredictionContext.cpp.
- Added first real unit test set and enable code coverage collection in XCode (for ANTLRInputStream).
- Reworked ANTLRFileStream::load, which is now more flexible (supports Unicode BOM + 3 possible encodings), can load from Unicode file names and has almost no platform code.
- Enabled strict data size and sign checks in XCode (clang) and fixed a million places...
- Started converting int to size_t where it makes more sense.
- Started working on const correctness.
- Fixed a ton of memory leaks.
- The ATN and ATNConfigSet classes now entirely work as value types. Same for Interval(Set). These seem to be the most critical data structures (ATNConfig + ATNState are pending).
- The abstract IntSet class is gone now.
- Murmur hash code now works with size_t instead of int (need to add unit tests for that).
- Fixed a number of TODOs and other smaller things.
- The Cpp template now properly handles grammar rule return values.
- Reworked the ATNConfigSet + the config lookup implementation it uses. The new implementation no longer needs the hand written Array2DHashSet class but instead relies now on std::unordered_set with custom hasher and comparer classes.
- Fixed a bug where the ATNConfigSet was deriving from std::set while in the original Java code it only implements the Set interface (not the config set itself is a set but the config lookup is). As a consequence all iterations over ATNConfigSet now iterate over ATNConfigSet->configLookup.
- Removed the Any class as it didn't solve the problems we had mind.
- Removed the no longer necessary Array2DHashSet, AbstractEqualityComparer and ObjectEqualityComparer classes.
- Instead there is a new ConfigLookup implementation with a templated config lookup implementation.
- Removed ATNConfig::equals, as this is already implement in the == operator overloading. So the operator is used instead where needed.
ATNs are top level structures, which are created and kept by parser/lexer classes (or their simulator equivalents). Hence there are now value types in their controlling class and passed around as const &.
IRecognizer was a template class without real need, which has been changed to make it a simple interface easily usable without having to find C++ hacks for fancy Java wildcard generics.
The Any class is loosly modelled after boost::Any and allows us to use equals() and hashCode() functions to be used where we have no common base class (like Java's Object class). By introducing this class we can replace all void* occurances that would otherwise not work.
- Reformatted every single file to have a consistent indentation style using only space chars, with 2 chars per indentation. Reduced huge indentation due to deep namespace nesting by not indenting namespaces.
- Reduced #include usage to a minimum.
- Made copyright header the first entry in all files.
- Moved the previously mac-only prefix file (antlrcpp-Prefix.h) to the runtime. It can now be used by all platforms and includes all necessary standard headers.
- Removed a number of unused files.
- ATN deserialization finally works.
- Changed a number of pointer to STL classes to just the STL classes and pass them around by const & where necessary.
Mike Lischke
Mike Lischke