Commit Graph

1915 Commits

Author SHA1 Message Date
Mike Lischke 0e5b501aa8 Merge branch 'feature/templates' into feature/templates-org 2016-04-23 16:42:26 +02:00
Mike Lischke 014d9fd593 Merge branch 'master' into feature/templates 2016-04-23 16:05:10 +02:00
Mike Lischke 77d0a6c32a Serialized ATN generation changed.
Originally the serialized ATN was coded as a wide string literal with embeded Unicode escapes (where necessary). This might fail on Windows however, as VS is very strict when checking Unicode code points and rejects compilation if it finds undefined code points. Hence this generation has been changed to hex numbers instead of that string literal.
2016-04-23 11:10:12 +02:00
Mike Lischke f292d14abc The parser is parsing for the first time.
- Removed ultra simple test grammar + parser. No longer needed.
- Removed long list of keywords from (regular) test grammar.
- Fixed a number of toString() methods to get better debug output.
- Moved Ref typedefs from Declarations.h to the individual classes as defining them on the forward declarations totally confuses the XCode debugger.
- Removed reference to the owning ATN in an ATNState. We cannot guarantee to have the correct address there due to the way the states are created. The reference is not needed anyway.
- ATNDeserializationOptions now has verifyATN set by default (as in the Java target).
- Had to add a workaround for a weird situation: static initialization in ATNDeseralizer stopped working for no apparent reason. Need to investigate this.
- Added a few support methods to the CPPUtils, mostly to ease debug output creation.
- Added console listener by default to the listeners list (as done in the Java target).
- Fixed translation mistakes in the CommonTokenStream class.
- Fixed some memory leaks and exception handling bugs.
2016-04-17 13:13:15 +02:00
Mike Lischke eb0241f767 Another refactoring round.
- Removed a few unused classes.
- More raw pointers to smart pointers conversion: RuleContext, ParserRuleContext, ParseTree, Token, ParseTreeWalker, Tree...
- BitSet is now used directly instead of all those dynamic allocations and is a derived class instead of a composite.
- Replced ATNState equals with == operator overload.
- Correct wrong iterator over ATNConfigsets.
- Added utilitiy function that mimics Java's generic toString().
2016-04-13 19:05:56 +02:00
Mike Lischke 7f8ad7bd2d Applied consistent exception model and fixed is<> helper.
- Exceptions are now consistently thrown by value and captured by reference. C++11 exception_ptr and nested_exception are used when exception references are neeeded or when implementing the equivalent of Java's nesting.
- The is<> helper didn't handle properly (const) references, which is now explicitly handled. Added new unit tests for that.
- Fixed a number of places where a catch all was used to implement a "finally" (which hides exceptions).
- Changed exceptions to hold (temporary) raw pointers instead of shared pointers, as otherwise it is tried to free wrapped pointers which might just be references to static objects. Might later be updated again when we continue with removing raw pointers.
- Some smaller fixes.
- The generated simple parser now runs through without any error (yet, it doesn't do anything useful).

- ANTLR C++ target template:
  - Added getListener and genVisitor bool members to ANTLR's LexerFile + ParserFile classes, so can use them in the template.
  - Made addition of listener #include dependent on the new genListener member, which allows to run parser generation without listeners/visitors.
2016-04-09 16:36:52 +02:00
Mike Lischke d1b59ca5af Next overhaul (more TODOs fixed)
- Added an even simpler grammar to ease debugging while getting the lib into a working state.
- Added helper template is<> to ease frequent type checks (for value types, ref types and shared_ptr). Added some unit tests for that as well.
- Changed the MurmurHash::hashCode() function to take shared_ptr as this is the only variant we need. Had to change the MurmurHash unit tests for that.
- Removed conflicting IntStream::_EOF (and other variants). We use the C runtime EOF value instead.
- Changed all references to semantic contexts, prediction context and the prediction context cache to use shared_ptr<>. Created *Ref typedefs to simplify usage.
- Adjusted the C++ string templates for that.
- Fixed a number of memory leaks + some cleanup.
2016-04-06 15:07:25 +02:00
Mike Lischke 1ca5b38868 Fixed some TODOs + exceptions.
- Reworked the exception hierarchy to conform with the Java hierarchy (where we mimic that). Ultimative base class is std::exception, which uses std::string (char* actually) for messages, so all exceptions use std::string for that as well. Consider that as first step to rework the entire lib to use std::string instead of std::wstring (with utf-8 for full Unicode support).
- Removed ASSERTException + TODOException and fixed the places where they were used.
- Removed ANTLRException, which was only an intermediate layer without an equivalent on Java side.
- Replaced some equals() calls by == (with defined operator overloading).
- Enhanced Arrays::equals() to ensure it compiles only if the actual types being compared support the != operator (both value + reference types).
- Made the Recognizer class template free by using plain polymorphism. Some adjustments were need also in the Cpp template to support that. Could convert the .inl file to .cpp then.
2016-03-31 18:32:44 +02:00
parrt b92f22769e [maven-release-plugin] prepare for next development iteration 2016-03-30 16:25:49 -07:00
parrt 567fcc6311 [maven-release-plugin] prepare release antlr4-master-4.5.3 2016-03-30 16:25:45 -07:00
parrt cd0913712a set code string versions to 4.5.3 2016-03-30 16:21:50 -07:00
parrt b78e0e9c1e add test and a guard in action translator. minor. Relates to https://github.com/antlr/antlr4/issues/1143 2016-03-30 11:17:57 -07:00
Mike Lischke 29dedd17c4 New unit tests + template enhancements + more memory handling work
- Added IntervalSet unit tests. Fixed a few bugs found by that.
- Enhanced the demo grammar so that we use as many as possible template rules from Cpp.stg. Still not fully done it seems.
- Fixed bugs in size determination for arrays (vectors now).
- Simplified PredictionContext and SemanticContext (one template parameter less).
- Removed no longer used Utils.h/cpp. Fixed CPPUtils.h/cpp.
- Extended LexerXXCommand template rules to take a new grammar parameter (code gen has been updated) as we need this context in the Cpp target. This change requires to update all existing templates! Cannot do here as this is an old revision.
- Some cleanup.
2016-03-30 20:11:05 +02:00
parrt 9e98714a1d Add ability to set parse tree internal node super class with option contextSuperClass. Provide impl in Java target that has altNum backing field. Add test across targets to set/get alt num. Fixes #1152. 2016-03-30 11:00:47 -07:00
parrt 7d16438a19 only gen "set stop token" if finally/after actions exist. 2016-03-30 09:48:33 -07:00
Terence Parr b05d86c150 Merge pull request #1131 from gagern/setStopToken
Set stop token in postamble
2016-03-30 16:43:07 +00:00
Terence Parr 69ff2669ee Merge pull request #1117 from ericvergnaud/python-del-keyword
added missing del keyword to Python targets
2016-03-30 02:29:51 +00:00
parrt 196c4d5913 argh! cutt out too much last push 2016-03-29 16:46:48 -07:00
parrt 3a48085f90 a piece of another branch snuck in! del. 2016-03-29 16:45:17 -07:00
parrt eeea98c00b tweak to michaelpj's PR; alter name. Indicate this fixes #1048. 2016-03-29 15:28:36 -07:00
parrt f2cbc6eb7c Merge branch 'left-recursive-prefix-alternatives' of git://github.com/michaelpj/antlr4 into michaelpj-left-recursive-prefix-alternatives 2016-03-29 15:23:59 -07:00
parrt 1beb8feb4f add "return" to python3 keywords 2016-03-29 14:44:17 -07:00
parrt 41a196082d Merge branch 'master' of github.com:antlr/antlr4 2016-03-29 14:40:17 -07:00
Terence Parr 9655a92d50 Merge pull request #1149 from Wingpad/master
Fixing Issue #1148
2016-03-29 21:39:43 +00:00
parrt cd9ed40ab0 Merge branch 'master' of github.com:antlr/antlr4 2016-03-28 15:32:42 -07:00
Martin von Gagern 2c71663a97 Fix Javadoc errors
This fixes all errors (but not the numerous warnings) reported by
Oracle JDK 1.8.0_72 in a simple run of “mvn javadoc:javadoc”.
2016-03-28 23:46:18 +02:00
parrt e4a4253219 more impl of get/set alt num; update doc 2016-03-28 14:10:29 -07:00
Mike Lischke 3f78367457 Next overhaul
- Added first real unit test set and enable code coverage collection in XCode (for ANTLRInputStream).
- Reworked ANTLRFileStream::load, which is now more flexible (supports Unicode BOM + 3 possible encodings), can load from Unicode file names and has almost no platform code.
- Enabled strict data size and sign checks in XCode (clang) and fixed a million places...
- Started converting int to size_t where it makes more sense.
- Started working on const correctness.
- Fixed a ton of memory leaks.
- The ATN and ATNConfigSet classes now entirely work as value types. Same for Interval(Set). These seem to be the most critical data structures (ATNConfig + ATNState are pending).
- The abstract IntSet class is gone now.
- Murmur hash code now works with size_t instead of int (need to add unit tests for that).
- Fixed a number of TODOs and other smaller things.

- The Cpp template now properly handles grammar rule return values.
2016-03-27 22:01:03 +02:00
Justin Szaday 93b1fa53b4 Fixing Issue #1148, Python Target Generates Invalid Code With Rule Named 'Return' 2016-03-25 22:16:54 -05:00
Mike Lischke e97820a27e Reworked handling of ATN instances throught the code + definition of IRecognizer.
ATNs are top level structures, which are created and kept by parser/lexer classes (or their simulator equivalents). Hence there are now value types in their controlling class and passed around as const &.

IRecognizer was a template class without real need, which has been changed to make it a simple interface easily usable without having to find C++ hacks for fancy Java wildcard generics.
2016-03-24 11:49:44 +01:00
Mike Lischke 6df5d025bf Complete formatting overhaul.
- Reformatted every single file to have a consistent indentation style using only space chars, with 2 chars per indentation. Reduced huge indentation due to deep namespace nesting by not indenting namespaces.
- Reduced #include usage to a minimum.
- Made copyright header the first entry in all files.
- Moved the previously mac-only prefix file (antlrcpp-Prefix.h) to the runtime. It can now be used by all platforms and includes all necessary standard headers.
- Removed a number of unused files.
2016-03-20 17:21:46 +01:00
Mike Lischke fec65477d8 ATN deserialization + some initialization and memory leak fixes
- ATN deserialization finally works.
- Changed a number of pointer to STL classes to just the STL classes and pass them around by const & where necessary.
2016-03-19 17:18:06 +01:00
Mike Lischke 11571fa092 A few more adjustments to make the merged changes work with this revision of antlr. 2016-03-18 15:27:17 +01:00
Mike Lischke 1672bc0739 Added all changes done so far. Since we are here on a very old revision I cannot simply merge, so all files have manually been copied and we have no history for these changes. 2016-03-18 14:22:42 +01:00
Mike Lischke 815856bdf0 Initial demo app setup + backend changes.
- The demo now uses a real setup to parse something and print output.
- Replaced empty UUID implementation by the guid implementation from Graeme Hill. Fixed uuid handling in a few places.
- Removed some obsolete (and mostly empty) lib files.
- Mac: the XCode project now does regenerate the files only after a grammar change, not always.
- Simplified ATNDeserializer + ATNSerializer and fixed a few things there (e.g. feature determination, duplicate copy op of the input).
- Removed some deprecated functions from ATNSimulator.h
- Fixed some bugs (e.g. uninitialized vars + leaks).
- Corrected a mistake I did in CppTarget which lead to wrong serialized ATN output.
- Cpp template:
  - Added proper static init code generation for Lexer + Parser classes.
  - Added some missing functions.
2016-03-18 09:59:09 +01:00
Mike Lischke 7e09002ebc First working version of a demo app that uses the new C++ target.
- Created a new command line target in the XCode project. Win + Linux yet outstanding.
- Reorganized the C++ runtime folder structure a bit
  * Put everything in a new folder "runtime"
  * Added a "demo" folder for the demo grammar + app
  * Renamed Apple folder to Mac in demo folder
  * Added a script with some descriptions to run parser regeneration (via jar or classes). This is also used in the XCode project to regenerate the files.
  * Moved all C++ runtime files up in the folder hierarchy. No need to mimic the deep nesting from Java.
- Some adjustments here and there in the C++ runtime for consistency.
- Overhaul of the Cpp.stg file to produce compilable code. Extracted file level templates into a new template Files.stg. Experimented with new named actions (@parser::listenerheader) but the result is not satisfying yet. Need to investigate more.
- Extended ANTLR to produce header files if a target class returns true in the new function needsHeader().
- Added generated folder from demo to .gitignore
- Added myself to contributors file + maven xml.
2016-03-15 21:02:47 +01:00
Mike Lischke 29ee73bd0a Separated top level file rules into an on ST group file.
Started formatting output and adding std::shared_ptr<> where needed.
2016-03-07 18:10:11 +01:00
Martin von Gagern e3e3f2bca0 Set stop token in postamble
This ensures that the stop token has been set before the @after action gets
executed.
2016-03-07 15:03:32 +01:00
Mike Lischke a524ca3232 Corrected package and some imports in CppTarget.java. 2016-03-06 19:34:45 +01:00
Mike Lischke e6b4ea736d Moved CppTarget.java to the target subfolder where the other targets are.
Corrected .gitignore which excluded target folders too generously.
2016-03-06 18:57:28 +01:00
Mike Lischke 1cec262cee Merge branch 'master' into feature/templates 2016-03-06 13:06:47 +01:00
Michael Peyton Jones 5967674531 Track 'prefix' and 'other' alternatives together
These alternatives are not treated differently after they are created,
and we want to retain their alternative ordering.
2016-02-22 13:05:38 +00:00
Eric Vergnaud 99b2d6616f added missing del keyword to Python targets 2016-02-18 23:39:51 +08:00
Terence Parr 184f711753 [maven-release-plugin] prepare for next development iteration 2016-01-30 10:01:53 -08:00
Terence Parr dcca95d4f0 [maven-release-plugin] prepare release 4.5.2 2016-01-30 10:01:48 -08:00
Terence Parr c92ddcbd06 update version to 4.5.2 in source / packaging stuff. 2016-01-30 09:54:47 -08:00
Terence Parr 1438ab817f why oh why does intellij skip changes when i push? 2015-12-08 11:38:35 -08:00
Terence Parr 99563b116e merging KvanTTT changes. 2015-12-08 11:38:08 -08:00
Terence Parr 6068a4a2c7 Merge pull request #1054 from abego/patch-4
Possible NullPointerException in ActionTranslater#translateActionChunk
2015-11-25 09:05:46 -08:00
Terence Parr 36d8d41def Merge pull request #1046 from abego/extraNullCheckInTestRig
Extra 'null' check in TestRig#process
2015-11-25 09:04:43 -08:00