Commit Graph

1728 Commits

Author SHA1 Message Date
Terence Parr da9186ed63 Merge pull request #134 from parrt/eval-preds-when-delaying-errors
When delaying error reporting because prediction escapes the decision en...
2014-04-02 09:41:09 -07:00
Sam Harwell ea4676b18a adaptivePredict never returns -1; it could return ATN.INVALID_ALT_NUMBER 2014-04-01 21:26:48 -05:00
Sam Harwell 8c8d4d754e Remove incorrectly generated call to adaptivePredict
This particular call to adaptivePredict did not correctly match the decision number to the preceding setState() call. In addition, since this code is generated only for a positive closure, the only possible alternative is the one that enters the loop; error handling is deferred to the alternative(s) within the body of the closure.
2014-04-01 21:26:47 -05:00
Terence Parr 69d4b423e1 When delaying error reporting because prediction escapes the decision entry rule, we should evaluate predicates to turn off any configurations that are invalid semantically. After that, we can collect the list of alternatives that are satisfied with the input thus far and then choose the minimum as before. Fixes #529 2014-04-01 16:49:30 -07:00
Sam Harwell f34adc916a Added Grammar.undefineRule to support undefining rules prior to AST construction 2014-03-29 16:13:36 -05:00
Sam Harwell 5a56cc1147 Documented Grammar.defineRule, and updated it to return true if the rule was added to the Grammar 2014-03-29 16:13:35 -05:00
Sam Harwell 2729f176b2 Add option TestPerformance.REPORT_SECOND_STAGE_RETRY 2014-03-29 13:12:35 -05:00
Sam Harwell 5be18445cd [maven-release-plugin] prepare for next development iteration 2014-03-25 10:57:46 -05:00
Sam Harwell 943aaa5d24 [maven-release-plugin] prepare release 4.2.1 2014-03-25 10:57:44 -05:00
Sam Harwell d8efb4ad44 Updated build to use latest releases of ANTLR 3 and ST 4 2014-03-25 10:48:00 -05:00
Sam Harwell dcff40df90 Generate complete documentation with the Tool 2014-03-25 10:43:59 -05:00
Sam Harwell 85f09126e2 Don't include the annotation processor registration in the shaded (complete) jar 2014-03-24 20:43:55 -05:00
Sam Harwell 6642a9222d Merge pull request #516 from sharwell/fix-515
Fix usage of assoc=right in Java-LR.g4 (fixes #515)
2014-03-24 20:05:21 -05:00
Sam Harwell 014785b70a Fix usage of assoc=right in Java-LR.g4 (fixes #515) 2014-03-24 20:05:19 -05:00
Terence Parr b4b91970bd Merge pull request #495 from aphyr/fix-load-string-grammar-npe
BasicSemanticChecks.checkGrammarName: allow null filenames.
2014-03-24 13:20:54 -07:00
Sam Harwell ea088eb893 Add compile error 159: RESERVED_RULE_NAME (fixes #500) 2014-03-23 21:49:27 -05:00
Sam Harwell 0ed651cbc6 Add compiler warning 158: FRAGMENT_ACTION_IGNORED (fixes #472) 2014-03-23 18:02:09 -05:00
Sam Harwell 2b3ef648f8 Add compiler warning 157: UNRECOGNIZED_ASSOC_OPTION (fixes #489) 2014-03-23 17:47:40 -05:00
Sam Harwell 480544b4fe Improve wording in description of error 152 2014-03-23 16:31:46 -05:00
Sam Harwell 3e054df98b Do not process grammars following an error during parsing (fixes #507) 2014-03-23 16:31:14 -05:00
Sam Harwell 7daae3a89b Add compiler error 156: unterminated string literal (improves reporting for #507) 2014-03-23 16:30:49 -05:00
Terence Parr 794e4a0b7b Merge pull request #501 from sharwell/fix-433
Fix 433
2014-03-23 11:30:00 -07:00
Terence Parr e66f663618 Merge pull request #503 from sharwell/fix-469
Fix 469
2014-03-23 11:25:59 -07:00
Sam Harwell cb0119e761 Add regression test for issue #509 2014-03-23 11:34:36 -05:00
Sam Harwell 6fc77cd88d Add regression test for #469 2014-03-20 09:20:31 -05:00
Sam Harwell f1692591cc Updated RuleFunction.addContextGetters to support multiple alternatives with the same label (fixes #433) 2014-03-20 08:35:18 -05:00
Sam Harwell 3c70fea32a Add regression test for issue #433 2014-03-20 08:35:17 -05:00
Kyle Kingsbury a74005972d BasicSemanticChecks.checkGrammarName: allow null filenames.
checkGrammarName assumes that all grammars come from actual files, but
grammars loaded using Tool.parseGrammarFromString use input streams
without source file names. This patch changes checkGrammarName to only verify
that grammar names and file names correspond when a file name is
available.
2014-03-18 16:54:43 -07:00
Sam Harwell d498292680 Add regression test for issue #461 (interpreter starting with LR rule) 2014-02-06 20:46:32 -06:00
Sam Harwell 06ab99d16d [maven-release-plugin] prepare for next development iteration 2014-02-04 17:19:12 -06:00
Sam Harwell 5e05b71e8b [maven-release-plugin] prepare release 4.2 2014-02-04 17:19:11 -06:00
Sam Harwell 8ef0f590bb Updated build to include annotation processor in shaded Tool jar 2014-02-04 17:18:48 -06:00
Sam Harwell 2eb6f45bd9 Fix generated code Javadoc incompatible with Java 8 2014-01-26 11:49:20 -06:00
Sam Harwell e63f95268f Update javac options during testing to address overly verbose warnings 2014-01-23 07:25:41 -06:00
Sam Harwell 0f25d1c4cf Merge pull request #436 from sharwell/fix-test-output
Fix test failure due to change error message text
2014-01-21 19:18:53 -08:00
Sam Harwell ef5b6eb34e Fix test failure due to change error message text 2014-01-21 21:17:17 -06:00
Sam Harwell 608c6a8a3a Update several usages of @NotNull and @Nullable based on warnings from NullUsageProcessor 2014-01-21 21:08:11 -06:00
Sam Harwell b6f6c99c3c Updated documentation for ErrorType 2014-01-19 21:00:56 -06:00
Sam Harwell a55901bc0d We are working on the 4.2 release 2014-01-19 13:43:57 -06:00
Terence Parr 9b85e9c636 Merge pull request #426 from sharwell/fix-175
Support grammar files encoded with UTF-8 with a byte order mark
2014-01-17 10:22:47 -08:00
Sam Harwell 14107cc40a Add regression test for expression performance (closes #192) 2014-01-17 08:54:43 -06:00
Sam Harwell da0864a2f8 Support grammar files encoded with UTF-8 with a byte order mark (fixes #175) 2014-01-17 08:37:49 -06:00
Sam Harwell 664d66d937 Fix incorrect combineMax implementation (fixes #299) 2014-01-16 20:35:03 -06:00
Sam Harwell 9889de87ac Add regression test highlighting issue #299 2014-01-16 20:35:03 -06:00
Sam Harwell 85a923f61e Fix IndexOutOfBoundsException when using ANTLR with Java 8 (fixes #337) 2014-01-16 17:13:18 -06:00
Sam Harwell dbfb230b11 Cleaner output in TestPerformance when checksum calculation is disabled 2014-01-16 16:11:21 -06:00
Sam Harwell f2966aaa80 Improve support for m2e (Maven/Eclipse integration) (fixes #319) 2014-01-16 15:44:12 -06:00
Sam Harwell 4fc32e1f7c Fix options not getting properly applied to extracted lexer in combined grammar (fixes #372) 2014-01-16 10:34:55 -06:00
Sam Harwell 05b8c22833 Syntax errors while lexing stop grammar compilation process (fixes #308) 2014-01-15 21:39:19 -06:00
Sam Harwell 511c60fe7b Merge pull request #417 from sharwell/fix-370
Fix 370
2014-01-15 12:33:11 -08:00
Sam Harwell 83ad25ea1d Add regression test for #370 2014-01-15 14:31:41 -06:00
Terence Parr d0b8ccefdc set filename 2014-01-15 12:14:11 -08:00
Terence Parr 9f28fca583 split action/command tests from lexer tests; add tests for new action stuff from Sam. 2014-01-15 11:36:37 -08:00
Terence Parr 3c51f7ad7b Add tests for ungrammatical tree patterns. Fixes #413 2014-01-15 11:15:31 -08:00
Sam Harwell 5e9b869f1d Updated lexer to support encoding commands in the ATN and interpreting them with LexerInterpreter 2014-01-14 20:17:37 -06:00
Terence Parr d4275ab53a Range check; apparently string literal map not set when we create interp vs code gen. 2014-01-14 19:26:39 -06:00
Terence Parr bc4f3a72aa add helper methods and rename some internal Tool methods 2014-01-14 19:26:32 -06:00
Sam Harwell 967e74aa0d Pass the ATN through the ATNSerializer prior to creating an interpreter (fixes #403) 2014-01-13 20:17:04 -06:00
Sam Harwell 266f7276d1 Initial implementation of a "precedence DFA" (fixes #400) 2014-01-09 18:11:17 -06:00
Sam Harwell bf22a0af5b Add regression test for #398 2014-01-09 06:51:49 -06:00
Terence Parr 6b2817f8bb get last not first when get() finds multiple matching nodes. 2013-12-20 12:47:19 -08:00
Terence Parr 8c5d088eb7 Merge pull request #378 from sharwell/polish
Tree patterns polish
2013-12-19 17:24:42 -08:00
Sam Harwell fb1880d82c Move ATNSerializer to runtime 2013-12-19 19:07:24 -06:00
Sam Harwell bc59f30857 Use ATNDeserializer methods instead of deprecated ATNSimulator methods 2013-12-19 19:07:23 -06:00
Sam Harwell 2a9a716c53 Remove unnecessary methods ParseTreeMatch.getText() and failed() (use getTree().getText() and !succeeded() instead) 2013-12-19 19:06:22 -06:00
Sam Harwell 72675075cf Remove unnecessary testing constructor 2013-12-19 19:06:21 -06:00
Sam Harwell 75b8174dc8 Clean up the result caching for getTokenTypeMap and getRuleIndexMap 2013-12-19 19:06:20 -06:00
Sam Harwell df61690758 Clean up the caching of ATN instances with bypass alternatives 2013-12-19 19:06:19 -06:00
Terence Parr 9ca6bf9bd3 fix null pointer bug with rule "a : a;" 2013-12-19 16:35:37 -08:00
Terence Parr bd91dc166d add getTokenTypeMap(), getRuleIndexMap() to recognizer. Gen new fields for that an ATN with bypass alts. Then methods for that: getATNWithBypassAlts(). Big changes to interface for ParseTreeMatch; create Parser.compileParseTreePattern() method. Convert rule names to rule indexes. 2013-11-24 14:04:46 -08:00
Terence Parr 4c52a103e1 cleanup 2013-11-22 11:31:59 -08:00
Terence Parr b2ec85d14d updated comments, cleaned up the API, made helper routines. 2013-11-22 11:08:16 -08:00
Terence Parr 168bce79d3 working on api 2013-11-22 09:55:40 -08:00
Terence Parr 4fdd520c2e got labels working 2013-11-21 16:01:02 -08:00
Terence Parr 1e83557819 reorg to pass round a match object in matches_() 2013-11-21 10:30:37 -08:00
Terence Parr e836544d30 add test for hidden tokens 2013-11-21 09:55:26 -08:00
Terence Parr e68757717f pass a lexer and a parser to the tree pattern match or instead of the classes. 2013-11-21 09:32:45 -08:00
Terence Parr 33d11709df Revert "fix comment"
This reverts commit e8f577e162.
2013-11-21 08:42:28 -08:00
Terence Parr e8f577e162 fix comment 2013-11-21 08:42:02 -08:00
Terence Parr 40789babf5 got tree pattern matching working on my test cases using a parser interpreter not been generated parser to compile patterns like "<ID> = <expr>;". It uses Sam's alteration to the ATN to have bypass alternatives for <expr> tags. 2013-11-20 17:54:57 -08:00
Terence Parr 2394b38995 added state to be consistent with enterRule(); most importantly, we need to set the state when we enter a recursive rule, which we are not doing at the moment. This was not a problem before because we never could directly call a recursive rule and the outer rule set the current state. Now that we are using recursive rules as start rules, we need to set the state. 2013-11-20 17:53:51 -08:00
Terence Parr 2791dd5619 Merge branch 'master' into tree-patterns 2013-11-20 16:57:16 -08:00
Terence Parr 8c57233db7 mv to runtime. damn git! 2013-11-20 16:53:49 -08:00
Terence Parr 1c71d05074 Merge branch 'master' into tree-patterns 2013-11-20 16:49:27 -08:00
Terence Parr cc07f8faec mv to runtime. was ref'ing v3 not v4 Token. 2013-11-20 16:47:17 -08:00
Terence Parr 578dcd0799 reformat 2013-11-20 16:36:34 -08:00
Terence Parr c7e9a410e9 rm unused field _contextStack 2013-11-20 16:35:55 -08:00
Terence Parr 48ce1f1809 Support getting the serialized ATN from a recognizer 2013-11-20 16:22:01 -08:00
Terence Parr d09510a936 Merge branch 'master' into tree-patterns 2013-11-20 16:09:36 -08:00
Terence Parr 41d9d5bfd4 Merge pull request #357 from sharwell/atn-deserializer
ATN deserializer
2013-11-20 14:33:07 -08:00
Terence Parr 42e5343287 Merge branch 'master' into tree-patterns 2013-11-20 14:31:08 -08:00
Terence Parr 5a6709aba6 update changes file, tweak parser interp test to use new api 2013-11-20 14:28:51 -08:00
Terence Parr 02f937b115 Merge branch 'master' into tree-patterns 2013-11-20 14:20:09 -08:00
Sam Harwell b447256f20 Extract ATN deserialization to a new class ATNDeserializer 2013-11-20 09:14:18 -06:00
Sam Harwell 6eb8111756 Generate method stub so external calls directly to LR rules do not require the precedence argument 2013-11-15 16:15:34 -06:00
Terence Parr 8cace4e182 pull master BaseTest 2013-11-14 14:47:49 -08:00
Terence Parr 2f902da3d2 pull master into branch 2013-11-14 14:43:50 -08:00
Sam Harwell 3309c9d85e Fix build warnings 2013-11-14 15:36:01 -06:00
Terence Parr 52981a1d97 Merge pull request #353 from sharwell/parser-interpreter
Parser interpreter
2013-11-14 12:06:45 -08:00
Terence Parr ab931e4449 Merge pull request #97 from sharwell/precedence-predicates
Precedence predicates
2013-11-14 11:54:55 -08:00
Sam Harwell 320fada3e3 When building trees in TestPerformance, compute checksum after the parse using ParseTreeWalker 2013-11-14 13:42:39 -06:00
Sam Harwell 49970a46b0 Improved handling of parse listeners in TestPerformance 2013-11-14 13:42:39 -06:00
Sam Harwell 33043d68c2 Add the USE_PARSER_INTERPRETER flag to TestPerformance 2013-11-14 13:42:38 -06:00
Sam Harwell f6024aae0a Updated interpreter APIs
* Add Grammar.createLexerInterpreter and Grammar.createParserInterpreter
* Rewrite LexerInterpreter to extend Lexer (greatly simplified)
2013-11-14 13:42:38 -06:00
Sam Harwell 508817a706 Fully quality type names from ANTLR 3 2013-11-14 13:42:37 -06:00
Sam Harwell 7165a42d21 Make sure to send the final exit rule event in ParserInterpreter 2013-11-14 13:42:37 -06:00
Sam Harwell e02968f0e7 Fix set and range transitions not matching input in ParserInterpreter 2013-11-14 13:42:36 -06:00
Sam Harwell b2a4bb91a7 Add ParserInterpreter implementation 2013-11-14 13:42:36 -06:00
Terence Parr a3d71db1bc Merge pull request #352 from parrt/xpath
Add XPath to collect parse tree nodes/subtrees.
2013-11-13 16:22:37 -08:00
Sam Harwell b14ca56441 Merge branch 'master' into precedence-predicates 2013-10-10 20:36:38 -05:00
Sam Harwell b80ad5052d Add regression test for issue #334 (already passes) 2013-09-30 22:35:03 -05:00
Terence Parr 8aeb90ef82 mv findAll to XPath; rm old test in playground. 2013-09-16 14:56:35 -07:00
Terence Parr 76633525bf rm comments 2013-09-16 14:50:28 -07:00
Terence Parr 01082414c3 rebuilt XPath using ANTLR itself; added error handling; added ! operator 2013-09-13 13:53:42 -07:00
Terence Parr 09ddded3ed cleaned up test rig. more tests. fix bugs. 2013-09-11 17:49:25 -07:00
Terence Parr 088cd4a9bc got test rig start. added support test rig stuff to exec parser. add findAll to ParseTree interface. 2013-09-11 14:59:53 -07:00
Terence Parr 3fda64f85b got a draft implementation 2013-09-11 11:25:07 -07:00
Terence Parr 8237952cbb stash 2013-09-10 20:11:01 -07:00
Terence Parr 913cda0721 adding more xpath expr types 2013-09-10 19:48:36 -07:00
Terence Parr d2caae46a7 working on xpath 2013-09-10 14:55:40 -07:00
Terence Parr 4055c24171 add comment 2013-09-09 17:30:04 -07:00
Terence Parr 76b4eef60a more unit tests; add <.> 2013-09-09 17:26:16 -07:00
Terence Parr 6a26f0499f a new test 2013-09-09 15:17:27 -07:00
Terence Parr 95aa103dc2 matches text now if not <ID> node 2013-09-09 15:13:37 -07:00
Terence Parr f995e47443 rm ruletag from ParserRuleContext; back to (expr <expr>) for tag in pattern 2013-09-09 14:54:44 -07:00
Terence Parr 90187039f0 foo 2013-09-09 14:45:46 -07:00
Terence Parr 0df4411ea7 got optimized tree back with <expr> as single node 2013-09-09 13:50:33 -07:00
Terence Parr cfac804b57 fix error handling to just match <expr> as token under expr RuleNode 2013-09-08 18:56:07 -07:00
Terence Parr 754348605d create match object for matching 2013-09-05 16:58:37 -07:00
Terence Parr 35e279cef1 change name from T.g4 to X.g4 to avoid weird classloader issue. TLexer lives somewhere in my damn system and I can't find it. 2013-09-05 10:35:20 -07:00
Terence Parr d2f7ba8339 cleanup 2013-09-04 17:25:14 -07:00
Terence Parr 4b87f84b99 got decent test rig 2013-09-04 17:09:14 -07:00
Terence Parr 531c31861a got basic pattern compilation going 2013-09-04 14:48:21 -07:00
Terence Parr 2aade3eecc avoid resource leak and cleanup (Coverity) 2013-08-31 17:46:23 -07:00
Terence Parr fc2dfcda96 dodgy code cleanup (Coverity) 2013-08-31 17:36:16 -07:00
Terence Parr 3e53c0c61d oops; deleted side-effecting "dead code" 2013-08-31 17:05:48 -07:00
Terence Parr 78298bb382 defined compareTo but not equals/hash (Coverity) 2013-08-31 16:56:07 -07:00
Terence Parr 875fa0a31d rm dead code (Coverity) 2013-08-31 16:52:07 -07:00
Terence Parr 8bff26d758 fix cut-paste error (Coverity) 2013-08-31 16:47:33 -07:00
Terence Parr 21248a42ca rm unneeded null ptr check (Coverity) 2013-08-31 16:37:16 -07:00
Terence Parr 1febc36e15 avoid potential null ptr deref (Coverity) 2013-08-31 16:32:59 -07:00
Terence Parr 0ac47437bc rm potential null ptr deref (Coverity) 2013-08-31 16:20:34 -07:00
Terence Parr 9c186c8d52 rm print statement 2013-08-31 12:20:57 -07:00
Sam Harwell 04d2b5b92d Add regression test for #315 (already passes) 2013-08-12 19:26:40 -05:00
Terence Parr 19fce5dd7d change ruleElementOption back to elementOption 2013-08-08 16:12:01 -07:00
Terence Parr 531afe5af8 changed grammar to allow <assoc=right> on | operator for alt. still allowed on token refs but ignored. Simplified left-recursion elemination rules. see CHANGES.TXT Aug 7, 2013. BREAKING CHANGE. Must alter ternary op alts in grammars. 2013-08-08 17:06:15 -05:00
Terence Parr 185ee4e18e add <options> notation to alt right after pipe. rename elementOption to ruleElementOption as it now can match options on alt. Move AltAST to have options. 2013-08-08 17:06:15 -05:00
Terence Parr d041a4828a fix poor function name: setTokenAssoc 2013-08-08 17:05:19 -05:00
Sam Harwell 274120c2b6 Now working on release 4.1.1 2013-07-02 19:11:04 -05:00
Sam Harwell 6236072e3b Documentation updates prior to release 2013-06-30 15:50:56 -05:00
Sam Harwell dbe1390c6f Update referenced maven plugin version numbers 2013-06-30 12:47:26 -05:00
Sam Harwell 05f7fc6500 Update development version to 4.1 (from 4.0.1) 2013-06-30 12:46:41 -05:00
Terence Parr b7133164f4 Merge pull request #294 from sharwell/fix-293
Remove tool/playground from source control, add to .gitignore
2013-06-29 11:58:42 -07:00
Sam Harwell 36acb139b7 Remove the SET_QUALIFIED_ATTR rule from ActionSplitter since it provides no additional functionality (fixes #295) 2013-06-28 10:28:55 -05:00
Sam Harwell 931df811db Remove tool/playground from source control, add to .gitignore (fixes #293) 2013-06-25 14:30:25 -05:00
Sam Harwell b0ddb45654 Merge pull request #284 from sharwell/epsilon-analysis
Add the EPSILON_CLOSURE error and EPSILON_OPTIONAL warning
2013-06-24 13:09:38 -07:00
Sam Harwell 82330a8a78 Support Unicode escape sequences in JavaUnicodeInputStream with indirection (multiple 'u' characters) (fixes #287) 2013-06-24 12:02:42 -05:00
Sam Harwell 40f41e2b1c Fix ATN created for non-greedy optional block with multiple alternatives 2013-06-14 21:05:58 -05:00
Sam Harwell 3bb774a508 Add the EPSILON_CLOSURE error and EPSILON_OPTIONAL warning 2013-06-14 20:53:34 -05:00
Sam Harwell cd73c212fb Add regression test for #239 2013-06-09 14:11:35 -05:00
Sam Harwell 6fe7154c44 Exclusive or operator is left associative (fixes #280) 2013-06-08 21:09:33 -05:00
Sam Harwell ba532cdd49 Support list labels on a set of tokens (fixes #270) 2013-06-08 11:59:47 -05:00
Sam Harwell 2a90413188 Use standard 0-based indexing 2013-06-01 22:07:27 -05:00
Sam Harwell 9167968b74 Only serialize unique sets (smaller serialized ATN) 2013-06-01 22:03:31 -05:00
Sam Harwell dc801ad6a7 Extend the supported input character range to include all UTF-16 code points (specifically, U+FFFF is now supported, fixes #267) 2013-06-01 22:03:18 -05:00
Sam Harwell f1f134c962 DiagnosticErrorListener includes rule names for each decision in its reports 2013-06-01 21:55:31 -05:00
Sam Harwell 2f2fd4585e Improved error message for unterminated string literals (fixes #243) 2013-05-26 13:45:38 -05:00
Sam Harwell d3af4a2f3a Report errors that occur while lexing a grammar (fixes #262) 2013-05-26 13:45:38 -05:00
Sam Harwell 611e5ebf02 Significantly improve the performance of JavaUnicodeInputStream.LA(1) 2013-05-24 09:40:48 -05:00
Sam Harwell 469ebdd6e8 Fix unit tests now that the parser might not compute the conflicting alts BitSet before calling report methods 2013-05-20 14:58:32 -05:00
Sam Harwell 13f121a16f Add @NotNull annotations to listener and visitor methods 2013-05-20 14:20:54 -05:00
Sam Harwell d6c3841b0a Generate Javadoc for the generated listener and visitor interfaces, classes, and methods 2013-05-20 14:19:53 -05:00
Sam Harwell f6255bd5be Add regression test for #248 (already passes) 2013-05-20 13:31:47 -05:00
Sam Harwell ec8457383c Calculate detailed transition statistics per decision 2013-05-18 14:22:09 -05:00
Sam Harwell d8f2a5ce3c Pass more information to ANTLRErrorListener methods 2013-05-18 14:17:36 -05:00
Sam Harwell 8d0a1bbd8d Show additional information about rules and viable alternatives for conflicts, ambiguity, and/or context sensitivity 2013-05-18 11:46:57 -05:00
Sam Harwell aad78ba7d2 Fix transition statistics calculation when a file contains no transitions 2013-05-18 11:40:36 -05:00
Sam Harwell f758e8e24d Some additional lexer tweaks 2013-05-18 11:19:28 -05:00
Sam Harwell 49f9e093e8 Clean up character and string literals in the lexer 2013-05-18 11:18:58 -05:00
Sam Harwell c835fc9a79 Add unambiguous shift and comparison operators to the lexer 2013-05-18 11:15:34 -05:00
Sam Harwell ad2b5876a5 Add separators and operators to the lexer 2013-05-18 11:13:35 -05:00
Sam Harwell 0f4ec397da Add NullLiteral to the lexer 2013-05-18 11:11:56 -05:00
Sam Harwell f5ed71d2b9 Improved support for Java 7 numeric literals 2013-05-18 11:10:53 -05:00
Sam Harwell 9bd3e965ef 'true' and 'false' are both lexed as a BooleanLiteral token 2013-05-18 11:09:22 -05:00
Sam Harwell d9e1289d13 Add keywords as explicit lexer rules; enum and assert are always keywords (could extend JavaLexer and override emit to change this behavior) 2013-05-18 11:06:02 -05:00
Sam Harwell 62cbab4f86 Use explicit groups so the ATN can collapse sets 2013-05-18 11:01:14 -05:00
Sam Harwell 0e5bcc2416 Report file names with lexer errors 2013-05-17 23:57:13 -05:00
Sam Harwell 5948f50288 Proper Unicode identifier handling 2013-05-17 23:56:32 -05:00
Sam Harwell d9d5100105 Use JavaUnicodeInputStream for proper handling of Unicode escape sequences in Java source code 2013-05-17 23:47:59 -05:00
Sam Harwell 1906e4a639 Use IntervalSet.or and Arrays.copyOf instead of duplicating their functionality 2013-05-14 09:58:51 -05:00
Sam Harwell 35ef516e28 Add some assertions 2013-05-14 09:58:02 -05:00
Sam Harwell c0d1fcc110 Use 0-based indexing for decision lookahead so we aren't wasting memory, and array indexing is now consistent with ATN state outgoing transitions indexing 2013-05-14 09:57:04 -05:00
Sam Harwell 32d4c85f0f Rethrow unit test execution exceptions wrapped in a RuntimeException 2013-05-13 20:27:50 -05:00
Sam Harwell af59f35481 Count full context transitions in statistics 2013-05-13 20:27:18 -05:00
Sam Harwell 0652b61a33 Default to FILE_GRANULARITY=true 2013-05-12 18:07:35 -05:00
Sam Harwell 0fac5ce133 Use System.nanoTime for more precise timing results 2013-05-12 17:52:23 -05:00
Sam Harwell 3694d51c4e Don't try to report detailed statistics for empty states (fixes exception in lexer stats reporting) 2013-05-12 17:48:33 -05:00
Sam Harwell 9ea4747524 Call Future.get() to propagate exceptions so it's easier to spot problems 2013-05-12 17:46:55 -05:00
Terence Parr ac7cc7daaf Merge pull request #247 from sharwell/parallel-testing
Parallel testing updates
2013-05-05 12:32:56 -07:00
Sam Harwell d67d924b0d Update additional classes to use MurmurHash hashing 2013-05-05 14:21:42 -05:00
Sam Harwell d75f8e9aa3 Add the DETAILED_DFA_STATE_STATS option to TestPerformance, which shows a breakdown of DFA states by rule (parsers) or mode (lexers) 2013-05-01 22:06:00 -05:00
Sam Harwell d147e9202d Use ->skip instead of ->channel(HIDDEN) to ensure reported token counts correspond to the number of parsed tokens in TestPerformance 2013-05-01 22:02:26 -05:00
Sam Harwell 28abae9cfb Add the TIME_PARSE_ONLY flag to TestPerformance 2013-04-30 22:03:49 -05:00
Sam Harwell cc0d050622 Display the current pass number in TestPerformance result summary (helps monitor progress on very long runs) 2013-04-30 22:00:35 -05:00
Sam Harwell 3558151e56 Support parallelizing TestPerformance across passes instead of across files (controlled by the FILE_GRANULARITY flag) 2013-04-30 21:58:32 -05:00
Sam Harwell 4dfa245898 Added the COMPUTE_TIMING_STATS and TIMING_CUMULATIVE options to TestPerformance (disabled by default) 2013-04-30 21:53:30 -05:00
Sam Harwell 8f7c43fbf6 Support computing and displaying transition statistics 2013-04-30 21:48:51 -05:00
Sam Harwell be57daa108 Remove unnecessary reflection, fix option to not reuse DFA 2013-04-30 21:41:55 -05:00
Sam Harwell 646f673d84 Track token count separately for each pass 2013-04-30 21:35:10 -05:00
Sam Harwell dda9ed9fab Remove unnecessary null checks 2013-04-24 12:48:09 -05:00
Sam Harwell 5821747a42 Add TestPerformance.COMPUTE_TRANSITION_STATS 2013-04-24 12:47:25 -05:00
Sam Harwell f015942b5e * Simplify creation of new DFA edges - only create the target state and let DFA code check for termination conditions
* Fix handling of previously cached error edges (always need to check previous state for reaching a rule stop state)
* Fix DFA created during SLL-only parsing (should be identical to the DFA created by full LL parsing)
2013-04-24 12:29:43 -05:00
Sam Harwell 15577fd21f Remove unnecessary method ParserInterpreter.predictATN 2013-04-24 10:32:20 -05:00
Sam Harwell be83edf6d0 Gather additional parsing statistics from each file 2013-04-22 17:35:16 -05:00
Sam Harwell f1fc42024f explicitConstructorInvocation was broken and unnecessary 2013-04-22 17:31:58 -05:00
Sam Harwell ad41c3b14f Assignments can appear in the middle of a ternary expression 2013-04-22 17:31:29 -05:00
Sam Harwell 4b6d0d9ea2 Support diamond operator, improved support for explicit generic invocations 2013-04-22 17:31:02 -05:00
Sam Harwell df336b4e59 Remove unreferenced rule from Java LR 2013-04-22 17:27:44 -05:00
Sam Harwell 69f22f8012 Allow extraneous ';' in annotation interface declaration (not allowed by grammar, but is allowed by compiler and appears in some source code) 2013-04-22 17:24:44 -05:00
Sam Harwell a670d7d2b0 Support Java 7 "try with resources" and "multi-catch" 2013-04-22 17:23:54 -05:00
Sam Harwell c8701a9434 Support directory filters in TestPerformance, add a variety of additional FilenameFilter implementations 2013-04-22 17:16:49 -05:00
Sam Harwell 210dc31a43 Print the proper statistics when only a subset of available files is parsed 2013-04-22 17:13:56 -05:00
Sam Harwell b9918b9e25 Remove unnecessary null checks 2013-04-22 17:10:07 -05:00
Sam Harwell 5280b68cab Add the TestPerformance.PRELOAD_SOURCES configuration field 2013-04-22 17:08:24 -05:00
Sam Harwell 94b06df72f Add TestPerformance.ENCODING configuration field (default UTF-8) 2013-04-22 16:53:53 -05:00
Sam Harwell e7b84c13dd Add comment to TestPerformance.PREDICTION_MODE configuration field 2013-04-22 16:49:36 -05:00
Sam Harwell f5294204ae Add the TestPerformance.MAX_FILES_PER_PARSE_ITERATION configuration field 2013-04-21 15:00:20 -05:00
Sam Harwell 82f4d07243 Default value for TestPerformance.BAIL_ON_ERROR is now false since it's overridden anyway for first phase of two-stage parsing 2013-04-21 14:56:28 -05:00
Sam Harwell e438b60f37 Add the SHUFFLE_FILES_AT_START and SHUFFLE_FILES_AFTER_ITERATIONS options to TestPerformance 2013-04-21 14:55:31 -05:00
Terence Parr bc4b530b3b tweak my tests for java. 2013-04-20 10:17:25 -07:00
Sam Harwell 7a7f4a7851 Must check rule transition follow states before eliminating states in the ATN (fixes #224) 2013-04-18 14:46:11 -05:00
Sam Harwell bf4f198fbf Fix build error 2013-04-18 14:43:56 -05:00
Sam Harwell 9236477438 Fix code in TestPerformance that clears the DFA 2013-04-15 11:12:16 -05:00
Sam Harwell db85cbc257 Fix EOF handling when closure operation was skipped (fixes #218) 2013-04-12 14:02:24 -05:00
Sam Harwell bb18e14e04 Throw UnsupportedOperationException in not implemented method 2013-04-12 13:20:10 -05:00
Sam Harwell 05f667d2e5 Initialize the elements of the decisionToDFA arrays when the array is created 2013-04-12 13:17:53 -05:00
Sam Harwell f4ae1cf471 Updated documentation 2013-04-04 13:09:54 -05:00
Sam Harwell 4433a57baa Strict handling of redefined rules (do not attempt to generate code), fixes #210 2013-04-04 13:09:31 -05:00
Sam Harwell 7235d71cc0 Fix NPE revealed by updated testing method 2013-04-04 13:07:43 -05:00
Sam Harwell 805430177c More thorough testing of tool error reporting 2013-04-04 13:06:22 -05:00
Sam Harwell 0e5cf32bf4 Merge branch 'fix-195' 2013-03-27 15:02:42 -05:00
Sam Harwell 2242948c03 Merge branch 'fix-76' 2013-03-27 15:02:17 -05:00
Sam Harwell cae9f7bbf1 Merge branch 'fix-202' 2013-03-27 15:02:07 -05:00
Sam Harwell 71cc6fba8a Merge branch 'fix-194' 2013-03-27 15:01:57 -05:00
Sam Harwell f75878b3e6 Merge branch 'fix-196' 2013-03-27 15:01:45 -05:00
Sam Harwell 961f68c865 Fix incorrect reports of label type conflicts for labels aliased across separate *labeled* outer alternatives (fixes #195) 2013-03-27 11:21:32 -05:00
Sam Harwell c592e41637 Split serialized ATN in Java target to ensure string literals are under 65535 bytes limit (fixes #76) 2013-03-27 11:04:28 -05:00
Sam Harwell 7f029a3c80 Avoid creating empty action methods in lexer (fixes #202) 2013-03-27 09:46:45 -05:00
Sam Harwell 46c12278a8 Add error 150: MISSING_LEXER_COMMAND_ARGUMENT; add error 151: UNWANTED_LEXER_COMMAND_ARGUMENT 2013-03-27 00:10:07 -05:00
Sam Harwell 551100ea37 Add error 149: INVALID_LEXER_COMMAND (fixes #190) 2013-03-26 23:57:11 -05:00