754348605d
create match object for matching
2013-09-05 16:58:37 -07:00
35e279cef1
change name from T.g4 to X.g4 to avoid weird classloader issue. TLexer lives somewhere in my damn system and I can't find it.
2013-09-05 10:35:20 -07:00
d2f7ba8339
cleanup
2013-09-04 17:25:14 -07:00
4b87f84b99
got decent test rig
2013-09-04 17:09:14 -07:00
531c31861a
got basic pattern compilation going
2013-09-04 14:48:21 -07:00
2aade3eecc
avoid resource leak and cleanup (Coverity)
2013-08-31 17:46:23 -07:00
fc2dfcda96
dodgy code cleanup (Coverity)
2013-08-31 17:36:16 -07:00
3e53c0c61d
oops; deleted side-effecting "dead code"
2013-08-31 17:05:48 -07:00
78298bb382
defined compareTo but not equals/hash (Coverity)
2013-08-31 16:56:07 -07:00
875fa0a31d
rm dead code (Coverity)
2013-08-31 16:52:07 -07:00
8bff26d758
fix cut-paste error (Coverity)
2013-08-31 16:47:33 -07:00
21248a42ca
rm unneeded null ptr check (Coverity)
2013-08-31 16:37:16 -07:00
1febc36e15
avoid potential null ptr deref (Coverity)
2013-08-31 16:32:59 -07:00
0ac47437bc
rm potential null ptr deref (Coverity)
2013-08-31 16:20:34 -07:00
9c186c8d52
rm print statement
2013-08-31 12:20:57 -07:00
04d2b5b92d
Add regression test for #315 (already passes)
2013-08-12 19:26:40 -05:00
19fce5dd7d
change ruleElementOption back to elementOption
2013-08-08 16:12:01 -07:00
531afe5af8
changed grammar to allow <assoc=right> on | operator for alt. still allowed on token refs but ignored. Simplified left-recursion elemination rules. see CHANGES.TXT Aug 7, 2013. BREAKING CHANGE. Must alter ternary op alts in grammars.
2013-08-08 17:06:15 -05:00
185ee4e18e
add <options> notation to alt right after pipe. rename elementOption to ruleElementOption as it now can match options on alt. Move AltAST to have options.
2013-08-08 17:06:15 -05:00
d041a4828a
fix poor function name: setTokenAssoc
2013-08-08 17:05:19 -05:00
274120c2b6
Now working on release 4.1.1
2013-07-02 19:11:04 -05:00
6236072e3b
Documentation updates prior to release
2013-06-30 15:50:56 -05:00
dbe1390c6f
Update referenced maven plugin version numbers
2013-06-30 12:47:26 -05:00
05f7fc6500
Update development version to 4.1 (from 4.0.1)
2013-06-30 12:46:41 -05:00
b7133164f4
Merge pull request #294 from sharwell/fix-293
...
Remove tool/playground from source control, add to .gitignore
2013-06-29 11:58:42 -07:00
36acb139b7
Remove the SET_QUALIFIED_ATTR rule from ActionSplitter since it provides no additional functionality ( fixes #295 )
2013-06-28 10:28:55 -05:00
931df811db
Remove tool/playground from source control, add to .gitignore ( fixes #293 )
2013-06-25 14:30:25 -05:00
b0ddb45654
Merge pull request #284 from sharwell/epsilon-analysis
...
Add the EPSILON_CLOSURE error and EPSILON_OPTIONAL warning
2013-06-24 13:09:38 -07:00
82330a8a78
Support Unicode escape sequences in JavaUnicodeInputStream with indirection (multiple 'u' characters) ( fixes #287 )
2013-06-24 12:02:42 -05:00
40f41e2b1c
Fix ATN created for non-greedy optional block with multiple alternatives
2013-06-14 21:05:58 -05:00
3bb774a508
Add the EPSILON_CLOSURE error and EPSILON_OPTIONAL warning
2013-06-14 20:53:34 -05:00
cd73c212fb
Add regression test for #239
2013-06-09 14:11:35 -05:00
6fe7154c44
Exclusive or operator is left associative ( fixes #280 )
2013-06-08 21:09:33 -05:00
ba532cdd49
Support list labels on a set of tokens ( fixes #270 )
2013-06-08 11:59:47 -05:00
2a90413188
Use standard 0-based indexing
2013-06-01 22:07:27 -05:00
9167968b74
Only serialize unique sets (smaller serialized ATN)
2013-06-01 22:03:31 -05:00
dc801ad6a7
Extend the supported input character range to include all UTF-16 code points (specifically, U+FFFF is now supported, fixes #267 )
2013-06-01 22:03:18 -05:00
f1f134c962
DiagnosticErrorListener includes rule names for each decision in its reports
2013-06-01 21:55:31 -05:00
2f2fd4585e
Improved error message for unterminated string literals ( fixes #243 )
2013-05-26 13:45:38 -05:00
d3af4a2f3a
Report errors that occur while lexing a grammar ( fixes #262 )
2013-05-26 13:45:38 -05:00
611e5ebf02
Significantly improve the performance of JavaUnicodeInputStream.LA(1)
2013-05-24 09:40:48 -05:00
469ebdd6e8
Fix unit tests now that the parser might not compute the conflicting alts BitSet before calling report methods
2013-05-20 14:58:32 -05:00
13f121a16f
Add @NotNull annotations to listener and visitor methods
2013-05-20 14:20:54 -05:00
d6c3841b0a
Generate Javadoc for the generated listener and visitor interfaces, classes, and methods
2013-05-20 14:19:53 -05:00
f6255bd5be
Add regression test for #248 (already passes)
2013-05-20 13:31:47 -05:00
ec8457383c
Calculate detailed transition statistics per decision
2013-05-18 14:22:09 -05:00
d8f2a5ce3c
Pass more information to ANTLRErrorListener methods
2013-05-18 14:17:36 -05:00
8d0a1bbd8d
Show additional information about rules and viable alternatives for conflicts, ambiguity, and/or context sensitivity
2013-05-18 11:46:57 -05:00
aad78ba7d2
Fix transition statistics calculation when a file contains no transitions
2013-05-18 11:40:36 -05:00
f758e8e24d
Some additional lexer tweaks
2013-05-18 11:19:28 -05:00
49f9e093e8
Clean up character and string literals in the lexer
2013-05-18 11:18:58 -05:00
c835fc9a79
Add unambiguous shift and comparison operators to the lexer
2013-05-18 11:15:34 -05:00
ad2b5876a5
Add separators and operators to the lexer
2013-05-18 11:13:35 -05:00
0f4ec397da
Add NullLiteral to the lexer
2013-05-18 11:11:56 -05:00
f5ed71d2b9
Improved support for Java 7 numeric literals
2013-05-18 11:10:53 -05:00
9bd3e965ef
'true' and 'false' are both lexed as a BooleanLiteral token
2013-05-18 11:09:22 -05:00
d9e1289d13
Add keywords as explicit lexer rules; enum and assert are always keywords (could extend JavaLexer and override emit to change this behavior)
2013-05-18 11:06:02 -05:00
62cbab4f86
Use explicit groups so the ATN can collapse sets
2013-05-18 11:01:14 -05:00
0e5bcc2416
Report file names with lexer errors
2013-05-17 23:57:13 -05:00
5948f50288
Proper Unicode identifier handling
2013-05-17 23:56:32 -05:00
d9d5100105
Use JavaUnicodeInputStream for proper handling of Unicode escape sequences in Java source code
2013-05-17 23:47:59 -05:00
1906e4a639
Use IntervalSet.or and Arrays.copyOf instead of duplicating their functionality
2013-05-14 09:58:51 -05:00
35ef516e28
Add some assertions
2013-05-14 09:58:02 -05:00
c0d1fcc110
Use 0-based indexing for decision lookahead so we aren't wasting memory, and array indexing is now consistent with ATN state outgoing transitions indexing
2013-05-14 09:57:04 -05:00
32d4c85f0f
Rethrow unit test execution exceptions wrapped in a RuntimeException
2013-05-13 20:27:50 -05:00
af59f35481
Count full context transitions in statistics
2013-05-13 20:27:18 -05:00
0652b61a33
Default to FILE_GRANULARITY=true
2013-05-12 18:07:35 -05:00
0fac5ce133
Use System.nanoTime for more precise timing results
2013-05-12 17:52:23 -05:00
3694d51c4e
Don't try to report detailed statistics for empty states (fixes exception in lexer stats reporting)
2013-05-12 17:48:33 -05:00
9ea4747524
Call Future.get() to propagate exceptions so it's easier to spot problems
2013-05-12 17:46:55 -05:00
ac7cc7daaf
Merge pull request #247 from sharwell/parallel-testing
...
Parallel testing updates
2013-05-05 12:32:56 -07:00
d67d924b0d
Update additional classes to use MurmurHash hashing
2013-05-05 14:21:42 -05:00
d75f8e9aa3
Add the DETAILED_DFA_STATE_STATS option to TestPerformance, which shows a breakdown of DFA states by rule (parsers) or mode (lexers)
2013-05-01 22:06:00 -05:00
d147e9202d
Use ->skip instead of ->channel(HIDDEN) to ensure reported token counts correspond to the number of parsed tokens in TestPerformance
2013-05-01 22:02:26 -05:00
28abae9cfb
Add the TIME_PARSE_ONLY flag to TestPerformance
2013-04-30 22:03:49 -05:00
cc0d050622
Display the current pass number in TestPerformance result summary (helps monitor progress on very long runs)
2013-04-30 22:00:35 -05:00
3558151e56
Support parallelizing TestPerformance across passes instead of across files (controlled by the FILE_GRANULARITY flag)
2013-04-30 21:58:32 -05:00
4dfa245898
Added the COMPUTE_TIMING_STATS and TIMING_CUMULATIVE options to TestPerformance (disabled by default)
2013-04-30 21:53:30 -05:00
8f7c43fbf6
Support computing and displaying transition statistics
2013-04-30 21:48:51 -05:00
be57daa108
Remove unnecessary reflection, fix option to not reuse DFA
2013-04-30 21:41:55 -05:00
646f673d84
Track token count separately for each pass
2013-04-30 21:35:10 -05:00
dda9ed9fab
Remove unnecessary null checks
2013-04-24 12:48:09 -05:00
5821747a42
Add TestPerformance.COMPUTE_TRANSITION_STATS
2013-04-24 12:47:25 -05:00
f015942b5e
* Simplify creation of new DFA edges - only create the target state and let DFA code check for termination conditions
...
* Fix handling of previously cached error edges (always need to check previous state for reaching a rule stop state)
* Fix DFA created during SLL-only parsing (should be identical to the DFA created by full LL parsing)
2013-04-24 12:29:43 -05:00
15577fd21f
Remove unnecessary method ParserInterpreter.predictATN
2013-04-24 10:32:20 -05:00
be83edf6d0
Gather additional parsing statistics from each file
2013-04-22 17:35:16 -05:00
f1fc42024f
explicitConstructorInvocation was broken and unnecessary
2013-04-22 17:31:58 -05:00
ad41c3b14f
Assignments can appear in the middle of a ternary expression
2013-04-22 17:31:29 -05:00
4b6d0d9ea2
Support diamond operator, improved support for explicit generic invocations
2013-04-22 17:31:02 -05:00
df336b4e59
Remove unreferenced rule from Java LR
2013-04-22 17:27:44 -05:00
69f22f8012
Allow extraneous ';' in annotation interface declaration (not allowed by grammar, but is allowed by compiler and appears in some source code)
2013-04-22 17:24:44 -05:00
a670d7d2b0
Support Java 7 "try with resources" and "multi-catch"
2013-04-22 17:23:54 -05:00
c8701a9434
Support directory filters in TestPerformance, add a variety of additional FilenameFilter implementations
2013-04-22 17:16:49 -05:00
210dc31a43
Print the proper statistics when only a subset of available files is parsed
2013-04-22 17:13:56 -05:00
b9918b9e25
Remove unnecessary null checks
2013-04-22 17:10:07 -05:00
5280b68cab
Add the TestPerformance.PRELOAD_SOURCES configuration field
2013-04-22 17:08:24 -05:00
94b06df72f
Add TestPerformance.ENCODING configuration field (default UTF-8)
2013-04-22 16:53:53 -05:00
e7b84c13dd
Add comment to TestPerformance.PREDICTION_MODE configuration field
2013-04-22 16:49:36 -05:00
f5294204ae
Add the TestPerformance.MAX_FILES_PER_PARSE_ITERATION configuration field
2013-04-21 15:00:20 -05:00
82f4d07243
Default value for TestPerformance.BAIL_ON_ERROR is now false since it's overridden anyway for first phase of two-stage parsing
2013-04-21 14:56:28 -05:00
e438b60f37
Add the SHUFFLE_FILES_AT_START and SHUFFLE_FILES_AFTER_ITERATIONS options to TestPerformance
2013-04-21 14:55:31 -05:00
bc4b530b3b
tweak my tests for java.
2013-04-20 10:17:25 -07:00
7a7f4a7851
Must check rule transition follow states before eliminating states in the ATN ( fixes #224 )
2013-04-18 14:46:11 -05:00
bf4f198fbf
Fix build error
2013-04-18 14:43:56 -05:00
9236477438
Fix code in TestPerformance that clears the DFA
2013-04-15 11:12:16 -05:00
db85cbc257
Fix EOF handling when closure operation was skipped ( fixes #218 )
2013-04-12 14:02:24 -05:00
bb18e14e04
Throw UnsupportedOperationException in not implemented method
2013-04-12 13:20:10 -05:00
05f667d2e5
Initialize the elements of the decisionToDFA arrays when the array is created
2013-04-12 13:17:53 -05:00
f4ae1cf471
Updated documentation
2013-04-04 13:09:54 -05:00
4433a57baa
Strict handling of redefined rules (do not attempt to generate code), fixes #210
2013-04-04 13:09:31 -05:00
7235d71cc0
Fix NPE revealed by updated testing method
2013-04-04 13:07:43 -05:00
805430177c
More thorough testing of tool error reporting
2013-04-04 13:06:22 -05:00
0e5cf32bf4
Merge branch 'fix-195'
2013-03-27 15:02:42 -05:00
2242948c03
Merge branch 'fix-76'
2013-03-27 15:02:17 -05:00
cae9f7bbf1
Merge branch 'fix-202'
2013-03-27 15:02:07 -05:00
71cc6fba8a
Merge branch 'fix-194'
2013-03-27 15:01:57 -05:00
f75878b3e6
Merge branch 'fix-196'
2013-03-27 15:01:45 -05:00
961f68c865
Fix incorrect reports of label type conflicts for labels aliased across separate *labeled* outer alternatives ( fixes #195 )
2013-03-27 11:21:32 -05:00
c592e41637
Split serialized ATN in Java target to ensure string literals are under 65535 bytes limit ( fixes #76 )
2013-03-27 11:04:28 -05:00
7f029a3c80
Avoid creating empty action methods in lexer ( fixes #202 )
2013-03-27 09:46:45 -05:00
46c12278a8
Add error 150: MISSING_LEXER_COMMAND_ARGUMENT; add error 151: UNWANTED_LEXER_COMMAND_ARGUMENT
2013-03-27 00:10:07 -05:00
551100ea37
Add error 149: INVALID_LEXER_COMMAND ( fixes #190 )
2013-03-26 23:57:11 -05:00
05f4b76fab
Add additional unit test for empty tokens{} block in grammar
2013-03-26 23:56:19 -05:00
b4413e8656
Fix handling of empty options{} block in grammar ( fixes #194 )
2013-03-26 23:55:53 -05:00
15a23c3cd9
Configs "in context" have special meaning for predicate transitions, so don't add them to closure busy as a visited state or some configs could be improperly eliminated from the closure set ( fixes #196 )
2013-03-26 23:11:20 -05:00
07689de50f
Updated @NotNull and @Nullable annotations, linked documentation
2013-03-26 18:41:39 -05:00
f6ad977e0d
ATN fields grammarType and maxTokenType are now final
2013-03-26 18:14:13 -05:00
36abbda44f
Create ATNType enumeration for ATN.grammarType field, use UUID instead of incrementing a version number for improved reliability across branches
2013-03-26 18:14:07 -05:00
c35f5ec40c
Do not require escape for $ in action when not followed by an ID start char ( fixes #176 )
2013-03-06 13:13:33 -06:00
56c053a5be
Remove unreferenced fragment rules
2013-03-06 13:00:29 -06:00
e54aa851cd
Fix generation of invalid Unicode escape sequences in Java code ( fixes #164 )
2013-03-06 12:16:44 -06:00
2ffb12028a
Directly reference ErrorType members from tests for easier cross-referencing
2013-03-06 11:09:33 -06:00
a9e1b8b2b6
Change error type 146 (EPSILON_TOKEN) to a warning ( fixes #180 )
2013-03-06 10:55:25 -06:00
fef6dd2885
Allow direct calls to left-recursive rules ( fixes #161 )
2013-03-06 10:48:42 -06:00
645f80971f
Merge remote-tracking branch 'sharwell/lexer-commands'
2013-02-23 17:59:15 -06:00
68050dd126
Move Java-specific character escaping code to JavaTarget, make methods in Target abstract
2013-02-23 17:56:54 -06:00
b4e66f18cd
Move Java "bad word" analysis to JavaTarget
2013-02-23 17:55:25 -06:00
9e1eed7614
Make Target an abstract base class, with default derived implementation JavaTarget
2013-02-23 17:55:23 -06:00
b642c72845
Avoid loading the target in the CodeGenerator constructor
2013-02-23 17:55:21 -06:00
d48ffa1459
Use accessors for fields in Target
2013-02-23 17:55:19 -06:00
4d854a521e
Move template loading to Target
2013-02-23 17:55:17 -06:00
920d5dec8b
Separate visibility and "bad word" analysis in Target
2013-02-23 17:55:15 -06:00
cb298530ba
Remove unused Target methods
2013-02-23 17:55:13 -06:00
371bcc83b2
Use accessor methods for CodeGenerator.templates and CodeGenerator.target
2013-02-23 17:55:11 -06:00
758c5cd37f
Use lexer commands instead of actions where available
2013-02-23 16:45:16 -06:00
18508e2209
Do not allow raw newline characters in lexer string literals
2013-02-22 14:12:04 -06:00
1bc4c5e6a4
Do not allow raw newline characters in a lexer char set
2013-02-22 14:12:00 -06:00
08386ba2dc
Simplify LEXER_CHAR_SET rule
2013-02-22 14:11:58 -06:00
3d59e47fb4
Remove unnecessary method Utils.replace (already exists as String.replace)
2013-02-22 14:08:51 -06:00
aae042c6db
Additional IntervalSet unit test (catches underlying cause of #153 )
2013-02-05 13:09:14 -06:00
Terence Parr
Sam Harwell
Terence Parr