forked from jasder/antlr
getting antlr build doc in
This commit is contained in:
Terence Parr
parent
8e286bb372
commit
7be6ff56a7
|
|
@ -0,0 +1,125 @@
|
|||
# Building ANTLR
|
||||
|
||||
Most programmers do not need the information on this page because they will simply download the appropriate jar(s) or use ANTLR through maven (via ANTLR's antlr4-maven-plugin). If you would like to fork the project and fix bugs or tweak the runtime code generation, then you will almost certainly need to build ANTLR itself. There are two components:
|
||||
|
||||
1. the tool that compiles grammars down into parsers and lexers in one of the target languages
|
||||
1. the runtime used by those generated parsers and lexers.
|
||||
|
||||
I will assume that the root directory is `/tmp` for the purposes of explaining how to build ANTLR in this document.
|
||||
|
||||
# Get the source
|
||||
|
||||
The first step is to get the Java source code from the ANTLR 4 repository at github. You can download the repository from github, but the easiest thing to do is simply clone the repository on your local disk:
|
||||
|
||||
```bash
|
||||
$ cd /tmp
|
||||
/tmp $ git clone git@github.com:antlr/antlr4.git
|
||||
Cloning into 'antlr4'...
|
||||
remote: Counting objects: 43273, done.
|
||||
remote: Compressing objects: 100% (57/57), done.
|
||||
remote: Total 43273 (delta 26), reused 0 (delta 0)
|
||||
Receiving objects: 100% (43273/43273), 18.76 MiB | 1.60 MiB/s, done.
|
||||
Resolving deltas: 100% (22419/22419), done.
|
||||
Checking connectivity... done.
|
||||
```
|
||||
|
||||
# Compile
|
||||
|
||||
```bash
|
||||
$ cd /tmp
|
||||
$ git clone git@github.com:antlr/antlr4.git
|
||||
Cloning into 'antlr4'...
|
||||
remote: Counting objects: 59858, done.
|
||||
remote: Compressing objects: 100% (57/57), done.
|
||||
remote: Total 59858 (delta 28), reused 9 (delta 9), pack-reused 59786
|
||||
Receiving objects: 100% (59858/59858), 31.10 MiB | 819.00 KiB/s, done.
|
||||
Resolving deltas: 100% (31898/31898), done.
|
||||
Checking connectivity... done.
|
||||
$ cd antlr4
|
||||
$ mvn compile
|
||||
..
|
||||
[INFO] Reactor Summary:
|
||||
[INFO]
|
||||
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.447 s]
|
||||
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 3.113 s]
|
||||
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 14.408 s]
|
||||
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 1.276 s]
|
||||
[INFO] ANTLR 4 Runtime Test Generator ..................... SUCCESS [ 0.773 s]
|
||||
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 6.920 s]
|
||||
[INFO] ------------------------------------------------------------------------
|
||||
[INFO] BUILD SUCCESS
|
||||
...
|
||||
```
|
||||
|
||||
# Testing tool and targets
|
||||
|
||||
In order to perform the tests on all target languages, make sure that you have `mono` and `nodejs` installed. For example, on OS X:
|
||||
|
||||
```bash
|
||||
$ brew install mono
|
||||
$ brew install node
|
||||
```
|
||||
|
||||
To run the tests and **install into local repository** `~/.m2/repository/org/antlr`, do this:
|
||||
|
||||
```bash
|
||||
$ mvn install
|
||||
...
|
||||
-------------------------------------------------------
|
||||
T E S T S
|
||||
-------------------------------------------------------
|
||||
Running org.antlr.v4.test.runtime.csharp.TestCompositeLexers
|
||||
dir /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeLexers-1446068612451
|
||||
Starting build /usr/bin/xbuild /p:Configuration=Release /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeLexers-1446068612451/Antlr4.Test.mono.csproj
|
||||
dir /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeLexers-1446068615081
|
||||
Starting build /usr/bin/xbuild /p:Configuration=Release /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeLexers-1446068615081/Antlr4.Test.mono.csproj
|
||||
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.451 sec
|
||||
Running org.antlr.v4.test.runtime.csharp.TestCompositeParsers
|
||||
dir /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeParsers-1446068615864
|
||||
antlr reports warnings from [-visitor, -Dlanguage=CSharp, -o, /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeParsers-1446068615864, -lib, /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeParsers-1446068615864, -encoding, UTF-8, /var/folders/s1/h3qgww1x0ks3pb30l8t1wgd80000gn/T/TestCompositeParsers-1446068615864/M.g4]
|
||||
...
|
||||
[INFO] ------------------------------------------------------------------------
|
||||
[INFO] Reactor Summary:
|
||||
[INFO]
|
||||
[INFO] ANTLR 4 ............................................ SUCCESS [ 0.462 s]
|
||||
[INFO] ANTLR 4 Runtime .................................... SUCCESS [ 9.163 s]
|
||||
[INFO] ANTLR 4 Tool ....................................... SUCCESS [ 3.683 s]
|
||||
[INFO] ANTLR 4 Maven plugin ............................... SUCCESS [ 1.897 s]
|
||||
[INFO] ANTLR 4 Runtime Test Generator ..................... SUCCESS [07:11 min]
|
||||
[INFO] ANTLR 4 Tool Tests ................................. SUCCESS [ 16.694 s]
|
||||
[INFO] ------------------------------------------------------------------------
|
||||
[INFO] BUILD SUCCESS
|
||||
[INFO] ------------------------------------------------------------------------
|
||||
[INFO] Total time: 07:43 min
|
||||
...
|
||||
```
|
||||
|
||||
You should see these jars (building 4.5.2-SNAPSHOT):
|
||||
|
||||
```bash
|
||||
/Users/parrt/.m2/repository/org/antlr $ find antlr4* -name '*.jar'
|
||||
antlr4/4.5/antlr4-4.5.jar
|
||||
antlr4/4.5.2-SNAPSHOT/antlr4-4.5.2-SNAPSHOT-tests.jar
|
||||
antlr4/4.5.2-SNAPSHOT/antlr4-4.5.2-SNAPSHOT.jar
|
||||
antlr4-maven-plugin/4.5/antlr4-maven-plugin-4.5.jar
|
||||
antlr4-maven-plugin/4.5.2-SNAPSHOT/antlr4-maven-plugin-4.5.2-SNAPSHOT.jar
|
||||
antlr4-runtime/4.5/antlr4-runtime-4.5.jar
|
||||
antlr4-runtime/4.5.2-SNAPSHOT/antlr4-runtime-4.5.2-SNAPSHOT.jar
|
||||
antlr4-runtime-testsuite/4.5.2-SNAPSHOT/antlr4-runtime-testsuite-4.5.2-SNAPSHOT-tests.jar
|
||||
antlr4-runtime-testsuite/4.5.2-SNAPSHOT/antlr4-runtime-testsuite-4.5.2-SNAPSHOT.jar
|
||||
antlr4-tool-testsuite/4.5.2-SNAPSHOT/antlr4-tool-testsuite-4.5.2-SNAPSHOT.jar
|
||||
```
|
||||
|
||||
Note that ANTLR is written in itself, which is why maven downloads antlr4-4.5.jar for boostrapping 4.5.2-SNAPSHOT purposes.
|
||||
|
||||
To build without running the tests (saves about 8 minutes), do this:
|
||||
|
||||
```bash
|
||||
mvn -DskipTests install
|
||||
```
|
||||
|
||||
## Building ANTLR in Intellij IDE
|
||||
|
||||
After download ANTLR source, just "import project from existing sources" and click on the "Maven Projects" tab in right gutter of IDE. It should build stuff in the background automatically and look like:
|
||||
|
||||
<img src=images/intellij-maven.png width=200>
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 79 KiB |
|
|
@ -55,4 +55,6 @@ This documentation is a reference and summarizes grammar syntax and the key sema
|
|||
|
||||
* Integrating ANTLR into Development Systems
|
||||
|
||||
* Build ANTLR itself.
|
||||
* [Build ANTLR itself](building-antlr.md)
|
||||
|
||||
* [Cutting an ANTLR Release](releasing-antlr.md)
|
||||
|
|
|
|||
|
|
@ -0,0 +1,251 @@
|
|||
# Cutting an ANTLR Release
|
||||
|
||||
## Github
|
||||
|
||||
Create a release candidate tag 4.x-rc-1 or full 4.5 tag
|
||||
|
||||
```bash
|
||||
git tag -a 4.5 -m 'ANTLR final release 4.5'
|
||||
git push origin 4.5
|
||||
```
|
||||
|
||||
Create a pre-release or full release at github; [Example 4.5-rc-1](https://github.com/antlr/antlr4/releases/tag/4.5-rc-1).
|
||||
|
||||
## Bump version
|
||||
|
||||
Edit the repository looking for 4.5 or whatever and update it. Bump version in the following files:
|
||||
|
||||
* runtime/Java/src/org/antlr/v4/runtime/RuntimeMetaData.java
|
||||
* runtime/Python2/setup.py
|
||||
* runtime/Python2/src/antlr4/Recognizer.py
|
||||
* runtime/Python3/setup.py
|
||||
* runtime/Python3/src/antlr4/Recognizer.py
|
||||
* runtime/CSharp/Antlr4.Runtime/Properties/AssemblyInfo.cs
|
||||
* runtime/JavaScript/src/antlr4/package.json
|
||||
* runtime/JavaScript/src/antlr4/Recognizer.js
|
||||
* tool/src/org/antlr/v4/codegen/target/CSharpTarget.java
|
||||
* tool/src/org/antlr/v4/codegen/target/JavaScriptTarget.java
|
||||
* tool/src/org/antlr/v4/codegen/target/Python2Target.java
|
||||
* tool/src/org/antlr/v4/codegen/target/Python3Target.java
|
||||
|
||||
Here is a simple script to display any line from the critical files with, say, `4.5` in it:
|
||||
|
||||
```bash
|
||||
find /tmp/antlr4 -type f -exec grep -l '4\.5' {} \;
|
||||
```
|
||||
|
||||
Commit to repository.
|
||||
|
||||
## Maven Repository Settings
|
||||
|
||||
First, make sure you have maven set up to communicate with staging servers etc... Create file `~/.m2/settings.xml` with appropriate username/password for staging server and gpg.keyname/passphrase for signing. Make sure it has strict visibility privileges to just you. On unix, it looks like:
|
||||
|
||||
```bash
|
||||
beast:~/.m2 $ ls -l settings.xml
|
||||
-rw------- 1 parrt staff 914 Jul 15 14:42 settings.xml
|
||||
```
|
||||
|
||||
Here is the file template
|
||||
|
||||
```xml
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!--
|
||||
User-specific configuration for maven. Includes things that should not
|
||||
be distributed with the pom.xml file, such as developer identity, along with
|
||||
local settings, like proxy information.
|
||||
-->
|
||||
<settings>
|
||||
<servers>
|
||||
<server>
|
||||
<id>sonatype-nexus-staging</id>
|
||||
<username>sonatype-username</username>
|
||||
<password>XXX</password>
|
||||
</server>
|
||||
<server>
|
||||
<id>sonatype-nexus-snapshots</id>
|
||||
<username>sonatype-username</username>
|
||||
<password>XXX</password>
|
||||
</server>
|
||||
</servers>
|
||||
<profiles>
|
||||
<profile>
|
||||
<activation>
|
||||
<activeByDefault>false</activeByDefault>
|
||||
</activation>
|
||||
<properties>
|
||||
<gpg.keyname>UUU</gpg.keyname>
|
||||
<gpg.passphrase>XXX</gpg.passphrase>
|
||||
</properties>
|
||||
</profile>
|
||||
</profiles>
|
||||
</settings>
|
||||
```
|
||||
|
||||
## Maven release
|
||||
|
||||
The maven deploy lifecycle phased deploys the artifacts and the poms for the ANTLR project to the [sonatype remote staging server](https://oss.sonatype.org/content/repositories/snapshots/).
|
||||
|
||||
```bash
|
||||
mvn deploy -DskipTests
|
||||
```
|
||||
|
||||
```bash
|
||||
mvn release:prepare
|
||||
```
|
||||
|
||||
It will start out by asking you the version number:
|
||||
|
||||
```
|
||||
...
|
||||
What is the release version for "ANTLR 4"? (org.antlr:antlr4-master) 4.5.2: : 4.5.2
|
||||
What is the release version for "ANTLR 4 Runtime"? (org.antlr:antlr4-runtime) 4.5.2: :
|
||||
What is the release version for "ANTLR 4 Tool"? (org.antlr:antlr4) 4.5.2: :
|
||||
What is the release version for "ANTLR 4 Maven plugin"? (org.antlr:antlr4-maven-plugin) 4.5.2: :
|
||||
What is the release version for "ANTLR 4 Runtime Test Generator"? (org.antlr:antlr4-runtime-testsuite) 4.5.2: :
|
||||
What is the release version for "ANTLR 4 Tool Tests"? (org.antlr:antlr4-tool-testsuite) 4.5.2: :
|
||||
What is SCM release tag or label for "ANTLR 4"? (org.antlr:antlr4-master) antlr4-master-4.5.2: : 4.5.2
|
||||
What is the new development version for "ANTLR 4"? (org.antlr:antlr4-master) 4.5.3-SNAPSHOT:
|
||||
...
|
||||
```
|
||||
|
||||
Maven will go through your pom.xml files to update versions from 4.5.2-SNAPSHOT to 4.5.2 for release and then to 4.5.3-SNAPSHOT after release, which is done with:
|
||||
|
||||
```bash
|
||||
mvn release:perform
|
||||
```
|
||||
|
||||
Maven will use git to push pom.xml changes. (big smile)
|
||||
|
||||
Now, go here:
|
||||
|
||||
[https://oss.sonatype.org/#welcome](https://oss.sonatype.org/#welcome)
|
||||
|
||||
and on the left click "Staging Repositories". You click the staging repo and close it, then you refresh, click it and release it. It's done when you see it here:
|
||||
|
||||
[http://repo1.maven.org/maven2/org/antlr/antlr4-runtime/](http://repo1.maven.org/maven2/org/antlr/antlr4-runtime/)
|
||||
|
||||
Copy the jars to antlr.org site and update download/index.html
|
||||
|
||||
```bash
|
||||
cp ~/.m2/repository/org/antlr/antlr4-runtime/4.5.2/antlr4-runtime-4.5.2.jar ~/antlr/sites/website-antlr4/download/antlr-runtime-4.5.2.jar
|
||||
cp ~/.m2/repository/org/antlr/antlr4/4.5.2/antlr4-4.5.2.jar ~/antlr/sites/website-antlr4/download/antlr-4.5.2-complete.jar
|
||||
cd ~/antlr/sites/website-antlr4/download
|
||||
git add antlr-4.5.2-complete.jar
|
||||
git add antlr-runtime-4.5.2.jar
|
||||
git commit -a -m 'add 4.5.2 jars'
|
||||
git push origin gh-pages
|
||||
```
|
||||
|
||||
Update on site:
|
||||
|
||||
* download.html
|
||||
* index.html
|
||||
* scripts/topnav.js
|
||||
|
||||
## Deploying Targets
|
||||
|
||||
### JavaScript
|
||||
|
||||
```bash
|
||||
cd runtime/JavaScript/src
|
||||
zip -r /tmp/antlr-javascript-runtime-4.5.2.zip antlr4
|
||||
cp /tmp/antlr-javascript-runtime-4.5.2.zip ~/antlr/sites/website-antlr4/download
|
||||
# git add, commit, push
|
||||
```
|
||||
|
||||
Move target to website
|
||||
|
||||
```bash
|
||||
pushd ~/antlr/sites/website-antlr4/download
|
||||
git add antlr-javascript-runtime-4.5.2.zip
|
||||
git commit -a -m 'update JS runtime'
|
||||
git push origin gh-pages
|
||||
popd
|
||||
```
|
||||
|
||||
### CSharp
|
||||
|
||||
```bash
|
||||
cd ~/antlr/code/antlr4/runtime/CSharp/runtime/CSharp
|
||||
# kill previous ones manually as "xbuild /t:Clean" didn't seem to do it
|
||||
rm Antlr4.Runtime/bin/net20/Release/Antlr4.Runtime.dll
|
||||
rm Antlr4.Runtime/obj/net20/Release/Antlr4.Runtime.dll
|
||||
# build
|
||||
xbuild /p:Configuration=Release Antlr4.Runtime/Antlr4.Runtime.mono.csproj
|
||||
# zip it up to get a version number on zip filename
|
||||
zip --junk-paths /tmp/antlr-csharp-runtime-4.5.2.zip Antlr4.Runtime/bin/net35/Release/Antlr4.Runtime.dll
|
||||
cp /tmp/antlr-csharp-runtime-4.5.2.zip ~/antlr/sites/website-antlr4/download
|
||||
```
|
||||
|
||||
Move target to website
|
||||
|
||||
```bash
|
||||
pushd ~/antlr/sites/website-antlr4/download
|
||||
git add antlr-csharp-runtime-4.5.2.zip
|
||||
git commit -a -m 'update C# runtime'
|
||||
git push origin gh-pages
|
||||
popd
|
||||
```
|
||||
|
||||
### Python
|
||||
|
||||
The Python targets get deployed with `setup.py`. First, set up `~/.pypirc` with tight privileges:
|
||||
|
||||
```bash
|
||||
beast:~ $ ls -l ~/.pypirc
|
||||
-rw------- 1 parrt staff 267 Jul 15 17:02 /Users/parrt/.pypirc
|
||||
```
|
||||
|
||||
```
|
||||
[distutils] # this tells distutils what package indexes you can push to
|
||||
index-servers =
|
||||
pypi
|
||||
pypitest
|
||||
|
||||
[pypi]
|
||||
repository: https://pypi.python.org/pypi
|
||||
username: parrt
|
||||
password: XXX
|
||||
|
||||
[pypitest]
|
||||
repository: https://testpypi.python.org/pypi
|
||||
username: parrt
|
||||
```
|
||||
|
||||
Then run the usual python set up stuff:
|
||||
|
||||
```bash
|
||||
cd ~/antlr/code/antlr4/runtime/Python2
|
||||
# assume you have ~/.pypirc set up
|
||||
python setup.py register -r pypi
|
||||
python setup.py sdist bdist_wininst upload -r pypi
|
||||
```
|
||||
|
||||
and do again for Python 3 target
|
||||
|
||||
```bash
|
||||
cd ~/antlr/code/antlr4/runtime/Python3
|
||||
# assume you have ~/.pypirc set up
|
||||
python setup.py register -r pypi
|
||||
python setup.py sdist bdist_wininst upload -r pypi
|
||||
```
|
||||
|
||||
Add links to the artifacts from download.html
|
||||
|
||||
## Update javadoc for runtime and tool
|
||||
|
||||
```bash
|
||||
cd ~/antlr/sites/website-antlr4
|
||||
git checkout gh-pages
|
||||
git pull origin gh-pages
|
||||
cd Java
|
||||
jar xvf ~/.m2/repository/org/antlr/antlr4-runtime/4.5.2/antlr4-runtime-4.5.2-javadoc.jar
|
||||
cd ../JavaTool
|
||||
jar xvf ~/.m2/repository/org/antlr/antlr4/4.5.2/antlr4-4.5.2-javadoc.jar
|
||||
git commit -a -m 'freshen api doc'
|
||||
git push origin gh-pages
|
||||
```
|
||||
|
||||
## Update Intellij plug-in
|
||||
|
||||
Rebuild antlr plugin with new antlr jar.
|
||||
Loading…
Reference in New Issue