How to build an ANTLR code generation target

The following instructions show how to go about starting a new back-end for a new language "XYZ" using the Java back-end as a basis.

All the functionality contained in the Java files in the antlr-3.1/runtime/Java/src/org/antlr/runtime directory is needed to create a full back-end. You must also copy from the code generation templates of another target, such as antlr-3.1/src/org/antlr/codegen/templates/Java/*.stg. Finally create antlr-3.1/src/org/antlr/codegen/XYZTarget.java if you need to override anything in Target.java, which you probably will.

But it is in fact pretty easy to get started:

• In src/org/antlr/codegen/templates/
  • create a directory XYZ
  • copy Java/Java.stg to XYZ/XYZ.stg

• I recommend building the ANTLR tool 'in place'. Do not create a jar or compile/copy to a build directory. When you run it with 'java -cp path-to-src-dir ...' it will use the original *.stg file, which you'll edit a lot - so rebuilding the tool would be quite a PITA.

• Create a directory antlr-3.1/runtime/XYZ. Here you can put anything you need (no need to clone Java 1:1).

• Start with a simple lexer like:

lexer grammar T;
options { language = XYZ; }
ZERO: '0';

N.B: The name after the word "grammar" needs to be the same as the filename in which you save it, i.e. T.g.

• Look at the generated code and try to figure out which templates in XYZ.stg you have to port to get valid XYZ code. What I did, is to comment out the Java code in all templates replacing it with something like FIXME([number]). Then you fix the templates until no FIXME remains in the
  output.

• You'll need a basic implementation of a character stream and base recognizer/lexer to get the example running. Just implement the methods that are actually needed to get the example running w/o errors.

• You'll either get the feeling "Wow, that was easy!" and move on (that happened to me) or "Eeek, what a pain!" and let someone else to the work.

Target Source Files

Your ANTLR code generation target will consist of several files in the target language. A number of run-time support files that you'll create directly, and a number of files that ANTLR will generate (in the target language) as the result of the user's grammar being processed by ANTLR. These files are generated from StringTemplates like ZYX.stg above.

ANTLR looks for the presence of certain template files and templates. Some are required, some are optional.

XYZ.stg

The principle template file. The templates in this file are used to generate the Lexer, Parser and Tree Parser generated by the user's grammar.

block() StringTemplate

closureBlock() StringTemplate

outputFile() StringTemplate

Formal parameters:

rule() StringTemplate

The rule() StringTemplate is instantiated by ANTLR's own grammar processing and added to the "rules" attribute. It takes the following parameters.

dfaState() StringTemplate

dfaLoopbackState() StringTemplate

A DFA state that is actually the loopback decision of a closure loop. If end-of-token (EOT) predicts any of the targets then it should act like a default clause (i.e., no error can be generated). This is used only in the lexer so that for ('a')* on the end of a rule anything other than 'a' predicts exiting.

AST.stg

ASTParser.stg

ST.stg