How do I use ANTLR v3 generated Lexer and Parser from Java?

In principle, the usage of antlr v3 is the same as for antlr v2. The example source code shown below is using the generated files from the java.g grammar.

Creating a character stream

First, the input needs to be converted into a org.antlr.runtime.CharStream. This interface is implemented e.g. by org.antlr.runtime.ANTLRStringStream which you can use if the input is in a String and by org.antlr.runtime.ANTLRFileStream which you can use if the input is in a File, and which actually extends the ANTLRStringStream.Both classes have a 1-arg constructor, taking the input string or a path to a file respectively. Here's an example for either case:

String input = "public static void main(String[] args) { }";
CharStream cs = new ANTLRStringStream(input);


String file = "/tmp/";
CharStream cs = new ANTLRFileStream(file);

Lexing the input to tokens

The so generated character stream needs to be lexed before it can be parsed:

JavaParserLexer lexer = new JavaParserLexer(cs);

A difference to antlr v2 is that the created lexer cannot be passed to the parser right away. The parser requires a org.antlr.runtime.TokenStream as argument.

CommonTokenStream tokens = new CommonTokenStream();

Parsing the token stream

With this TokenStream an instance of the parser can be created

JavaParser parser = new JavaParser(tokens);

The parser can then be used to parse the input. To parse a Java 'compilationUnit' - one of the nonterminals defined by java.g - run

RuleReturnScope result = parser.compilationUnit();

If an output mode is specified in the grammar, then the result of the parse is available in result.getTree() or result.getTemplate(), as appropriate. Depending on the used CharStream the code block needs to be surrounded by a try/catch taking care of a It might also be possible that using the parser throws a org.antlr.runtime.RecognitionException.

Parsing a tree

If you want to use a TreeParser, things get more complicated:

First, you need to create an AST by invoking the relevant rule method on the parser:

final fsqParser.formula_return parserResult = parser.formula();
final CommonTree ast = (CommonTree) parserResult.getTree();
if (ast == null) {
    // line is empty
    throw new ParseException(queryString, 0);

 Then you wrap the AST in a node stream and feed that to the tree parser, finally calling the relevant rule on the tree parser:

final fsqTreeParser treeParser = new fsqTreeParser(new CommonTreeNodeStream(ast));
final Formula queryFormula = treeParser.formula();