ANTLR v3 C# Code generator and Runtime library

Johannes Luber (Maintainer)
jaluber AT gmx.de

Kunle Odutola
kunle UNDERSCORE odutola AT hotmail.com

Micheal Jordan

Contents

• #Status
• #Architecture
• #Target Platforms
  • #Runtime Location
  • #Supported build tools
• #Performance
• #Usage
  • #Generating C# code from a grammar
  • #Specifying the namespace for your recognizer
  • #Syntactic differences from the Java target
  • #Debugging with ANTLRWorks

Status

As of September 2007, the C# code generator and runtime are NOT in sync with the latest release and development versions of the ANTLR tool and Java language target. The last release of the C# code generator and runtime were sync'ed to ANTLR v3.0 release from July 2007. Nevertheless, no major problems have been reported by those using the C# codegen and runtime with ANTLR v3.0.1 since it's release in August 2007.

As of October 2007, the ANTLR source depot contains an early beta of the C# codegen and runtime for the upcoming ANTLR v3.1 release. Starting from the end of October, ANTLR daily builds of v3.1 have been available for those wishing to test the C# support with the new features of ANTLR v3.1. As before, development progress going forwards is likely to be sporadic.

As of May 2008, the C# target is in sync with the Java target of ANTLR v3.1. The new version breaks source compatibility with the old version. To a certain extent, regeneration of grammars does help, but certain fields have been renamed to follow .NET conventions, which means that the first is capitalized (example: .tree is now .Tree). The exception is .st, which is now .ST. Additionally, the new target CSharp2 has been introduced. The reason is three-fold: Firstly, the old target CSharp has to remain compatible to .NET 1.1. So the target can't take advantage of C# 2 features in the code generation templates and the runtime requires twice the number of files, effectively doubling the amount of maintenance.

Secondly, a certain bug fix requires a C# 2 feature or the change of templates for each time it appears - and this has to be done by the user. The details of doing so are found in a section below. Thirdly, because the backwards compatibility sucks majorly, I want to deprecate it without forcing people to abandon .NET 1.1 compatibility immediately. The easiest way of doing so was to introduce a new target which allows me to change even the code generation templates. As working on further enhancements will at least break binary compatibility, most of the changes will be done for ANTLR v3.2. During the life-time of ANTLR v3.1, the public API of the C# target will be frozen. If you wish to future-proof your grammar, change them to the CSharp2 target. It is planned to remove the CSharp target for ANTLR v3.3.

The C# code generation templates and the CLR runtime library are feature complete for both targets. The C# targets leverage the existing C# StringTemplate implementations to support the broadest range of the features that ANTLR provides. The long open issue of unit tests has finally been tackled with the adoption of MbUnit and the inclusion (in the v3.1 version) of a wide range of tests for the runtime library. As before, basic sanity checks will done by ensuring that the sample grammars in the examples-v3 archive works as designed. This is currently a work-in-progess for the v3.1 release.

Architecture

As with all other targets, the C# code generation and runtime are modeled on the Java version. This means the C# target supports features such as grammar development/prototyping and remote debugging with the AntlrWorks GUI which is very important for ANTLR users.

Target Platforms

Microsoft .NET v1.1 and v2.0
Mono

Runtime Location

The compiled libraries are found in the distribution under the directory "runtime/csharp". Both targets use the same runtime. Intermediary builds may not have the current version and can be compiled by using the build tools.

Supported build tools

Microsoft Visual Studio 2003 and 2005
Nant v0.85

Performance

The V3 target generates code that is easily faster than that generated by the V2 target (especially the lexers). We probably won't be able to match the bare-metal performance of the code generated by Jim Idle's C target or Ric Klaren's C++ target, but we expect to be very competitive with the other targets.

Usage

Generating C# code from a grammar

To specify that the ANTLR tool should generate C# code (rather than the default of generating Java code) for a grammar, set the grammar-level option named language to the value CSharp2 as shown below:

grammar MyGrammar;

options
{
    language=CSharp2;
}

// rest of grammar follows
....

Specifying the namespace for your recognizer

You can specify that your generated recognizer should be declared within a specific namespace as shown below. By default all recognizers are generated as top-level types with no enclosing namespace.

grammar MyGrammar;

options
{
    language=CSharp2;
}

@parser::namespace { My.Custom.NameSpace.For.Parser.In.Combined.Grammar } // Or just @namespace { ... }

@lexer::namespace { My.Custom.NameSpace.For.Lexer.In.Combined.Grammar }

// rest of grammar follows
....

lexer grammar MyGrammar;

options
{
    language=CSharp2;
}

@namespace { My.Custom.NameSpace.For.Lexer }

// rest of grammar follows
....

parser grammar MyGrammar;

options
{
    language=CSharp2;
}

@namespace { My.Custom.NameSpace.For.Parser }

// rest of grammar follows
....

tree grammar MyGrammar;

options
{
    language=CSharp2;
}

@namespace { My.Custom.NameSpace.For.TreeParser }

// rest of grammar follows
....

Syntactic differences from the Java target

The C# target uses language features like properties as the official coding guidelines which cause the general documentation to differ from what can be really used for the C# target. The rule of thumb is, that the attributes of rules are accessed with a capital letter at the beginning. Nonetheless there are exceptions so the goal is to have a comprehensive overview. If there are any errors, please fix them or send an email to the mailing list.

Another problem, you may encounter while using the CSharp target, is that value types are initialized with null (happens e.g. while using labels). The cause lies in the following definition of CSharp.stg:

csharpTypeInitMap ::= [
    "int":"0",
    "uint":"0",
    "long":"0",
    "ulong":"0",
    "float":"0.0",
    "double":"0.0",
    "bool":"false",
    "byte":"0",
    "sbyte":"0",
    "short":"0",
    "ushort":"0",
    "char":"char.MinValue",
    default:"null" // anything other than an atomic type
]

As you can see, only the inbuilt value types are supported. As adding value types to this map is an open-ended task, the maintainer does not make any changes in that structure for all users. Any changes have to be done by the user locally repeatedly for new used version of ANTLR. It is recommend to switch to the CSharp2 target and use .NET 2 or higher as target platforms, as the problem has been fixed there in an environment-independent manner.

Debugging with ANTLRWorks

The ANTLRWorks tool is written in Java and is only able to debug Java recognizers directly. Nevertheless, you can debug your C# recognizers with ANTLRWorks by using the Remote Debugging feature of ANTLRWorks. In ANTLRWorks, Remote Debugging works by connectiong to a running instance of a debug-instrumented recognizer (generated with the -debug switch to ANTLR) over the network.

To debug your C# recognizer with ANTLRWorks:

1. Generate a debuggable version of your recognizer by specifying the -debug option to ANTLR
2. Create a driver program that creates your recognizer and runs some test input through it (see the examples-v3 archive for sample driver programs)
3. Compile your driver and recognizer to produce your executable file(s)
4. Execute your driver program (it will launch your recognizer and appear to hang - it's just waiting for ANTLRWorks to connect)
5. Start ANTLRWorks (or switch to it if it is already running) and click the menu Debugger|Debug Remote...
6. Click Connect to accept the default host and port values (localhost and 49153 respectively)
7. ANTLRWorks should now start debugging your recognizer!

   Warning

   ANTLRWorks remote debugging has only been tested for C# Parsers. TreeParsers and Lexers should work but...