Group Files

Template Group Files

Overview of Template Groups

StringTemplate 2.0 introduced the notion of a template group file that has two main attractions. First, it allows you to define lots of small templates more conveniently because they may all be defined within a single file. Second, (unlike in a simple template file) you may specify formal template arguments that help StringTemplate detect errors (such as setting unknown attributes) and make the templates easier to read. Here is a sample group file with two templates, vardef and method, that could be used to generate C files:

group simple;

vardef(type,name) ::= "<type> <name>;"

method(type,name,args) ::= <<
<type> <name>(<args; separator=",">) {
  <statements; separator="\n">
}
>>

All groups use <...> expression  delimiters by default. Single line templates are enclosed in double quotes while multi-line templates are enclosed in double angle-brackets. Every template must define arguments even if the formal argument list is blank.

Using the templates from a template group file (or from a string) is straightforward. The StringTemplateGroup class has a number of constructors, one of which allows you to pass in a string or file or whatever:

String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a Reader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.getInstanceOf("vardef");
t.setAttribute("type", "int");
t.setAttribute("name", "foo");
System.out.println(t);

String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a System.IO.TextReader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.GetInstanceOf("vardef");
t.SetAttribute("type", "int");
t.SetAttribute("name", "foo");
Console.Out.WriteLine(t);

templates = "group simple; vardef(type,name) ..."; # templates from above
# Use the constructor that accepts a Reader
group = stringtemplate3.StringTemplateGroup(file=StringIO(templates))
t = group.getInstanceOf("vardef")
t["type"] = "int"
t["name"] = "foo"
print str(t)

The output would be: "int foo;".

Supergroups and interfaces

Template groups may derive from other template groups, thus inheriting all of the features (templates and maps) from the supergroup. Group inheritance provides an appropriate model whereby, for example, a variation on a code generation target may be defined by describing how it differs from a previously defined target. Considering Java 1.4 versus 1.5, a Java1_5 group could specify how to alter the main Java (1.4) group templates in order to use generics and enumerated types.

Group inheritance would not yield its full potential without template polymorphism. A parser template instantiated via the Java1_5 group should always look for templates in Java1_5 rather than the Java supergroup even though that template is lexically defined within group Java.

Templates in a subgroup override same-named templates in a supergroup just as in class inheritance in other languages. StringTemplate does not support overloaded templates so group inheritance does not take formal arguments into consideration.

The supergroup for a group may be changed dynamically using the setSuperGroup() method. If, however, a group must always derive from another group, use the following syntax in the descendant template group file:

group mygroup : supergroup;
...

If your group must satisfy a particular interface (see Group interfaces) then use the following syntax:

group mygroup implements anInterface, andAnotherInterface;
...

or if the group inherits from a supergroup and implements an interface:

group mygroup : supergroup implements anInterface;
...

Maps

There are situations where you need to translate a string in one language to a string in another language. For example, you might want to translate integer to int when translating Pascal to C. You could pass a Map or IDictionary (e.g. hashtable) from the model into the templates, but then you have output literals in your model!  The StringTemplate solution is to support a mapping feature. For example, here is a map defined in a template group file, using which ANTLR v3 knows how to initialize local variables to their default values:

typeInitMap ::= [
        "int":"0",
        "long":"0",
        "float":"0.0",
        "double":"0.0",
        "boolean":"false",
        "byte":"0",
        "short":"0",
        "char":"0",
        default:"null" // anything other than an atomic type
]

To use the map in a template, refer to it as you would an attribute. Continuing the example, <typeInitMap.int> returns "0" from the map. If your type name is an attribute not a constant like int, then use an indirect field access: <typeInitMap.(typeName)>.

Map strings are actually templates that can refer to attributes that will become visible via dynamic scoping of attributes once the map entry has been embedded within a template. This is useful for referencing things like attribute username from within map values. That attribute will eventually become visible when the map a value is embedded within, say, a page template.

Large strings, such as those with newlines, can be specified with the usual large template delimiters from the group file format: <<...>>.

The default and other mappings can have empty values (implying no value). if no key is matched by the map then an empty value is returned, which is the same as using "default :" explicitly. The keyword key is available if you would like to refer to the key that maps to this value. This is particularly useful if you would like to filter certain words but otherwise leave a value unchanged; use default : key to return the key unmolested if it is not found in the map.

Maps are defined in the group's scope and are visible if no attribute hides them. For example, if you define a formal argument called typeInitMap in template foo then foo cannot see the map defined in the group (though you could pass it in as another parameter). If a name is not an attribute and it's not in the group's maps table, then the super group is consulted etc... You may not redefine a map and it may not have the same name as a template in that group. The default value is used if you use a key as a property that doesn't exist. For example <typeInitMap.foo> returns "null". The default clause must be at the end of the map.

You'll note that the square brackets will denote data structure in other areas too such as [a,b,c,...] which makes a singe multi-valued attribute out of other attributes so you can iterate across them.

Template Group file format

group
    :   "group" ID ( ':' ID  )?  ( "implements" ID (',' ID  )* )? ';'
        ( template | mapdef )+
    ;

template
    :   (   '@' ID '.' ID
        |   ID
        )
        '(' (args)? ')' "::="
        (   STRING      // "..."
        |   BIGSTRING   // <<...>>
        )
    |   ID "::=" ID     // alias one template to another
    ;

args:   arg ( ',' arg )*
    ;

arg :   ID '=' STRING               // x="..."
    |   ID '=' ANONYMOUS_TEMPLATE   // x={...}
    |   ID
    ;

mapdef
    :   ID "::=" map
    ;

map :   '['
            keyValuePair (',' keyValuePair)*
            ( ',' "default" ':' keyValue )?
        ']'
    ;

keyValuePair
    :   STRING ':' keyValue
    ;

keyValue
    :   BIGSTRING
    |   STRING
    |   "key"
    |
    ;

Both /* ... */ and single-line // ... comments are allowed outside of templates. Inside templates, you must use <!...!>.

An aside: All along, during my website construction days, I kept in mind that any text output follows a format and, thus, output sentences conform to a language. Consequently, a grammar should describe the output rather than a bunch of ad hoc print statements in code. This helped me formalize the study of templates because I could compare templates (output grammars) to well established ideas from formal language theory and context-free grammars. This allowed me to show, among other things, that StringTemplate can easily generate any document describable with an XML DTD even though it is deliberately limited. The group file format should look very much like a grammar to you.

Group files have a .stg file extension.

Template Group loaders

When a template group file derives from another group, StringTemplate has to know how to load that group and its supergroups. StringTemplate 2.3 introduces the StringTemplateGroupLoader interface to describe objects that know how to load groups and interfaces.

public interface StringTemplateGroupLoader {
    /** Load the group called groupName from somewhere.  Return null
     *  if no group is found.
     */
    public StringTemplateGroup loadGroup(String groupName);

    /** Load a group with a specified superGroup.  Groups with
     *  region definitions must know their supergroup to find templates
     *  during parsing.
     */
    public StringTemplateGroup loadGroup(String groupName,
                                         StringTemplateGroup superGroup);


    /** Load the interface called interfaceName from somewhere.  Return null
     *  if no interface is found.
     */
    public StringTemplateGroupInterface loadInterface(String interfaceName);
}

public interface IStringTemplateGroupLoader
{
    /// <summary>
    /// Loads the named StringTemplateGroup instance from somewhere.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName);

    /// <summary>
    /// Loads the named StringTemplateGroup instance with the specified supergroup from somewhere.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName, StringTemplateGroup superGroup);

    /// <summary>
    /// Loads the named StringTemplateGroup instance with the specified supergroup from somewhere.
    /// Configure to use lexer of specified type.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName, StringTemplateGroup superGroup, Type lexer);

    /// <summary>
    /// Loads the named StringTemplateGroupInterface instance from somewhere.
    /// </summary>
    StringTemplateGroupInterface LoadInterface(string interfaceName);
}

class StringTemplateGroupLoader(object):
    def loadGroup(self, groupName, superGroup=None):
        raise NotImplementedError

    def loadInterface(self, interfaceName):
        raise NotImplementedError

By default, there are two implementations: PathGroupLoader and CommonGroupLoader. PathGroupLoader is a simple loader that looks only in the directory(ies) you specify in the ctor (Note that you can specify the char encoding). CommonGroupLoader, on the other hand, is a loader that also looks in the directory(ies) you specify in the ctor, but it uses the classpath rather than absolute dirs so it can be used when the ST application is jar'd up. Use Static method:

StringTemplateGroup.registerGroupLoader(loader);

StringTemplateGroup.registerGroupLoader(loader)

to specify a loader. For example, here is how ANTLR loads its templates:

// get a group loader containing main templates dir and target subdir
String templateDirs =
    classpathTemplateRootDirectoryName+":"+
    classpathTemplateRootDirectoryName+"/"+language;
StringTemplateGroupLoader loader =
    new CommonGroupLoader(templateDirs.toString(),
                          ErrorManager.getStringTemplateErrorListener());
StringTemplateGroup.registerGroupLoader(loader);

// first load main language template
StringTemplateGroup coreTemplates =
    StringTemplateGroup.loadGroup(language);

In order to use the group file format inheritance specifier, group sub : sup, you must specify a loader.

Formal argument default values

Sometimes it is convenient to have default values for formal arguments that are used when no value is set by the model. For example, when generating a parser in Java from ANTLR, I want the super class of the generated object to be Parser unless the ANTLR user uses an option to set the super class to some custom class. For example, here is a partial parser template definition:

parser(name, rules, superClass="Parser") ::= ...

Any argument may be given a default value by following the name with an equals sign and a string or an anonymous template.

Formal argument error handling

When using a group file format to specify templates, you must specify the formal arguments for that template. If you try to set an attribute via setAttribute that is not specifically formally defined in that template, you will get the following exception:

If you reference an attribute that is not formally defined in that template or any enclosing template, you also get the same exception.

Newline handling

The first newline following the << in a template definition is ignored as it is usually used just to get the first line of text for the template at the start of a line. In other words, if you want to have a blank line at the start of your template, use:

foo() ::= <<

2nd line is not blank, but first is
>>

or

foo() ::= <<<\n>
same as before; newline then this line
>>

The last newline before the >> is also ignored and is not included in the output. To add a final newline, add an extra line or <\n> before the >>:

foo() ::= <<
rodent

>>

or

foo() ::= <<
rodent<\n>
>>

The following template:

foo() ::= <<
rodent
>>

on the other hand, is identical to

foo() ::= "rodent"