Related material

Principles

• It is highly recommended that you read the (academically-oriented) paper, Enforcing Model-View Separation in Template Engines.
• DRAFT A Functional Language For Generating Structured Text
• Internationalization and Localization of Web Applications In Action
• Language Translation Using ANTLR and StringTemplate
• Intelligent Web Site Page Generation
• The Role of Template Engines in Code Generation

Alerts to Developers

• The StringTemplates distribution includes many unit tests that also represent a useful set of examples. The tests are defined in:

  Java	TestStringTemplate.java
  C#	TestStringTemplate.cs
  Python	TestStringTemplate.py

• Please see the Release Notes, changes, and Bugs page
  • These pages generally discuss the Java version of StringTemplate but, some of the information they contain might apply to other implementations.

Introduction

Most programs that emit source code or other text output are unstructured blobs of generation logic interspersed with print statements. The primary reason is the lack of suitable tools and formalisms. The proper formalism is that of an output grammar because you are not generating random characters--you are generating sentences in an output language. This is analogous to using a grammar to describe the structure of input sentences. Rather than building a parser by hand, most programmers will use a parser generator. Similarly, we need some form of unparser generator to generate text. The most convenient manifestation of the output grammar is a template engine such as StringTemplate.

A template engine is simply a code generator that emits text using templates, which are really just "documents with holes" in them where you can stick values. StringTemplate breaks up your template into chunks of text and attribute expressions, which are by default enclosed in dollar signs $attribute-expression$ (to make them easy to see in HTML files). StringTemplate ignores everything outside of attribute expressions, treating it as just text to spit out when you call:

For example, the following template has two chunks, a literal and a reference to attribute name:

Hello, $name$

Using templates in code is very easy. Here is the requisite example that prints "Hello, World":

import org.antlr.stringtemplate.*;

StringTemplate hello = new StringTemplate("Hello, $name$");
hello.setAttribute("name", "World");
System.out.println(hello.toString());

using Antlr.StringTemplate;

StringTemplate hello = new StringTemplate("Hello, $name$");
hello.SetAttribute("name", "World");
Console.Out.WriteLine(hello.ToString());

import stringtemplate3

hello = stringtemplate3.StringTemplate("Hello, $name$")
hello["name"] = "World"
print str(hello)

StringTemplate is not a "system" or "engine" or "server"; it is a library with two primary classes of interest: StringTemplate and StringTemplateGroup. You can directly create a StringTemplate in code, you can load a template from a file, and you can load a single file with many templates (a template group file).

Motivation And Philosophy

StringTemplate was born and evolved during the development of http://www.jGuru.com. The need for such dynamically-generated web pages has led to the development of numerous other template engines in an attempt to make web application development easier, improve flexibility, reduce maintenance costs, and allow parallel code and HTML development. These enticing benefits, which have driven the proliferation of template engines, derive entirely from a single principle: separating the specification of a page's business logic and data computations from the specification of how a page displays such information.

These template engines are in a sense a reaction to the completely entangled specifications encouraged by JSP (Java Server Pages), ASP (Active Server Pages) and, even ASP.NET. With separate encapsulated specifications, template engines promote component reuse, pluggable site "looks", single-points-of-change for common components, and high overall system clarity. In the code generation realm, model-view separation guarantees retargetability.

The normal imperative programming language features like setting variables, loops, arithmetic expressions, arbitrary method calls into the model, etc... are not only unnecessary, but they are very specifically what is wrong with ASP/JSP. Recall that ASP/JSP (and ASP.NET) allow arbitrary code expressions and statements, allowing programmers to incorporate computations and logic in their templates. A quick scan of template engines reveals an unfortunate truth--all but a few are Turing-complete languages just like ASP/JSP/ASP.NET. One can argue that they are worse than ASP/JSP/ASP.NET because they use languages peculiar to that template engine. Many tool builders have clearly lost sight of the original problem we were all trying to solve. We programmers often get caught up in cool implementations, but we should focus on what should be built not what can be built.

The fact that StringTemplate does not allow such things as assignments (no side-effects) should make you suspicious of engines that do allow it. The templates in ANTLR v3's code generator are vastly more complicated than the sort of templates typically used in web pages creation with other template engines yet, there hasn't been a situation where assignments were needed. If your template looks like a program, it probably is--you have totally entangled your model and view.

After examining hundreds of template files that I created over years of jGuru.com (and now in ANTLR v3) development, I found that I needed only the following four basic canonical operations (with some variations):

• attribute reference; e.g., $phoneNumber$
• template reference (like #include or macro expansion); e.g., $searchbox()$
• conditional include of subtemplate (an IF statement); e.g., $if(title)$<title>$title$</title>$endif$
• template application to list of attributes; e.g., $names:bold()$

where template references can be recursive.

Language theory supports my premise that even a minimal StringTemplate engine with only these features is very powerful--such an engine can generate the context-free languages (see Enforcing Strict Model-View Separation in Template Engines); e.g., most programming languages are context-free as are any XML pages whose form can be expressed with a DTD.

While providing all sorts of dangerous features like assignment that promote the use of computations and logic in templates, many engines miss the key elements. Certain language semantics are absolutely required for generative programming and language translation. One is recursion. A template engine without recursion seems unlikely to be capable of generating recursive output structures such as nested tables or nested code blocks.

Another distinctive StringTemplate language feature lacking in other engines is lazy-evaluation. StringTemplate's attributes are lazily evaluated in the sense that referencing attribute "a" does not actually invoke the data lookup mechanism until the template is asked to render itself to text. Lazy evaluation is surprisingly  useful in both the web and code generation worlds because such order decoupling allows code to set attributes when it is convenient or efficient not necessarily before a template that references those attributes is created. For example, a complicated web page may consist of many nested templates many of which reference $userName$, but the value of userName does not need to be set by the model until right before the entire page is rendered to text via ToString(). You can build up the complicated page, setting attribute values in any convenient order.

StringTemplate implements a "poor man's" form of lazy evaluation by simply requiring that all attributes be computed a priori. That is, all attributes must be computed and pushed into a template before it is written to text; this is the so-called "push method" whereas most template engines use the "pull method". The pull method appears more conventional because programmers mistakenly regard templates as programs, but pulling attributes introduces order-of-computation dependencies. Imagine a simple web page that displays a list of names (using some mythical Java-based template engine notation):

<html>
<body>
<ol>
$foreach n in names$
  <li>$n$</li>
$end$
</ol>
There are $numberNames$ names.
</body>
</html>

Using the pull method, the reference to names invokes model.getNames(), which presumably loads a list of names from the database. The reference to numberNames invokes model.getNumberNames() which necessarily uses the internal data structure computed by getNames() to compute names.size() or whatever. Now, suppose a designer moves the numberNames reference to the <title> tag, which is before the reference to names in the foreach statement. The names will not yet have been loaded, yielding a null pointer exception at worst or a blank title at best. You have to anticipate these dependencies and have getNumberNames() invoke getNames() because of a change in the template.

I'm stunned that other template engine authors with whom I've spoken think this is ok. Any time I can get the computer to do something automatically for me that removes an entire class of programming errors, I'll take it! Automatic garbage collection is the obvious analogy here.

The pull method requires that programmers do a topological sort in their minds anticipating any order that a programmer or designer could induce. To ensure attribute computation safety (i.e., avoid hidden dependency landmines), I have shown trivially in my academic paper that pull reduces to push in the worst case. With a complicated mesh of templates, you will miss a dependency, thus, creating a really nasty, difficult-to-find bug.

StringTemplate mission

When developing StringTemplate, I recalled Frederick Brook's book, "Mythical Man Month", where he identified conceptual integrity as a crucial product ingredient. For example, in UNIX everything is a stream. My concept, if you will, is strict model-view separation. My mission statement is therefore:

"StringTemplate shall be as simple, consistent, and powerful as possible without sacrificing strict model-view separation."

I ruthlessly evaluate all potential features and functionality against this standard. Over the years, however, I have made certain concessions to practicality that one could consider as infringing ever-so-slightly into potential model-view entanglement. That said, StringTemplate still seems to enforce separation while providing excellent functionality.

I let my needs dictate the language and tool feature set. The tool evolved as my needs evolved. I have done almost no feature "backtracking". Further, I have worked really hard to make this little language self-consistent and consistent with existing syntax/metaphors from other languages. There are very few special cases and attribute/template scoping rules make a lot of sense even if they are unfamiliar or strange at first glance. Everything in the language exists to solve a very real need.

StringTemplate language flavor

Just so you know, I've never been a big fan of functional languages and I laughed really hard when I realized (while writing the academic paper) that I had implemented a functional language. The nature of the problem simply dictated a particular solution. We are generating sentences in an output language so we should use something akin to a grammar. Output grammars are inconvenient so tool builders created template engines. Restricted template engines that enforce the universally-agreed-upon goal of strict model-view separation also look remarkably like output grammars as I have shown. So, the very nature of the language generation problem dictates the solution: a template engine that is restricted to support a mutually-recursive set of templates with side-effect-free and order-independent attribute references.

Expression elements

See Expressions

Statements

See Conditionally included subtemplates

Groups

See Group Files

group name
t1(args) ::= "template1"
t2(args) ::= <<
template2>>

Also

group name implements interfacename;
...

where the interface is defined via:

interface interfacename;
t1(args);
t2(args);

Reserved words

Don't use these as attribute names or template names:

Defining Templates

Creating Templates With Code

Here is a simple example that creates and uses a template on the fly:

StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("table", "User");

StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("table", "User");

query = stringtemplate3.StringTemplate("SELECT $column$ FROM $table$;")
query["column"] = "name"
query["table"] = "User"

where StringTemplate considers anything in $...$ to be something it needs to pay attention to. By setting attributes, you are "pushing" values into the template for use when the template is printed out. The attribute values are set by referencing their names. Invoking toString() on query would yield

SELECT name FROM User;

You can set an attribute multiple times, which simply means that the attribute is multi-valued. For example, adding another value to the attribute named column as shown below makes the attribute multi-valued:

StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User");

StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");

query = stringtemplate3.StringTemplate("SELECT $column$ FROM $table$;")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User"

Invoking toString() on query would now yield

SELECT nameemail FROM User;

Ooops...there is no separator between the multiple values. If you want a comma, say, between the column names, then change the template to record that formatting information:

StringTemplate query = new StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User");

StringTemplate query = new StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");

query = stringtemplate3.StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User"

Note that the right-hand-side of the separator specification in this case is a string literal; therefore, we have escaped the double-quotes as the template is specified in a string. In general, the right-hand-side can be any attribute expression. Invoking toString() on query would now yield

SELECT name,email FROM User;

Attributes can be any object at all. StringTemplate calls toString() on each object as it writes the template out. The separator is not used unless the attribute is multi-valued.

Loading Templates From Files

To load a template from the disk you must use a StringTemplateGroup that will manage all the templates you load, caching them so you do not waste time talking to the disk for each template fetch request (you can change it to not cache; see below). You may have multiple template groups. Here is a simple example that loads the previous SQL template from a file /tmp/theQuery.st:

SELECT $column; separator=","$ FROM $table$;

The code below creates a StringTemplateGroup called myGroup rooted at /tmp so that requests for template theQuery forces a load of file /tmp/theQuery.st.

StringTemplateGroup group = new StringTemplateGroup("myGroup", "/tmp");
StringTemplate query = group.getInstanceOf("theQuery");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User");

StringTemplateGroup group = new StringTemplateGroup("myGroup", "/tmp");
StringTemplate query = group.GetInstanceOf("theQuery");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");

group = stringtemplate3.StringTemplateGroup("myGroup", "/tmp")
query = group.getInstanceOf("theQuery")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User"

If you have a directory hierarchy of templates such as file /tmp/jguru/bullet.st, you would reference them relative to the root; in this case, you would ask for template jguru/bullet().

Loading Templates relative to an implementation specific location

// Look for templates in CLASSPATH as resources
StringTemplateGroup group = new StringTemplateGroup("mygroup");
StringTemplate st = group.getInstanceOf("com/mycompany/server/templates/page");

// Look for templates relative to assembly location
StringTemplateGroup group = new StringTemplateGroup("mygroup", (string)null);
StringTemplate st = group.GetInstanceOf("templates/page");

If page.st references, say, searchbox template, it must be fully qualified as:
<font size=2>SEARCH</font>: $com/mycompany/server/templates/page/searchbox()$

This is inconvenient and ST may add the invoking template's path prefix automatically in the future.

Caching

By default templates are loaded from disk just once. During development, however, it is convenient to turn caching off. Also, you may want to turn off caching so that you can quickly update a running site. You can set a simple refresh interval using StringTemplateGroup.setRefreshInterval(...). When the interval is reached, all templates are thrown out. Set interval to 0 to refresh constantly (no caching). Set the interval to a huge number like Integer.MAX_INT or Int32.MaxValue to have no refreshing at all.

StringTemplateGroup group = new StringTemplateGroup("myGroup", "/tmp");
group.setRefreshInterval(0);  // no caching
group.setRefreshInterval(Integer.MAX_INT);  // no refreshing

group = stringtemplate3.StringTemplateGroup("myGroup", "/tmp")
group.refreshInterval = 0  # no caching
group.refreshInterval = sys.maxint # no refreshing

By default, expressions in a template are delimited by dollar signs: $...$. This works great for the most common case of HTML generation because the attribute expressions are clearly highlighted in the text. Sometimes, with other formats like SQL statement generation, you may want to change the template expression delimiters to avoid a conflict and to make the expressions stand out.

The start and stop strings are limited to either $...$ or <...> (unless you build your own lexical analyzer to break apart templates into chunks). group file templates use <...> delimiters by default (in v2.2 $...$ was the default delimiter). Templates created with the StringTemplate object constructor still use $...$ by default.

To specify that StringTemplate should use a specific delimiter you must create a StringTemplateGroup:

StringTemplateGroup group =
  new StringTemplateGroup("sqlstuff", "/tmp", AngleBracketTemplateLexer.class);
StringTemplate query =
  new StringTemplate(group, "SELECT <column> FROM <table>;");
query.setAttribute("column", "name");
query.setAttribute("table", "User");

StringTemplateGroup group =
     new StringTemplateGroup("sqlstuff", "/tmp", typeof(AngleBracketTemplateLexer));
StringTemplate query = new StringTemplate(group, "SELECT <column> FROM <table>;");
query.SetAttribute("column", "name");
query.SetAttribute("table", "User");

group = stringtemplate3.StringTemplateGroup("sqlstuff", "/tmp", lexer="angle-bracket")
query = stringtemplate3.StringTemplate("SELECT <column> FROM <table>;", group=group)
query["column"] = "name"
query["table"] = "User"

All templates created through the group or in anyway associated with the group will assume your the angle bracket delimiters. It's smart to be consistent across all files of similar type such as "all HTML templates use $...$" and "all SQL templates use <...>".

Template Group Files

Overview of Template Groups

StringTemplate 2.0 introduced the notion of a template group file that has two main attractions. First, it allows you to define lots of small templates more conveniently because they may all be defined within a single file. Second, (unlike in a simple template file) you may specify formal template arguments that help StringTemplate detect errors (such as setting unknown attributes) and make the templates easier to read. Here is a sample group file with two templates, vardef and method, that could be used to generate C files:

group simple;

vardef(type,name) ::= "<type> <name>;"

method(type,name,args) ::= <<
<type> <name>(<args; separator=",">) {
  <statements; separator="\n">
}
>>

All groups use <...> expression  delimiters by default. Single line templates are enclosed in double quotes while multi-line templates are enclosed in double angle-brackets. Every template must define arguments even if the formal argument list is blank.

Using the templates from a template group file (or from a string) is straightforward. The StringTemplateGroup class has a number of constructors, one of which allows you to pass in a string or file or whatever:

String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a Reader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.getInstanceOf("vardef");
t.setAttribute("type", "int");
t.setAttribute("name", "foo");
System.out.println(t);

String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a System.IO.TextReader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.GetInstanceOf("vardef");
t.SetAttribute("type", "int");
t.SetAttribute("name", "foo");
Console.Out.WriteLine(t);

templates = "group simple; vardef(type,name) ..."; # templates from above
# Use the constructor that accepts a Reader
group = stringtemplate3.StringTemplateGroup(file=StringIO(templates))
t = group.getInstanceOf("vardef")
t["type"] = "int"
t["name"] = "foo"
print str(t)

The output would be: "int foo;".

Supergroups and interfaces

Template groups may derive from other template groups, thus inheriting all of the features (templates and maps) from the supergroup. Group inheritance provides an appropriate model whereby, for example, a variation on a code generation target may be defined by describing how it differs from a previously defined target. Considering Java 1.4 versus 1.5, a Java1_5 group could specify how to alter the main Java (1.4) group templates in order to use generics and enumerated types.

Group inheritance would not yield its full potential without template polymorphism. A parser template instantiated via the Java1_5 group should always look for templates in Java1_5 rather than the Java supergroup even though that template is lexically defined within group Java.

Templates in a subgroup override same-named templates in a supergroup just as in class inheritance in other languages. StringTemplate does not support overloaded templates so group inheritance does not take formal arguments into consideration.

The supergroup for a group may be changed dynamically using the setSuperGroup() method. If, however, a group must always derive from another group, use the following syntax in the descendant template group file:

group mygroup : supergroup;
...

If your group must satisfy a particular interface (see Group interfaces) then use the following syntax:

group mygroup implements anInterface, andAnotherInterface;
...

or if the group inherits from a supergroup and implements an interface:

group mygroup : supergroup implements anInterface;
...

Maps

There are situations where you need to translate a string in one language to a string in another language. For example, you might want to translate integer to int when translating Pascal to C. You could pass a Map or IDictionary (e.g. hashtable) from the model into the templates, but then you have output literals in your model!  The StringTemplate solution is to support a mapping feature. For example, here is a map defined in a template group file, using which ANTLR v3 knows how to initialize local variables to their default values:

typeInitMap ::= [
        "int":"0",
        "long":"0",
        "float":"0.0",
        "double":"0.0",
        "boolean":"false",
        "byte":"0",
        "short":"0",
        "char":"0",
        default:"null" // anything other than an atomic type
]

To use the map in a template, refer to it as you would an attribute. Continuing the example, <typeInitMap.int> returns "0" from the map. If your type name is an attribute not a constant like int, then use an indirect field access: <typeInitMap.(typeName)>.

Map strings are actually templates that can refer to attributes that will become visible via dynamic scoping of attributes once the map entry has been embedded within a template. This is useful for referencing things like attribute username from within map values. That attribute will eventually become visible when the map a value is embedded within, say, a page template.

Large strings, such as those with newlines, can be specified with the usual large template delimiters from the group file format: <<...>>.

The default and other mappings can have empty values (implying no value). if no key is matched by the map then an empty value is returned, which is the same as using "default :" explicitly. The keyword key is available if you would like to refer to the key that maps to this value. This is particularly useful if you would like to filter certain words but otherwise leave a value unchanged; use default : key to return the key unmolested if it is not found in the map.

Maps are defined in the group's scope and are visible if no attribute hides them. For example, if you define a formal argument called typeInitMap in template foo then foo cannot see the map defined in the group (though you could pass it in as another parameter). If a name is not an attribute and it's not in the group's maps table, then the super group is consulted etc... You may not redefine a map and it may not have the same name as a template in that group. The default value is used if you use a key as a property that doesn't exist. For example <typeInitMap.foo> returns "null". The default clause must be at the end of the map.

You'll note that the square brackets will denote data structure in other areas too such as [a,b,c,...] which makes a singe multi-valued attribute out of other attributes so you can iterate across them.

Template Group file format

group
    :   "group" ID ( ':' ID  )?  ( "implements" ID (',' ID  )* )? ';'
        ( template | mapdef )+
    ;

template
    :   (   '@' ID '.' ID
        |   ID
        )
        '(' (args)? ')' "::="
        (   STRING      // "..."
        |   BIGSTRING   // <<...>>
        )
    |   ID "::=" ID     // alias one template to another
    ;

args:   arg ( ',' arg )*
    ;

arg :   ID '=' STRING               // x="..."
    |   ID '=' ANONYMOUS_TEMPLATE   // x={...}
    |   ID
    ;

mapdef
    :   ID "::=" map
    ;

map :   '['
            keyValuePair (',' keyValuePair)*
            ( ',' "default" ':' keyValue )?
        ']'
    ;

keyValuePair
    :   STRING ':' keyValue
    ;

keyValue
    :   BIGSTRING
    |   STRING
    |   "key"
    |
    ;

Both /* ... */ and single-line // ... comments are allowed outside of templates. Inside templates, you must use <!...!>.

An aside: All along, during my website construction days, I kept in mind that any text output follows a format and, thus, output sentences conform to a language. Consequently, a grammar should describe the output rather than a bunch of ad hoc print statements in code. This helped me formalize the study of templates because I could compare templates (output grammars) to well established ideas from formal language theory and context-free grammars. This allowed me to show, among other things, that StringTemplate can easily generate any document describable with an XML DTD even though it is deliberately limited. The group file format should look very much like a grammar to you.

Group files have a .stg file extension.

Template Group loaders

When a template group file derives from another group, StringTemplate has to know how to load that group and its supergroups. StringTemplate 2.3 introduces the StringTemplateGroupLoader interface to describe objects that know how to load groups and interfaces.

public interface StringTemplateGroupLoader {
    /** Load the group called groupName from somewhere.  Return null
     *  if no group is found.
     */
    public StringTemplateGroup loadGroup(String groupName);

    /** Load a group with a specified superGroup.  Groups with
     *  region definitions must know their supergroup to find templates
     *  during parsing.
     */
    public StringTemplateGroup loadGroup(String groupName,
                                         StringTemplateGroup superGroup);


    /** Load the interface called interfaceName from somewhere.  Return null
     *  if no interface is found.
     */
    public StringTemplateGroupInterface loadInterface(String interfaceName);
}

public interface IStringTemplateGroupLoader
{
    /// <summary>
    /// Loads the named StringTemplateGroup instance from somewhere.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName);

    /// <summary>
    /// Loads the named StringTemplateGroup instance with the specified supergroup from somewhere.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName, StringTemplateGroup superGroup);

    /// <summary>
    /// Loads the named StringTemplateGroup instance with the specified supergroup from somewhere.
    /// Configure to use lexer of specified type.
    /// </summary>
    StringTemplateGroup LoadGroup(string groupName, StringTemplateGroup superGroup, Type lexer);

    /// <summary>
    /// Loads the named StringTemplateGroupInterface instance from somewhere.
    /// </summary>
    StringTemplateGroupInterface LoadInterface(string interfaceName);
}

class StringTemplateGroupLoader(object):
    def loadGroup(self, groupName, superGroup=None):
        raise NotImplementedError

    def loadInterface(self, interfaceName):
        raise NotImplementedError

By default, there are two implementations: PathGroupLoader and CommonGroupLoader. PathGroupLoader is a simple loader that looks only in the directory(ies) you specify in the ctor (Note that you can specify the char encoding). CommonGroupLoader, on the other hand, is a loader that also looks in the directory(ies) you specify in the ctor, but it uses the classpath rather than absolute dirs so it can be used when the ST application is jar'd up. Use Static method:

StringTemplateGroup.registerGroupLoader(loader);

StringTemplateGroup.registerGroupLoader(loader)

to specify a loader. For example, here is how ANTLR loads its templates:

// get a group loader containing main templates dir and target subdir
String templateDirs =
    classpathTemplateRootDirectoryName+":"+
    classpathTemplateRootDirectoryName+"/"+language;
StringTemplateGroupLoader loader =
    new CommonGroupLoader(templateDirs.toString(),
                          ErrorManager.getStringTemplateErrorListener());
StringTemplateGroup.registerGroupLoader(loader);

// first load main language template
StringTemplateGroup coreTemplates =
    StringTemplateGroup.loadGroup(language);

In order to use the group file format inheritance specifier, group sub : sup, you must specify a loader.

Formal argument default values

Sometimes it is convenient to have default values for formal arguments that are used when no value is set by the model. For example, when generating a parser in Java from ANTLR, I want the super class of the generated object to be Parser unless the ANTLR user uses an option to set the super class to some custom class. For example, here is a partial parser template definition:

parser(name, rules, superClass="Parser") ::= ...

Any argument may be given a default value by following the name with an equals sign and a string or an anonymous template.

Formal argument error handling

When using a group file format to specify templates, you must specify the formal arguments for that template. If you try to set an attribute via setAttribute that is not specifically formally defined in that template, you will get the following exception:

If you reference an attribute that is not formally defined in that template or any enclosing template, you also get the same exception.

Newline handling

The first newline following the << in a template definition is ignored as it is usually used just to get the first line of text for the template at the start of a line. In other words, if you want to have a blank line at the start of your template, use:

foo() ::= <<

2nd line is not blank, but first is
>>

or

foo() ::= <<<\n>
same as before; newline then this line
>>

The last newline before the >> is also ignored and is not included in the output. To add a final newline, add an extra line or <\n> before the >>:

foo() ::= <<
rodent

>>

or

foo() ::= <<
rodent<\n>
>>

The following template:

foo() ::= <<
rodent
>>

on the other hand, is identical to

foo() ::= "rodent"

Group Interfaces

To promote retargetable code generators, ST supports interface implementation a la Java interfaces where a template group that implements an interface must implement all templates in the interface and with the proper argument lists. The interface is the published, executable documentation for building back-ends for the code generator and has proven to be an excellent way to inform programmers responsible for the various targets of changes to the requirements.

The developers of the ANTLR code generation targets always have the same two questions: Initially they ask, "What is the set of templates I have to define for my target?'' and then, during development, they ask, "Has a change to the code generation logic forced any changes to the requirements of my template library?"

Originally, the answer to the first question involved abstracting the list of templates and their formal arguments from the existing Java target. The answer to the second question involved using a difference tool to point out changes in the Java target from repository check-in to check-in. Without a way to formally notify target developers and to automatically catch logic-template mismatches, bugs creep in that become apparent only when the stale template definitions are exercised by the code generator. This situation is analogous to programs in dynamically typed languages like Python where method signature changes can leave landmines in unexercised code. In short, there were no good answers.

ST now supports group interfaces that describe a collection of template signatures, names and formal arguments, in a manner analogous to Java interfaces. Interfaces clearly identify the set of all templates that a target must define as well as the attributes they operate on. The first question regarding the required set of templates now has a good answer.

Interfaces also provide a form of type safety whereby a target is examined upon code generator startup to see that it satisfies the interface. Here is a piece of the ANTLR main target interface:

interface ANTLRCore;
parser(name, scopes, tokens, tokenNames, rules,
       numRules, cyclicDFAs, bitsets, ASTLabelType,
       superClass, labelType, members);
rule(ruleName, ruleDescriptor, block, emptyRule,
       description, exceptions);
/** What file extension to use; e.g., ".java" */
codeFileExtension();
...

All of the various targets then implement the interface; e.g.,

group Java implements ANTLRCore;

The code generator, which loads target templates, notifies developers of any inconsistencies immediately upon startup effectively answering the second question regarding notification of template library changes. Group interfaces provide excellent documentation, promote consistency, and reduce hidden bugs.

Interfaces look exactly like groups except that they don't have template implementations for the template declarations although they must have the complete parameter list. Further, a template may be defined as optional using the optional keyword:

optional headerFile(actionScope, actions, docComment, recognizer, ...);

Template Group inheritance overview

A template group may derive from other template groups, thus inheriting all of the features (templates and maps) from the supergroup. The descendant template group can then add more templates and maps, override inherited templates and maps or modify inherited templates in a finer-grained manner using the template regions feature.

Template Group inheritance

Recall that a StringTemplateGroup is a collection of related templates such as all templates associated with the look of a web site. If you want to design a second similar look for that site (such as for premium users), you don't really want to cut-n-paste the original template files for use in the new look. Subsequent changes to the original template files would not be propagated to the new look.

Just like you would do with a class definition in other languages, a template group may inherit features (templates and maps) from another template group, the supergroup. If template t is not found in a group, it is looked up in the supergroup, if present. This works regardless of whether you use a group file format or load templates from the disk via a StringTemplateGroup object. Currently you cannot use the group file syntax to specify a supergroup. I am investigating how this should work. In the meantime, you must explicitly set the supergroup in code.

I think this is now handled by the syntax discussed in Group Files. - GW

group mygroup : supergroup;
...

From the unit tests, here is a simple inheritance of a template, bold:

StringTemplateGroup supergroup = new StringTemplateGroup("super");
StringTemplateGroup subgroup = new StringTemplateGroup("sub");
supergroup.defineTemplate("bold", "<b>$it$</b>");
subgroup.setSuperGroup(supergroup);
StringTemplate st = new StringTemplate(subgroup, "$name:bold()$");
st.setAttribute("name", "Terence");
String expecting = "<b>Terence</b>";

StringTemplateGroup supergroup = new StringTemplateGroup("super");
StringTemplateGroup subgroup = new StringTemplateGroup("sub");
supergroup.DefineTemplate("bold", "<b>$it$</b>");
subgroup.SuperGroup = supergroup;
StringTemplate st = new StringTemplate(subgroup, "$name:bold()$");
st.SetAttribute("name", "Terence");
string expecting = "<b>Terence</b>";

supergroup = stringtemplate3.StringTemplateGroup("super")
subgroup = stringtemplate3.StringTemplateGroup("sub", superGroup=group)
supergroup.defineTemplate("bold", "<b>$it$</b>")
st = stringtemplate3.StringTemplate("$name:bold()$", group=subgroup)
st["name"] = "Terence"
expecting = "<b>Terence</b>"

The supergroup has a bold definition but the subgroup does not. Referencing $name:bold()$ from a template in the subgroup works because StringTemplate looks into the supergroup if a referenced template is not found in the subgroup..

A template in a subgroup may override a template inhererited from a supergroup:

supergroup.defineTemplate("bold", "<b>$it$</b>");
subgroup.defineTemplate("bold", "<strong>$it$</strong>");

supergroup.DefineTemplate("bold", "<b>$it$</b>");
subgroup.DefineTemplate("bold", "<strong>$it$</strong>");

supergroup.defineTemplate("bold", "<b>$it$</b>");
subgroup.defineTemplate("bold", "<strong>$it$</strong>");

And a template in a subgroup may refer to a template in a supergroup via super.template():

StringTemplateGroup group = new StringTemplateGroup(...);
StringTemplateGroup subGroup = new StringTemplateGroup(...);
subGroup.setSuperGroup(group);
group.defineTemplate("page", "$font()$:text");
group.defineTemplate("font", "Helvetica");
subGroup.defineTemplate("font", "$super.font()$ and Times");
StringTemplate st = subGroup.getInstanceOf("page");

StringTemplateGroup group = new StringTemplateGroup(...);
StringTemplateGroup subGroup = new StringTemplateGroup(...);
subGroup.SuperGroup = group;
group.DefineTemplate("page", "$font()$:text");
group.DefineTemplate("font", "Helvetica");
subGroup.DefineTemplate("font", "$super.font()$ and Times");
StringTemplate st = subGroup.GetInstanceOf("page");

group = stringtemplate3.StringTemplateGroup(...)
subGroup = stringtemplate3.StringTemplateGroup(...)
subGroup.setSuperGroup(group)
group.defineTemplate("page", "$font()$:text")
group.defineTemplate("font", "Helvetica")
subGroup.defineTemplate("font", "$super.font()$ and Times")
st = subGroup.getInstanceOf("page")

The expression st.ToString() results in "Helvetica and Times:text".

Just like object-oriented programming languages, StringTemplate has polymorphism. That is, template names are looked up dynamically relative to the invoking template's group.

The classic demonstration of dynamic message sends, for example, would be the following example (this catches my students all the time):

Java	class A { public void page() {bold();} public void bold() {System.out.println("A.bold");} } class B extends A { public void bold() {System.out.println("B.bold");} } ... A a = new B(); a.page();
C#	class A { public void page() {bold();} override public void bold() {Console.Out.WriteLine("A.bold");} } class B : A { virtual public void bold() {Console.Out.WriteLine("B.bold");} } ... ... A a = new B(); a.page();

This prints "B.bold" not "A.bold" because the receiver determines how to answer a message not the type of the variable. So, I have created a B object meaning that any message, such as bold(), invoked will first look in class B for bold().

Similarly, a template's group determines where it starts looking for a template. In this case, both super and sub groups define a bold template mirroring the code above. Because I create template st as a member of subGroup, any reference to bold (say while processing st.ToString()) prompts StringTemplate to start looking for the bold template in subGroup, even though bold is referenced via the page template which is a member of the supergroup..

StringTemplateGroup group = new StringTemplateGroup("super");
StringTemplateGroup subGroup = new StringTemplateGroup("sub");
subGroup.setSuperGroup(group);
group.defineTemplate("bold", "<b>$it$</b>");
group.defineTemplate("page", "$name:bold()$");
subGroup.defineTemplate("bold", "<strong>$it$</strong>");
StringTemplate st = subGroup.getInstanceOf("page");
st.setAttribute("name", "Ter");
String expecting = "<strong>Ter</strong>";

StringTemplateGroup group = new StringTemplateGroup("super");
StringTemplateGroup subGroup = new StringTemplateGroup("sub");
subGroup.SuperGroup = group;
group.DefineTemplate("bold", "<b>$it$</b>");
group.DefineTemplate("page", "$name:bold()$");
subGroup.DefineTemplate("bold", "<strong>$it$</strong>");
StringTemplate st = subGroup.GetInstanceOf("page");
st.SetAttribute("name", "Ter");
string expecting = "<strong>Ter</strong>";

group = stringtemplate3.StringTemplateGroup("super")
subGroup = stringtemplate3.StringTemplateGroup("sub", superGroup=group)
group.defineTemplate("bold", "<b>$it$</b>")
group.defineTemplate("page", "$name:bold()$")
subGroup.defineTemplate("bold", "<strong>$it$</strong>")
st = subGroup.getInstanceOf("page")
st["name"] = "Ter"
expecting = "<strong>Ter</strong>"

StringTemplate group maps also inherit. If an attribute reference is not found, StringTemplate looks for a map in its group with that name. If not found, the super group is checked.

See more extensive details regarding template and attribute lookup here: Template and attribute lookup rules

Template regions

ST introduces a finer-grained alternative to template inheritance, dubbed regions. (Regions are similar to a feature in Django). This feature allows a programmer to mark a location or series of lines in a template, and give it a name. A subgroup which inherits this template can provide replacement code to override just the named region. This avoids having to override the supergroup's template with a whole replacement template, when just a small addition or replacement is needed. While regions are syntactic sugar on top of template inheritance, the improvement in simplicity and clarity over normal coarser-grained inheritance is substantial.

Add text at a location

For example, in a code-generation scenario, imagine using the following template called method to produce the text for a method:

group Java;
method(name,code) ::= <<
public void <name>() {
    <code>
}
>>

Suppose that you also want the option for the method template to place debugging statements into the generated method code. (To be clear about this example: this would be debugging code in the generated Java method, not code to debug the template processing itself.)

You could start placing debug text into the existing template, making it optional using the conditionally-included subtemplates feature, placing <if(...)> etc  around the debugging lines. But that clutters up the templates of the Java group considerably, and also fails to achieve proper separation of concerns.

Instead you would like to have all debugging stuff encapsulated in a separate template group which focuses on debugging. In that template group, you could create an overriding template for method by copying and pasting the entire existing method template and inserting your additions. But then you are duplicating all of that output literal text, which breaks the "single point of change principle."

Instead just leave a hole in the main method template that a subgroup can override, here a location marked with <@preamble()>:

group Java;
method(name,code) ::= <<
public void <name>() {
    <@preamble()>
    <code>
}
>>

In a template subgroup focusing on debugging (group dbg), define the region using a fully qualified name which includes the region's surrounding template name, @method.preamble(), and supply the replacement text:

group dbg : Java;
@method.preamble() ::= <<System.out.println("enter");>>

Regions are like subtemplates scoped within a template, hence, the fully-qualified name of a region is @t.r() where t is the enclosing template and r is the region name.

Replace a region of existing template text

Consider another problem where you would like, in a template subgroup, to replace a small portion of a large inherited template. Imagine you have a template that generates conditional statements in the output language, but you would also like to be able to generate a debug version of these statements which track the fact that an expression was evaluated.

(To be clear about this example, ths template's purpose is to produce "if" statements in the output language, here Java. That "if" is unrelated to the issue of using template <if(...)> expressions, which we are discussing how to avoid.)

Again, to avoid mingling debug version code with your main templates, you want to avoid "if dbg" type template expressions. Instead, mark the region within the template that might be replaced by an inheriting subgroup focusing on debugging. Here the code is marked with the pair of markers <@eval>...<@end>:

group Java;
test(expr,code) ::= "if (<@eval><expr><@end>) {<code>}"

where <@r>..<@end> marks the region called r. Now a template subgroup can override (replace) this region:

group dbg : Java;
@test.eval() ::= "trackAndEval(<expr>)"

Regions may not have parameters, but because of the dynamic scoping of attributes, the overridden region may access all of the attributes of the surrounding template.

In an overridden region, @super.r()refers to the supergroup template's original region contents.

(I'm guessing this is trying to say: Within the replacement template text, ie: right-hand-side, you can use the symbol @super.r() to insert the original region contents.  Also guessing that "super" is a keyword, and should not be replaced, while "r" should be replaced with the actual region name. Pretty sure this needs to be enclosed in expression delimiters, not just bare. -- GW)

There are many situations when you want to conditionally include some text or another template. StringTemplate provides simple IF-statements to let you specify conditional includes. For example, in a dynamic web page you usually want a slightly different look depending on whether or not the viewer is "logged in" or not. Without a conditional include, you would need two templates: page_logged_in and page_logged_out. You can use a single page definition with if(expr) attribute actions instead:

<html>
...
<body>
$if(member)$
$gutter/top_gutter_logged_in()$
$else$
$gutter/top_gutter_logged_out()$
$endif$
...
</body>
</html>

where template top_gutter_logged_in is located in the gutter subdirectory of my StringTemplateGroup.

IF actions test the presence or absence of an attribute unless the object is a Boolean/bool, in which case it tests the attribute for true/false. The only operator allowed is "not" and means either "not present" or "not true". For example, "$if(!member)$...$endif$".

You can also use elseif to make a chain of tests:

$if(x)$
...
$elseif(y)$
...
$elseif(z)$
...
$else$
...
$endif$

The first true expression "wins".

Whitespace in conditionals issue

There is a simple, but not perfect rule: kill a single newline after <if>, <<, <else>, and <endif> (but for <endif> only if it's on a line by itself) . Kill newlines before <else> and <endif> and >>. For example,

a <if(foo)>big<else>small<endif> dog

is identical to:

a <if(foo)>
big
<else>
small
<endif>
 dog

It is very difficult to get the newline rule to work "properly" because sometimes you want newlines and sometimes you don't. I
decided to chew up as many as is reasonable and then let you explicitly say <\n> when you need to.

Table of Contents for Expressions

Expressions

Attribute References

Named attributes

The most common thing in a template besides plain text is a simple named attribute reference such as:

Your email: $email$

The template will look up the value of email and insert it into the output stream when you ask the template to print itself out. If email has no value, then it evaluates to the empty string and nothing is printed out for that attribute expression. When working with group files, if email is not defined in the formal parameter list of an enclosing template, an exception is thrown.

If the attribute is multi-value such as an instance of a list, the elements are emitted without separator one after the other. If there are null values in the list, these are ignored by default. Given template $values$ with attribute values=9,6,null,2,null then the output would be:

962

To use a separator in between those multiple values, use the separator option:

$values; separator=", "$

The output would be:

9, 6, 2

To emit a special value for each null element in a list, use the null option:

$values; null="-1", separator=", "$

Again using values=9,6,null,2,null then the output would be:

9, 6, -1, 2, -1

Property references

If a named attribute is an aggregate with a property or a simple data field, you may reference that property using attribute.property. For example:

Your name: $person.name$
Your email: $person.email$

StringTemplate ignores the actual object type stored in attribute person and simply looks for one of the following via reflection (in search order):

An exception is thrown if that property is not defined on the target object.

Because the type is ignored, you can pass in whatever existing aggregate (class) you have such as User or Person:

User u = database.lookupPerson("parrt@jguru.com");
st.setAttribute("person", u);

User u = database.LookupPerson("parrt@jguru.com");
st.SetAttribute("person", u);

u = database.lookupPerson("parrt@jguru.com")
st["person"] = u

Or, if a suitable aggregate doesn't exist, you can make a connector or "glue" object and pass that in instead:

st.setAttribute("person", new Connector());

st.SetAttribute("person", new Connector());

st["person"] = Connector()

where Connector is defined as:

public class Connector {
    public String getName() { return "Terence"; }
    public String getEmail() { return "parrt@jguru.com"; }
}

public class Connector {
    public string Name  { get {return "Terence";} }
    public string Email { get { return "parrt@jguru.com";} }
}

class Connector(object):
    def getName(self):
        return "Terence"

    def getEmail(self):
        return "parrt@jguru.com"

The ability to reference aggregrate properties saves you the trouble of having to pull out the properties with code like this:

User u = database.lookupPerson("parrt@jguru.com");
st.setAttribute("name", u.getName());
st.setAttribute("email", u.getEmail());

User u = database.lookupPerson("parrt@jguru.com");
st.SetAttribute("name", u.Name);
st.SetAttribute("email", u.Email);

u = database.lookupPerson("parrt@jguru.com")
st["name"] = u.getName()
st["email"] = u.getEmail()

and having template:

Your name: $name$
Your email: $email$

Indirect property names

Sometimes the property name is itself variable, in which case you need to use indirect property access notation:

$person.(propertyName)$

where propertyName is an attribute whose value is the name of a property to fetch from person. Using the examples from above, propertyName could hold the value of either name or email.

propertyName may actually be an expression instead of a simple attribute name.

Map key/value pair access

StringTemplate a = new StringTemplate("$user.name$, $user.phone$");
HashMap user = new HashMap();
user.put("name", "Terence");
user.put("phone", "none-of-your-business");
a.setAttribute("user", user);
String results = a.toString();

StringTemplate a = new StringTemplate("$user.name$, $user.phone$");
Hashtable user = new Hashtable();
user.Add("name", "Terence");
user.Add("phone", "none-of-your-business");
a.SetAttribute("user", user);
string results = a.ToString();

a = stringtemplate3.StringTemplate("$user.name$, $user.phone$")
user = {}
user["name"] = "Terence"
user["phone"] = "none-of-your-business"
a["user"] = user
results = str(a)

StringTemplate interprets Map objects to have two predefined properties: keys and values that yield a list of all keys and the list of all values, respectively. When applying a template to a map, StringTemplate iterates over the values so that <aMap> is a shorthand for <aMap.values>. Similarly <aMap.keys> walks over the keys. You can list all of the elements in a map like this:

<aMap.keys:{k| <k> maps to <aMap.(k)>}>.

Note the use of the indirect property reference <aMap.(k)>, which says to take the value of the k as the key in the lookup. Clearly without the parentheses the normal map lookup mechanism would treat k as a literal and try to look up k in the map.  Also note that the map must have keys that are Strings for indirect property referencing to work, because the key is first rendered into a string by ST and then that is used to look up the value in the map.

Difficult property names

Some property names cause parse errors because of clashes with built in keywords or because they do not match the rules for IDs as used by String Template. These difficult property names can be dealt with by quoting the property name in combination with the indirect property construct:

$person.("first")$       --- Build in keyword
$person.("1")$           --- non ID conforment name

Difficult properties names are quite likely to occur when dealing with maps. Map keys can be defined using arbitrary strings, including spac