StringTemplate 2.2 Documentation
StringTemplate Documentation
Terence Parr
University of San Francisco
parrt AT cs.usfca.edu
Copyright 2003-2005
http://www.stringtemplate.org
C# version created by
Kunle Odutola
kunle UNDERSCORE odutola AT hotmail.com
Copyright 2005-2006
(ST# - C# StringTemplate released under BSD License)
Version 2.3b7 May 11, 2006
Python version created by
Marq Kole
marq DOT kole AT xs4all DOT nl
Copyright 2003-2006
Version 2.2, June 15, 2006
Contents
Related material
Please see the changes and bugs page. That page generally discusses the Java version of StringTemplate but, some of the information it contains might apply to other implementations.
The StringTemplates distribution includes many unit tests that also represent a useful set of examples. The tests are defined in:
Java |
|
|---|---|
C# |
|
Python |
|
It is highly recommended that you read the (academically-oriented) paper, Enforcing Model-View Separation in Template Engines. There are some more examples including nested menu generation that will be of interest. You may also find the following resources useful or interesting:
Introduction
Most programs that emit source code or other text output are unstructured blobs of generation logic interspersed with print statements. The primary reason is the lack of suitable tools and formalisms. The proper formalism is that of an output grammar because you are not generating random characters--you are generating sentences in an output language. This is analogous to using a grammar to describe the structure of input sentences. Rather than building a parser by hand, most programmers will use a parser generator. Similarly, we need some form of unparser generator to generate text. The most convenient manifestation of the output grammar is a template engine such as StringTemplate.
A template engine is a simply a code generator that emits text using templates, which are really just "documents with holes" in them where you can stick values. StringTemplate breaks up your template into chunks of text and attribute expressions, which are by default enclosed in dollar signs $attribute-expression$ (to make them easy to see in HTML files). StringTemplate ignores everything outside of attribute expressions, treating it as just text to spit out when you call:
Java |
|
|---|---|
C# |
|
Python |
|
For example, the following template has two chunks, a literal and a reference to attribute name:
Hello, $name$
Using templates in code is very easy. Here is the requisite example that prints "Hello, World":
Java |
import org.antlr.stringtemplate.*;
StringTemplate hello = new StringTemplate("Hello, $name$");
hello.SetAttribute("name", "World");
System.out.println(hello.toString());
|
|---|---|
C# |
using Antlr.StringTemplate;
StringTemplate hello = new StringTemplate("Hello, $name$");
hello.SetAttribute("name", "World");
Console.Out.WriteLine(hello.ToString());
|
Python |
import stringtemplate
hello = stringtemplate.StringTemplate("Hello, $name$")
hello["name"] = "World"
print str(hello)
|
StringTemplate is not a "system" or "engine" or "server"; it is a library with two primary classes of interest: StringTemplate and StringTemplateGroup. You can directly create a StringTemplate in code, you can load a template from a file, and you can load a single file with many templates (a template group file).
Motivation And Philosophy
StringTemplate was born and evolved during the development of http://www.jGuru.com. The need for such dynamically-generated web pages has led to the development of numerous other template engines in an attempt to make web application development easier, improve flexibility, reduce maintenance costs, and allow parallel code and HTML development. These enticing benefits, which have driven the proliferation of template engines, derive entirely from a single principle: separating the specification of a page's business logic and data computations from the specification of how a page displays such information.
These template engines are in a sense are a reaction to the completely entangled specifications encouraged by JSP (Java Server Pages), ASP (Active Server Pages) and, even ASP.NET. With separate encapsulated specifications, template engines promote component reuse, pluggable site "looks", single-points-of-change for common components, and high overall system clarity. In the code generation realm, model-view separation guarantees retargetability.
When developing StringTemplate, I recalled Frederick Brook's book, "Mythical Man Month", where he identified conceptual integrity as a crucial product ingredient. For example, in UNIX everything is a stream. My concept, if you will, is strict model-view separation. My mission statement is therefore:
"StringTemplate shall be as simple, consistent, and powerful as possible without sacrificing strict model-view separation."
I ruthlessly evaluate all potential features and functionality against this standard. Over the years, however, I have made certain concessions to practicality that one could consider as infringing ever-so-slightly into potential model-view entanglement. That said, StringTemplate still seems to enforce separation while providing excellent functionality.
I let my needs dictate the language and tool feature set. The tool evolved as my needs evolved. I have done almost no feature "backtracking". Further, I have worked really hard to make this little language self-consistent and consistent with existing syntax/metaphors from other languages. There are very few special cases and attribute/template scoping rules make a lot of sense even if they are unfamiliar or strange at first glance. Everything in the language exists to solve a very real need.
After examining hundreds of template files that I created over years of jGuru.com (and now in ANTLR v3) development, I found that I needed only the following four basic canonical operations (with some variations):
attribute reference; e.g.,
$phoneNumber$template reference (like #include or macro expansion); e.g.,
$searchbox()$conditional include of subtemplate (an IF statement); e.g.,
$if(title)$<title>$title$</title>$endif$template application to list of attributes; e.g.,
$names:bold()$where template references can be recursive.
Language theory supports my premise that even a minimal StringTemplate engine with only these features is very powerful--such an engine can generate the context-free languages (see Enforcing Strict Model-View Separation in Template Engines); e.g., most programming languages are context-free as are any XML pages whose form can be expressed with a DTD.
The normal imperative programming language features like setting variables, loops, arithmetic expressions, arbitrary method calls into the model, etc... are not only unnecessary, but they are very specifically what is wrong with ASP/JSP. Recall that ASP/JSP (and ASP.NET) allow arbitrary code expressions and statements, allowing programmers to incorporate computations and logic in their templates. A quick scan of template engines reveals an unfortunate truth--all but a few are Turing-complete languages just like ASP/JSP/ASP.NET. One can argue that they are worse than ASP/JSP/ASP.NET because they use languages peculiar to that template engine. Many tool builders have clearly lost sight of the original problem we were all trying to solve. We programmers often get caught up in cool implementations, but we should focus on what should be built not what can be built.
The fact that StringTemplate does not allow such things as assignments (no side-effects) should make you suspicious of engines that do allow it. The templates in ANTLR v3's code generator are vastly more complicated than the sort of templates typically used in web pages creation with other template engines yet, there hasn't been a situation where assignments were needed. If your template looks like a program, it probably is--you have totally entangled your model and view.
While providing all sorts of dangerous features like assignment that promote the use of computations and logic in templates, many engines miss the key elements. Certain language semantics are absolutely required for generative programming and language translation. One is recursion. A template engine without recursion seems unlikely to be capable of generating recursive output structures such as nested tables or nested code blocks.
Another distinctive StringTemplate language feature lacking in other engines is lazy-evaluation. StringTemplate's attributes are lazily evaluated in the sense that referencing attribute "a" does not actually invoke the data lookup mechanism until the template is asked to render itself to text. Lazy evaluation is surprising useful in both the web and code generation worlds because such order decoupling allows code to set attributes when it is convenient or efficient not necessarily before a template that references those attributes is created. For example, a complicated web page may consist of many nested templates many of which reference $userName$, but the value of userName does not need to be set by the model until right before the entire page is rendered to text via ToString(). You can build up the complicated page, setting attribute values in any convenient order.
StringTemplate implements a "poor man's" form of lazy evaluation by simply requiring that all attributes be computed a priori. That is, all attributes must be computed and pushed into a template before it is written to text; this is the so-called "push method" whereas most template engines use the "pull method". The pull method appears more conventional because programmers mistakenly regard templates as programs, but pulling attributes introduces order-of-computation dependencies. Imagine a simple web page that displays a list of names (using some mythical Java-based template engine notation):
<html>
<body>
<ol>
$foreach n in names$
<li>$n$</li>
$end$
</ol>
There are $numberNames$ names.
</body>
</html>
Using the pull method, the reference to names invokes model.getNames(), which presumably loads a list of names from the database. The reference to numberNames invokes model.getNumberNames() which necessarily uses the internal data structure computed by getNames() to compute names.size() or whatever. Now, suppose a designer moves the numberNames reference to the <title> tag, which is before the reference to names in the foreach statement. The names will not yet have been loaded, yielding a null pointer exception at worst or a blank title at best. You have to anticipate these dependencies and have getNumberNames() invoke getNames() because of a change in the template.
I'm stunned that other template engine authors with whom I've spoken think this is ok. Any time I can get the computer to do something automatically for me that removes an entire class of programming errors, I'll take it!. Automatic garbage collection is the obvious analogy here.
The pull method requires that programmers do a topological sort in their minds anticipating any order that a programmer or designer could induce. To ensure attribute computation safety (i.e., avoid hidden dependency landmines), I have shown trivially in my academic paper that pull reduces to push in the worst case. With a complicated mesh of templates, you will miss a dependency, thus, creating a really nasty, difficult-to-find bug.
Just so you know, I've never been a big fan of functional languages and I laughed really hard when I realized (while writing the academic paper) that I had implemented a functional language. The nature of the problem simply dictated a particular solution. We are generating sentences in an output language so we should use something akin to a grammar. Output grammars are inconvenient so tool builders created template engines. Restricted template engines that enforce the universally-agreed-upon goal of strict model-view separation also look remarkably like output grammars as I have shown. So, the very nature of the language generation problem dictates the solution: a template engine that is restricted to support a mutually-recursive set of templates with side-effect-free and order-independent attribute references.
Defining Templates
Creating Templates With Code
Here is a simple example that creates and uses a template on the fly:
Java |
StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("table", "User");
|
|---|---|
C# |
StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("table", "User");
|
Python |
import stringtemplate
query = stringtemplate.StringTemplate("SELECT $column$ FROM $table$;")
query["column"] = "name"
query["table"] = "User" |
where StringTemplate considers anything in $...$ to be something it needs to pay attention to. By setting attributes, you are "pushing" values into the template for use when the template is printed out. The attribute values are set by referencing their names. Invoking toString() on query would yield
SELECT name FROM User;
You can set an attribute multiple times, which simply means that the attribute is multi-valued. For example, adding another value to the attribute named column as shown below makes the attribute multi-valued:
Java |
StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User");
|
|---|---|
C# |
StringTemplate query = new StringTemplate("SELECT $column$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");
|
Python |
query = stringtempatle.StringTemplate("SELECT $column$ FROM $table$;")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User"
|
Invoking toString() on query would now yield
SELECT nameemail FROM User;
Ooops...there is no separator between the multiple values. If you want a comma, say, between the column names, then change the template to record that formatting information:
Java |
StringTemplate query = new StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User");
|
|---|---|
C# |
StringTemplate query = new StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");
|
Python |
query = stringtemplate.StringTemplate("SELECT $column; separator=\",\"$ FROM $table$;")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User
|
Note that the right-hand-side of the separator specification in this case is a string literal; therefore, we have escaped the double-quotes as the template is specified in a string. In general, the right-hand-side can be any attribute expression. Invoking toString() on query would now yield
SELECT name,email FROM User;
Attributes can be any object at all. StringTemplate calls toString() on each object as it writes the template out. The separator is not used unless the attribute is multi-valued.
Loading Templates From Files
To load a template from the disk you must use a StringTemplateGroup that will manage all the templates you load, caching them so you do not waste time talking to the disk for each template fetch request (you can change it to not cache; see below). You may have multiple template groups. Here is a simple example that loads the previous SQL template from a file /tmp/theQuery.st:
SELECT $column; separator=","$ FROM $table$;
The code below creates a StringTemplateGroup called myGroup rooted at /tmp so that requests for template theQuery forces a load of file /tmp/theQuery.st.
Java |
StringTemplateGroup group = new StringTemplateGroup("myGroup", "/tmp");
StringTemplate query = group.getInstanceOf("theQuery");
query.setAttribute("column", "name");
query.setAttribute("column", "email");
query.setAttribute("table", "User"); |
|---|---|
C# |
StringTemplateGroup group = new StringTemplateGroup("myGroup", "/tmp");
StringTemplate query = group.GetInstanceOf("theQuery");
query.SetAttribute("column", "name");
query.SetAttribute("column", "email");
query.SetAttribute("table", "User");
|
Python |
group = stringtemplate.StringTemplateGroup("myGroup", "/tmp")
query = group.getInstanceOf("theQuery")
query["column"] = "name"
query["column"] = "email"
query["table"] = "User" |
If you have a directory hierarchy of templates such as file /tmp/jguru/bullet.st, you would reference them relative to the root; in this case, you would ask for template jguru/bullet().
Note
StringTemplate strips whitespace from the front and back of all loaded template files. You can add, for example, <\n> at the end of the file to get an extra carriage return.
Loading Templates relative to an implementation specific location
Java | |
|---|---|
C# | |
Python |
If page.st references, say, searchbox template, it must be fully qualified as:
<font size=2>SEARCH</font>: $com/mycompany/server/templates/page/searchbox()$
This is inconvenient and ST may add the invoking template's path prefix automatically in the future.
StringTemplate Group Files
StringTemplate 2.0 introduced the notion of a group file that has two main attractions. First, it allows you to define lots of small templates more conveniently because they may all be defined within a single file. Second, you may specify formal template arguments that help StringTemplate detect errors (such as setting unknown attributes) and make the templates easier to read. Here is a sample group file (I'm using <...> delimiters) with two templates, vardef and method, that could be used to generate C files:
group simple;
vardef(type,name) ::= "<type> <name>;"
method(type,name,args) ::= <<
<type> <name>(<args; separator=",">) {
<statements; separator="\n">
}
\>>
Single line templates are enclosed in double quotes while multi-line templates are enclosed in double angle-brackets. Every template must define arguments even if the formal argument list if blank.
Using templates in a group file is straightforward. The StringTemplateGroup class has a number of constructors, one of which allows you to pass in a string or file or whatever:
Java |
String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a Reader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.getInstanceOf("vardef");
t.setAttribute("type", "int");
t.setAttribute("name", "foo");
System.out.println(t); |
|---|---|
C# |
String templates = "group simple; vardef(type,name) ..."; // templates from above
// Use the constructor that accepts a System.IO.TextReader
StringTemplateGroup group = new StringTemplateGroup(new StringReader(templates));
StringTemplate t = group.GetInstanceOf("vardef");
t.SetAttribute("type", "int");
t.SetAttribute("name", "foo");
Console.Out.WriteLine(t);
|
Python |
templates = "group simple; vardef(type,name) ..."; # templates from above
# Use the constructor that accepts a Reader
group = stringtemplate.StringTemplateGroup(StringIO(templates))
t = group.getInstanceOf("vardef")
t["type"] = "int"
t["name"] = "foo"
print str(t)
|
The output would be: "int foo;".
Formal argument default values
Sometimes it is convenient to have default values for formal arguments that are used when no value is set by the model. For example, when generating a parser in Java from ANTLR, I want the super class of the generated object to be Parser unless the ANTLR user uses an option to set the super class to some custom class. For example, here is a partial parser template definition:
parser(name, rules, superClass="Parser") ::= ...
Any argument may be given a default value by following the name with an equals sign and a string or an anonymous template.
Formal argument error handling
When using a group file format to specify templates, you must specify the formal arguments for that template. If you try to set an attribute via setAttribute that is not specifically formally defined in that template, you will get the following exception:
Java |
|
|---|---|
C# |
|
Python |
|
If you reference an attribute that is not formally defined in that template or any enclosing template, you also get the same exception.
Maps
There are situations where you need to translate a string in one language to a string in another language. For example, you might want to translate integer to int when translating Pascal to C. You could pass a Map or IDictionary (e.g. hashtable) from the model into the templates, but then you have output literals in your model! The only solution is to have StringTemplate support mappings. For example, here is how ANTLR v3 knows how to initialize local variables to their default values:
typeInitMap ::= [
"int":"0",
"long":"0",
"float":"0.0",
"double":"0.0",
"boolean":"false",
"byte":"0",
"short":"0",
"char":"0",
default:"null" // anything other than an atomic type
]
To use the map in a template, refer to it as you would an attribute For example, <typeInitMap.int> which returns "0". If your type name is an attribute not a constant like int, then use an indirect field access: <typeInitMap.(typeName)>.
Maps are defined in the group's scope and are visible if no attribute hides them. For example, if you define a formal argument called typeInitMap in template foo then foo cannot see the map defined in the group (though you could pass it in as another parameter). If a name is not an attribute and it's not in the group's maps table, then the super group is consulted etc... You may not redefine a map and it may not have the same name as a template in that group. The default value is used if you use a key as a property that doesn't exist. For example <typeInitMap.foo> returns "null".
You'll note that the square brackets will denote data structure in other areas too such as [a,b,c,...] which makes a singe multi-valued attribute out of other attributes so you can iterate across them.
Newline handling
The first newline following the << in a template definition is ignored as it is usually used just to get the first line of text for the template at the start of a line. In other words, if you want to have a blank line at the start of your template, use:
foo() ::= <<
2nd line is not blank, but first is
\>>
or
foo() ::= <<<\n>
same as before; newline then this line
\>>
The last newline before the >> is also ignored and is included in the output. To add a final newline, add an extra line or <\n> before the >>:
foo() ::= <<
rodent
\>>
or
foo() ::= <<
rodent<\n>
\>>
The following template:
foo() ::= <<
rodent
\>>
on the other hand, is identical to
foo() ::= "rodent"
Group file format
group
: "group" ID ';' (template|mapdef)*
;
template
: ID '(" (args)? ')' "::=" TEMPLATE
| ID "::=" ID // alias one template to be another
;
args: arg (',' arg)*
;
arg : ID '=' STRING // "..."
| ID '=' ANONYMOUS_TEMPLATE // {...}
;
mapdef
: ID "::=" map
;
map : '[' keyValuePair (',' keyValuePair)* ']'
;
keyValuePair
: STRING ':' STRING
| "default" ':' STRING
;
An aside: All along, during my website construction days, I kept in mind that any text output follows a format and, thus, output sentences conform to a language. Consequently, a grammar should describe the output rather than a bunch of ad hoc print statements in code. This helped me formalize the study of templates because I could compare templates (output grammars) to well established ideas from formal language theory and context-free grammars. This allowed me to show, among other things, that
StringTemplatecan easily generate any document describable with an XML DTD even though it is deliberately limited. The group file format should look very much like a grammar to you.
Scoping rules and attribute look-up
See the scoping rules section for information on how formal arguments affect attribute look up.
Group files have a .stg file extension.
Functionality
Attribute References
Named attributes
The most common thing in a template besides plain text is a simple named attribute reference such as:
Your email: $email$
The template will look up the value of email and insert it into the output stream when you ask the template to print itself out. If email has no value, then it evaluates to the empty string and nothing is printed out for that attribute expression.
Property references
If a named attribute is an aggregate with a property or a simple data field, you may reference that property using attribute.property. For example:
Your name: $person.name$
Your email: $person.email$
StringTemplate ignores the actual object type stored in attribute person and simply looks for one of the following via reflection (in search order):
Java |
If found, a return value is obtained via reflection. The If the property is not accessible ala JavaBeans, |
|---|---|
C# |
If found, a return value is obtained via reflection. The As shown above, if the property is not accessible as a C# property, |
Python |