Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 2 Next »

v3.1 required (but change state to "this" and should be ok in 3.0.1)

You need to override the default behavior which tries to recover and keep lexing. Just override the nextToken() method as shown in the following example:

grammar T;

@lexer::members {
	public Token nextToken() {
		while (true) {
			state.token = null;
			state.channel = Token.DEFAULT_CHANNEL;
			state.tokenStartCharIndex = input.index();
			state.tokenStartCharPositionInLine = input.getCharPositionInLine();
			state.tokenStartLine = input.getLine();
			state.text = null;
			if ( input.LA(1)==CharStream.EOF ) {
				return Token.EOF_TOKEN;
			}
			try {
				mTokens();
				if ( state.token==null ) {
					emit();
				}
				else if ( state.token==Token.SKIP_TOKEN ) {
					continue;
				}
				return state.token;
			}
			catch (RecognitionException re) {
				reportError(re);
				throw new RuntimeException("Bailing out!"); // or throw Error
			}
		}
	}
}

a : 'hi' INT {System.out.println("not printed upon lexical error");} ;

INT : '0'..'9'+ ;
WS : (' '|'\n')+ {skip();} ;

Running through ANTLRWorks with input "hi 34" works fine, but input "hi mom" results in the following:

line 1:3 no viable alternative at character 'm'
Exception in thread "main" java.lang.RuntimeException: Bailing out!
	at TLexer.nextToken(TLexer.java:37)
	at org.antlr.runtime.CommonTokenStream.fillBuffer(CommonTokenStream.java:119)
	at org.antlr.runtime.CommonTokenStream.LT(CommonTokenStream.java:238)
	at org.antlr.runtime.CommonTokenStream.LA(CommonTokenStream.java:300)
	at org.antlr.runtime.BaseRecognizer.match(BaseRecognizer.java:71)
	at TParser.a(TParser.java:77)
	at __Test__.main(__Test__.java:14)

You cannot throw an exception because the interface for TokenStream was designed to not transmit lexical errors through to the parser. What would the parser do with a lexical error? Better to throw an exception that makes the entire system go back to the invoking main method or whatever. Java Does not allow throwing exceptions without altering the method interface so we must use either an Error object or a RuntimeObject, which are considered unchecked.

Other targets will not have this problem. just override and get rid of the catch-clause.

  • No labels