How to get a list of all valid options for the next token?
To get the list of FOLLOW sets for each rule, all you need is to access the FOLLOW stack with:
BitSet fset = state.following[state._fsp];
ANTLR pushes the current set of tokens that can follow a particular invocation of the rule before it enters the rule. The Rule catches NoViableAltExceptions and so you can simply access the top of stack to get the set of tokens that can follow this rule. Manually set an exception handler for each rule to fill in this code:
catch (RecognitionException re) { reportError(re); recover(input,re); }
or use @rulecatch to set it for every rule.
ANTLR does single token insertion and deletion. It passes the set of tokens that can follow a token reference to match():
public Object match(IntStream input, int ttype, BitSet follow) {...}
ANTLR calls recoverFromMismatchedToken() if the current token does not match. In that routine, it checks the following tokens. the comment is illuminating:
/** Attempt to recover from a single missing or extra token. * * EXTRA TOKEN * * LA(1) is not what we are looking for. If LA(2) has the right token, * however, then assume LA(1) is some extra spurious token. Delete it * and LA(2) as if we were doing a normal match(), which advances the * input. * * MISSING TOKEN * * If current token is consistent with what could come after * ttype then it is ok to "insert" the missing token, else throw * exception For example, Input "i=(3;" is clearly missing the * ')'. When the parser returns from the nested call to expr, it * will have call chain: * * stat -> expr -> atom * * and it will be trying to match the ')' at this point in the * derivation: * * => ID '=' '(' INT ')' ('+' atom)* ';' * ^ * match() will see that ';' doesn't match ')' and report a * mismatched token error. To recover, it sees that LA(1)==';' * is in the set of tokens that can follow the ')' token * reference in rule atom. It can assume that you forgot the ')'. */