Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: clarify independence form parser.

THIS IS A WORK IN PROGRESS AND MAY BE COMPLETELY WRONG!

 Basics

The relevant part to notice is that the Lexer is independent from the Parser.  That is, it doesn't know what the parser expects and does not take any parser rules into account.  Therefore, you have to make sure the tokens the parser expects are actually generated by the Lexer.  This implies that a rule that matches almost anything will probably not be what you want, since it will always match anything, regardless of whether some other rule might match; see detailed rules below.  If you are getting a MismatchedTokenException or UnwantedTokenException telling you that 0!=0, this might well be the case.

The input stream is analyzed to identify the next token.
It does this by evaluating rules.
The winning rule injects a token (usually composed of an extract from the input stream) into the token stream.
A rule wins by passing a set of gates in order.
A conceptual description of these gates follows.
The implementation details of these gates is omitted here.
A bug is when the gate does not behave as described.
An surprise is when you don't agree with the gate structure or behavior.

...