please consider this piece of a lexer I am constructing:
let newline = "\n\r" | '\n' | '\r'
let dot = "\."
let lineTerminator = "\."newline
rule tokenize = parse
...
| lineTerminator { lexbuf.EndPos <- lexbuf.EndPos.NextLine; tokenize lexbuf; }
| newline { lexbuf.EndPos <- lexbuf.EndPos.NextLine; tokenize lexbuf }
| eof { EOF }
| _ { failwithf "unrecognized input: '%s'" <| lexeme lexbuf }
On a line terminating with a dot, the current buffer will be at the character where the dot is and say that it cannot recognize the input.
I really don't know what I'm doing wrong.