From the Regexp::Grammars documentation:
The difference between a token and a rule is that a token treats any whitespace within it exactly as a normal Perl regular expression would. That is, a sequence of whitespace in a token is ignored if the
/xmodifier is in effect, or else matches the same literal sequence of whitespace characters (if/xis not in effect).In a rule, most sequences of whitespace are treated as matching the implicit subrule
<.ws>, which is automatically predefined to match optional whitespace (i.e.\s*)....
In other words, a rule such as:
<rule: sentence> <noun> <verb> | <verb> <noun>is equivalent to a token with added non-capturing whitespace matching:
<token: sentence> <.ws> <noun> <.ws> <verb> | <.ws> <verb> <.ws> <noun>
Is there a way to get the rule to ignore the leading implicit <.ws>? In the example above, it would be equivalent to:
<token: sentence> <noun> <.ws> <verb>
| <verb> <.ws> <noun>