how to use the Lua5.1 lemon grammar?

Question

I found the Lua 5.1 grammar for Lemon here (Listing 1 at end of page):

%fallback  OPEN '(' .

chunk      ::= block .

semi       ::= ';' .
semi       ::= .

block      ::= scope statlist .
block      ::= scope statlist laststat semi .
ublock     ::= block 'until' exp .

scope      ::= .
scope      ::= scope statlist binding semi.

statlist   ::= .
statlist   ::= statlist stat semi .

stat       ::= 'do' block 'end' .
stat       ::= 'while' exp 'do' block 'end' .
stat       ::= repetition 'do' block 'end' .
stat       ::= 'repeat' ublock .
stat       ::= 'if' conds 'end' .
stat       ::= 'function' funcname funcbody .
stat       ::= setlist '=' explist1 .
stat       ::= functioncall .

repetition ::= 'for' NAME '=' explist23 .
repetition ::= 'for' namelist 'in' explist1 .

conds      ::= condlist .
conds      ::= condlist 'else' block .
condlist   ::= cond .
condlist   ::= condlist 'elseif' cond .
cond       ::= exp 'then' block .

laststat   ::= 'break' .
laststat   ::= 'return' .
laststat   ::= 'return' explist1 .

binding    ::= 'local' namelist .
binding    ::= 'local' namelist '=' explist1 .
binding    ::= 'local' 'function' NAME funcbody .

funcname   ::= dottedname .
funcname   ::= dottedname ':' NAME .

dottedname ::= NAME .
dottedname ::= dottedname '.' NAME .

namelist   ::= NAME .
namelist   ::= namelist ',' NAME .

explist1   ::= exp .
explist1   ::= explist1 ',' exp .
explist23  ::= exp ',' exp .
explist23  ::= exp ',' exp ',' exp .

%left      'or' .
%left      'and' .
%left      '<' '<=' '>' '>=' '==' '~=' .
%right     '..' .
%left      '+' '-' .
%left      '*' '/' '%' .
%right     'not' '#' .
%right     '^' .

exp        ::= 'nil'|'true'|'false'|NUMBER|STRING|'...' .
exp        ::= function .
exp        ::= prefixexp .
exp        ::= tableconstructor .
exp        ::= 'not'|'#'|'-' exp .         ['not']
exp        ::= exp 'or' exp .
exp        ::= exp 'and' exp .
exp        ::= exp '<'|'<='|'>'|'>='|'=='|'~=' exp .
exp        ::= exp '..' exp .
exp        ::= exp '+'|'-' exp .
exp        ::= exp '*'|'/'|'%' exp .
exp        ::= exp '^' exp .

setlist    ::= var .
setlist    ::= setlist ',' var .

var        ::= NAME .
var        ::= prefixexp '[' exp ']' .
var        ::= prefixexp '.' NAME .

prefixexp  ::= var .
prefixexp  ::= functioncall .
prefixexp  ::= OPEN exp ')' .

functioncall ::= prefixexp args .
functioncall ::= prefixexp ':' NAME args .

args        ::= '(' ')' .
args        ::= '(' explist1 ')' .
args        ::= tableconstructor .
args        ::= STRING .

function    ::= 'function' funcbody .

funcbody    ::= params block 'end' .

params      ::= '(' parlist ')' .

parlist     ::= .
parlist     ::= namelist .
parlist     ::= '...' .
parlist     ::= namelist ',' '...' .

tableconstructor ::= '{' '}' .
tableconstructor ::= '{' fieldlist '}' .
tableconstructor ::= '{' fieldlist ','|';' '}' .

fieldlist   ::= field .
fieldlist   ::= fieldlist ','|';' field .

field       ::= exp .
field       ::= NAME '=' exp .
field       ::= '[' exp ']' '=' exp .

I tried to compile it with lemon -c lua51.y but got lots of errors:

lua51.y:3: %fallback argument "'" should be a token
lua51.y:3: %fallback argument "(" should be a token
lua51.y:3: %fallback argument "'" should be a token
lua51.y:7: Illegal character on RHS of rule: "'".
lua51.y:12: Illegal character on RHS of rule: "'".
lua51.y:20: Illegal character on RHS of rule: "'".
lua51.y:21: Illegal character on RHS of rule: "'".
lua51.y:22: Illegal character on RHS of rule: "'".
lua51.y:23: Illegal character on RHS of rule: "'".
lua51.y:24: Illegal character on RHS of rule: "'".
lua51.y:25: Illegal character on RHS of rule: "'".
lua51.y:26: Illegal character on RHS of rule: "'".
lua51.y:29: Illegal character on RHS of rule: "'".
lua51.y:30: Illegal character on RHS of rule: "'".
lua51.y:33: Illegal character on RHS of rule: "'".
lua51.y:35: Illegal character on RHS of rule: "'".
lua51.y:36: Illegal character on RHS of rule: "'".
lua51.y:38: Illegal character on RHS of rule: "'".
lua51.y:39: Illegal character on RHS of rule: "'".
lua51.y:40: Illegal character on RHS of rule: "'".
lua51.y:42: Illegal character on RHS of rule: "'".
lua51.y:43: Illegal character on RHS of rule: "'".
lua51.y:44: Illegal character on RHS of rule: "'".
lua51.y:47: Illegal character on RHS of rule: "'".
lua51.y:50: Illegal character on RHS of rule: "'".
lua51.y:50: Token "'" should be either "%" or a nonterminal name.
lua51.y:50: Token "NAME" should be either "%" or a nonterminal name.
lua51.y:50: Token "." should be either "%" or a nonterminal name.
lua51.y:53: Illegal character on RHS of rule: "'".
lua51.y:56: Illegal character on RHS of rule: "'".
lua51.y:57: Illegal character on RHS of rule: "'".
lua51.y:58: Illegal character on RHS of rule: "'".
lua51.y:60: Can't assign a precedence to "'".
lua51.y:60: Can't assign a precedence to "or".
lua51.y:60: Can't assign a precedence to "'".
lua51.y:61: Can't assign a precedence to "'".
lua51.y:61: Can't assign a precedence to "and".
lua51.y:61: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "<".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "<".
lua51.y:62: Can't assign a precedence to "=".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to ">".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to ">".
lua51.y:62: Can't assign a precedence to "=".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "=".
lua51.y:62: Can't assign a precedence to "=".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:62: Can't assign a precedence to "~".
lua51.y:62: Can't assign a precedence to "=".
lua51.y:62: Can't assign a precedence to "'".
lua51.y:63: Can't assign a precedence to "'".
lua51.y:63: Token "." should be either "%" or a nonterminal name.
lua51.y:63: Token "'" should be either "%" or a nonterminal name.
lua51.y:63: Token "." should be either "%" or a nonterminal name.
lua51.y:64: Can't assign a precedence to "'".
lua51.y:64: Can't assign a precedence to "+".
lua51.y:64: Can't assign a precedence to "'".
lua51.y:64: Can't assign a precedence to "'".
lua51.y:64: Can't assign a precedence to "-".
lua51.y:64: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "*".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "/".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:65: Can't assign a precedence to "%".
lua51.y:65: Can't assign a precedence to "'".
lua51.y:66: Can't assign a precedence to "'".
lua51.y:66: Can't assign a precedence to "not".
lua51.y:66: Can't assign a precedence to "'".
lua51.y:66: Can't assign a precedence to "'".
lua51.y:66: Can't assign a precedence to "#".
lua51.y:66: Can't assign a precedence to "'".
lua51.y:67: Can't assign a precedence to "'".
lua51.y:67: Can't assign a precedence to "^".
lua51.y:67: Can't assign a precedence to "'".
lua51.y:69: Illegal character on RHS of rule: "'".
lua51.y:69: Token "." should be either "%" or a nonterminal name.
lua51.y:69: Token "." should be either "%" or a nonterminal name.
lua51.y:69: Token "'" should be either "%" or a nonterminal name.
lua51.y:69: Token "." should be either "%" or a nonterminal name.
lua51.y:73: Illegal character on RHS of rule: "'".
lua51.y:73: The precedence symbol must be a terminal.
lua51.y:73: Missing "]" on precedence mark.
lua51.y:73: Token "'" should be either "%" or a nonterminal name.
lua51.y:73: Token "]" should be either "%" or a nonterminal name.
lua51.y:74: Illegal character on RHS of rule: "'".
lua51.y:75: Illegal character on RHS of rule: "'".
lua51.y:76: Illegal character on RHS of rule: "'".
lua51.y:77: Illegal character on RHS of rule: "'".
lua51.y:77: Token "." should be either "%" or a nonterminal name.
lua51.y:77: Token "'" should be either "%" or a nonterminal name.
lua51.y:77: Expected to see a ":" following the LHS symbol "exp".
lua51.y:79: Illegal character on RHS of rule: "'".
lua51.y:79: Illegal declaration keyword: "'".
lua51.y:80: Illegal character on RHS of rule: "'".
lua51.y:83: Illegal character on RHS of rule: "'".
lua51.y:86: Illegal character on RHS of rule: "'".
lua51.y:87: Illegal character on RHS of rule: "'".
lua51.y:87: Token "'" should be either "%" or a nonterminal name.
lua51.y:87: Token "NAME" should be either "%" or a nonterminal name.
lua51.y:87: Token "." should be either "%" or a nonterminal name.
lua51.y:91: Illegal character on RHS of rule: "'".
lua51.y:94: Illegal character on RHS of rule: "'".
lua51.y:96: Illegal character on RHS of rule: "'".
lua51.y:97: Illegal character on RHS of rule: "'".
lua51.y:101: Illegal character on RHS of rule: "'".
lua51.y:103: Illegal character on RHS of rule: "'".
lua51.y:105: Illegal character on RHS of rule: "'".
lua51.y:109: Illegal character on RHS of rule: "'".
lua51.y:109: Token "." should be either "%" or a nonterminal name.
lua51.y:109: Token "." should be either "%" or a nonterminal name.
lua51.y:109: Token "'" should be either "%" or a nonterminal name.
lua51.y:109: Token "." should be either "%" or a nonterminal name.
lua51.y:110: Illegal character on RHS of rule: "'".
lua51.y:110: Token "." should be either "%" or a nonterminal name.
lua51.y:110: Token "." should be either "%" or a nonterminal name.
lua51.y:110: Token "'" should be either "%" or a nonterminal name.
lua51.y:110: Token "." should be either "%" or a nonterminal name.
lua51.y:112: Illegal character on RHS of rule: "'".
lua51.y:113: Illegal character on RHS of rule: "'".
lua51.y:114: Illegal character on RHS of rule: "'".
lua51.y:117: Illegal character on RHS of rule: "'".
lua51.y:120: Illegal character on RHS of rule: "'".
lua51.y:121: Illegal character on RHS of rule: "'".

What's missing?

Answer 1

That looks like my work, from some years ago. It was intended for a modified version of lemon which I created but no longer maintain, which generated a parser in Lua.

In order to use it with the out-of-the-box lemon parser, you will have to replace all instances of quoted literals with named tokens. See the section on Terminals and Nonterminals in the lemon documentation. (There's a note about this in the page you linked.)

Unlike yacc/bison, lemon does not require you to declare token names, because such names must start with a capital letter. Like yacc/bison, lemon will generate a header file with definitions for those names, which you can #include in your scanner implementation. (It is easy to use flex to generate a scanner for lemon parsers, but you should read the lemon documentation carefully; lemon parsers are called by the scanner rather than calling the scanner.)

---

There are a couple of lemon features which are not present in yacc/bison which would complicate converting this grammar for yacc/bison:

• The | notation in lemon is undocumented, and different from the use of | in yacc/bison; it can only be used to create a list of alternative terminal tokens. In yacc/bison you could create a non-terminal of this form, but non-terminals do not participate in precedence rules; in lemon, the list of alternatives is a "Multiterminal" which doesn't interfere with precedence rules. So in yacc/bison you would have to duplicate productions using token alternatives for each alternative terminal. Another yacc/bison approach would be to use a single lexical token for all the alternatives in a group, distinguishing between them by their semantic value, which could be an enum. That will work fine for the operator terminals, but will not work for alternatives like ','|';' because those are not consistent throughout the grammar.

• There is no yacc/bison equivalent for %fallback, nor is there a simple work-around.