Token does not recognize '&' and single quote

Question

I was trying to match everything between parentheses. I met an error that it won't recognize '&' and '''. However, in IntelliJ antlr preview, it works fine. Please help me with the error. Thank you so much.

line 1:0 token recognition error at: '&'

line 3:23 token recognition error at: '&'

line 3:30 token recognition error at: '''

//test cases
&
""
{"pattern": "\\:(\\s+)[&\\\"\\']"}

grammar g;

start   : root+ EOF ;

root    : expr
        | LPARAM
        | RPARAM
        ;

expr    : list
        | atom
        ;

list    : LPARAM expr+ RPARAM
        ;

atom    : INT |' ';
//anybutp:

INT     : [\u0000-\u0019]*|[\u0021-\u0027]* | [\u002a-\uffff]*;
LPARAM  : '(';
RPARAM  : ')';
WS: [ \t\n\r]+ -> skip;

When in put is

&

''

line 1:0 token recognition error at: '&'

line 2:0 token recognition error at: '''

line 2:1 token recognition error at: '''

line 2:2 mismatched input '' expecting {'+', '-', '*', '', '/', '[', ']', '.', '^', '$', '{', '}', '|', '#', '@', '<', '?', '=', ',', '"', ':', '!', '>', ';', '`', '~', '_', '%', '()', INT, '(', ')'}

import org.antlr.v4.runtime.CharStream;

import org.antlr.v4.runtime.CommonTokenStream;
import org.antlr.v4.runtime.tree.ParseTree;
import static org.antlr.v4.runtime.CharStreams.fromFileName;
import java.io.IOException;
import org.antlr.v4.runtime.Parser;
import org.antlr.v4.runtime.ParserRuleContext;
import org.antlr.v4.runtime.tree.ParseTree;
import org.antlr.v4.runtime.tree.ParseTreeWalker;
import org.antlr.v4.runtime.tree.Trees;



public class AntlerParser {
    public static void main (String[] args) throws IOException{
        String src = "C:\\Users\\vicky\\IdeaProjects\\regex\\src\\text.txt";

        CharStream charStream = fromFileName(src);
        gLexer lexer = new gLexer(charStream);
        CommonTokenStream commonTokenStream= new CommonTokenStream(lexer);
        gParser parser = new gParser(commonTokenStream);
        ParseTree tree = parser.start();

        MyVisitor visitor = new MyVisitor();
        visitor.visit(tree);
/*print out lists*/
        MyListener listener = new MyListener();
        ParseTreeWalker walker = new ParseTreeWalker();
        walker.walk(listener,tree);

        int count = tree.getChildCount();
        System.out.println("done"+count);
    }
}

Answer 1

Tested with ANTLR 4.9.

The code:

String source =
        "&\n" +
        "\"\"\n" +
        "{\"pattern\": \"\\\\:(\\\\s+)[&\\\\\\\"\\\\']\"}";

gLexer lexer = new gLexer(CharStreams.fromString(source));
gParser parser = new gParser(new CommonTokenStream(lexer));
ParseTree root = parser.start();

System.out.println(root.toStringTree(parser));

prints:

(start (root (expr (atom &))) (root (expr (atom \n))) (root (expr (atom ""))) (root (expr (atom \n))) (root (expr (atom {))) (root (expr (atom "))) (root (expr (atom pattern))) (root (expr (atom "))) (root (expr (atom :))) (root (expr (atom  ))) (root (expr (atom "))) (root (expr (atom \\:))) (root (expr (list ( (expr (atom \\s+)) )))) (root (expr (atom [))) (root (expr (atom &))) (root (expr (atom \\\))) (root (expr (atom "))) (root (expr (atom \\))) (root (expr (atom '))) (root (expr (atom ]))) (root (expr (atom "))) (root (expr (atom }))) <EOF>)

without any error messages.

If you get error mssages, it is most likely because you have not regenerated your lexer- and parser classes from your (new) grammar. Every time you update your grammar, you must regenerate the lexer- and parser classes.