I'm using the grun.bat (java org.antlr.v4.runtime.misc.TestRig %*).
Antlr version is 4.5
Running grun.bat with -tokens option on the input text results in 0 errors but the non-ASCII characters are not recognized so their token is '????'
Adding the encoding option to the grun.bat:
java -Dfile.encoding=UTF-8 org.antlr.v4.runtime.misc.TestRig %*
results in getting an error per non-ASCII character, e.g.
line 5:1 token recognition error at: 'ם'
The non-ASCII characters of the input text are included in utf-8.
How can I get the non-ASCII characters to be displayed correctly in the tokens?