ANTLR - non-deterministic behaviour during run/debug

Question

I'm trying to use ANTLR (tried 3.3 and 3.4). Strange thing happens when I try to run my test code. Please see my very simple code first, I'll explain my problem afterwards.

Test grammar:

lexer grammar CSVLexer;
Comma 
  :  ','
  ;
LineBreak
  :  '\r'? '\n'
  |  '\r'
  ;
SimpleValue
  :  ~(',' | '\r' | '\n' | '"')+
  ;
QuotedValue
  :  '"' ('""' | ~'"')* '"'
  ;

Test code:

import org.antlr.runtime.*;
public class JMain {
    public static void main(String[] args) throws Exception {
        String source = "val1,val2,val3";
        CSVLexer lexer = new CSVLexer(new ANTLRStringStream(source));
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        int n = 1;
        for (Object o : tokens.getTokens()) {
            CommonToken token = (CommonToken) o;
            System.out.println("token(" + n + ") = " + token.getText().replace("\n", "\\n"));
            n++;
        }
    }
}

Compiation:

$ export CLASSPATH=antlr3.jar
$ java org.antlr.Tool CSVLexer.g
$ javac *.java
$ java JMain

The first problem is that it doesn't print anything :). I work in Intellij IDEA (10) and I tried to run the code from here without success (with ANTLR in CLASSPATH). HOWEVER, if I put breakpoint on int i = 1, debug and wait, say 1-3 seconds before continuing, it works! If I run it in debug mode without breakpoint it doesn't. Could someone please explain to me what could be the problem? Thank you.

EDIT:

So I tried tu put:

tokens.fill()

right after

CommonTokenStream tokens = new CommonTokenStream(lexer);

and it worked. However then I started debugging CommonTokenStream constructor (several mins) and when I got on tokens.fill() I got IndexOutOfBoundException. So somehow something in background did something but I can't see no other threads in IDEA.

EDIT2:

Problem solved. It appears that it is necessary to call fill() first and that BufferedTokenStream probably wasn't meant to be used like this. I followed this tutorial http://bkiers.blogspot.com/2011/03/2-introduction-to-antlr.html where it probably somehow worked for author (I wonder why). IntelliJ IDEA debugger called toString() on local objects and it in turn called fill() so that's why it worked when I set breakpoint. toString() modyfing state is evil!