Quex 0.69.4 generates include file that includes itself causing infinite loop

Question

I work on inherited code base that uses Quex 0.65.7. It compiles, but requires -fpermissive during compilation, which isn't something that I'm willing to live with. As code requiring -fpermissive option in g++ lives in files provided as a base by Quex itself, I'm trying to build lexer using Quex 0.69.4, installed using quex_0.69.4-0_all.deb file from project's download page. Lexer definition is very simple. It lives in file lexer.qx, wholly pasted below:

header {
        #include <cstdlib>
}

define {
        P_IDENTIFIER          [_a-zA-Z][_a-zA-Z0-9]*
}

mode ONE_AND_ONLY
{
    <<EOF>>     => RULE_TOKEN_TERMINATION;

    [ \t\r\n]+  { }
    "/*"([^*]|[\r\n]|("*"+([^*/]|[\r\n])))*"*"+"/" { }
    "//"[^\n]*"\n"           { }
    "+"    => RULE_TOKEN_PLUS;
    "-"    => RULE_TOKEN_MINUS;
    "/"    => RULE_TOKEN_DIVIDE;
    "*"    => RULE_TOKEN_TIMES;
    ";"    => RULE_TOKEN_SEMICOLON;
    "("    => RULE_TOKEN_LEFTPAREN;
    ")"    => RULE_TOKEN_RIGHTPAREN;
    "||"   => RULE_TOKEN_LOGICALOR;
    "&&"   => RULE_TOKEN_LOGICALAND;
    "<="   => RULE_TOKEN_LESSEQUAL;
    ">="   => RULE_TOKEN_GREATEREQUAL;
    "<"    => RULE_TOKEN_LESS;
    ">"    => RULE_TOKEN_GREATER;
    "["    => RULE_TOKEN_LEFTSQBRACKET;
    "]"    => RULE_TOKEN_RIGHTSQBRACKET;
    "=="   => RULE_TOKEN_EQUAL;
    {P_IDENTIFIER} => RULE_TOKEN_IDENTIFIER(Lexeme);
    [0-9]+      => RULE_TOKEN_INTEGER(Lexeme);
}

From this, lexer is generated as follows:

quex -i ../lexer.qx -o zs_expr_eng --output-directory . --foreign-token-id-file ../../grammar/grammar.h --token-id-prefix RULE_TOKEN_ --fes xx --foreign-token-id-file-show

File mentioned in --foreign-token-id-file option is equally simple:

#define RULE_TOKEN_TERMINATION                      1
#define RULE_TOKEN_UNINITIALIZED                    2
#define RULE_TOKEN_SEMICOLON                        3
#define RULE_TOKEN_IDENTIFIER                       4
#define RULE_TOKEN_LEFTSQBRACKET                    5
#define RULE_TOKEN_RIGHTSQBRACKET                   6
#define RULE_TOKEN_LOGICALOR                        7
#define RULE_TOKEN_LOGICALAND                       8
#define RULE_TOKEN_LESSEQUAL                        9
#define RULE_TOKEN_GREATEREQUAL                    10
#define RULE_TOKEN_LESS                            11
#define RULE_TOKEN_GREATER                         12
#define RULE_TOKEN_EQUAL                           13
#define RULE_TOKEN_PLUS                            14
#define RULE_TOKEN_MINUS                           15
#define RULE_TOKEN_DIVIDE                          16
#define RULE_TOKEN_TIMES                           17
#define RULE_TOKEN_LEFTPAREN                       18
#define RULE_TOKEN_RIGHTPAREN                      19
#define RULE_TOKEN_INTEGER                         20

Output from option --foreign-id-file-show indicates that all tokens were read and are understood:

command line: Token ids found in file '../../grammar/grammar.h' {
command line:      RULE_TOKEN_DIVIDE         => 'DIVIDE'
command line:      RULE_TOKEN_EQUAL          => 'EQUAL'
command line:      RULE_TOKEN_GREATER        => 'GREATER'
command line:      RULE_TOKEN_GREATEREQUAL   => 'GREATEREQUAL'
command line:      RULE_TOKEN_IDENTIFIER     => 'IDENTIFIER'
command line:      RULE_TOKEN_INTEGER        => 'INTEGER'
command line:      RULE_TOKEN_LEFTPAREN      => 'LEFTPAREN'
command line:      RULE_TOKEN_LEFTSQBRACKET  => 'LEFTSQBRACKET'
command line:      RULE_TOKEN_LESS           => 'LESS'
command line:      RULE_TOKEN_LESSEQUAL      => 'LESSEQUAL'
command line:      RULE_TOKEN_LOGICALAND     => 'LOGICALAND'
command line:      RULE_TOKEN_LOGICALOR      => 'LOGICALOR'
command line:      RULE_TOKEN_MINUS          => 'MINUS'
command line:      RULE_TOKEN_PLUS           => 'PLUS'
command line:      RULE_TOKEN_RIGHTPAREN     => 'RIGHTPAREN'
command line:      RULE_TOKEN_RIGHTSQBRACKET => 'RIGHTSQBRACKET'
command line:      RULE_TOKEN_SEMICOLON      => 'SEMICOLON'
command line:      RULE_TOKEN_TERMINATION    => 'TERMINATION'
command line:      RULE_TOKEN_TIMES          => 'TIMES'
command line:      RULE_TOKEN_UNINITIALIZED  => 'UNINITIALIZED'
command line: }

Lexer generation finishes successfully, but among generated files is zs_expr_eng-token.hxx and its content is pasted below:

#include "zs_expr_eng-token.hxx"

This is not everything, there are also 7 new line characters - so it is basically empty and includes itself without guards. This of course means that compiler fails to compile this source code:

Compiling prog.cpp
In file included from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                 from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
<many many maaaaany lines cut>
                  from lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1,
                  from lexer/package-arm-linux-gnueabihf/./lib/declarations:22,
                  from zs_expr_eng.hxx:49,
                  from grammar.y:13,
                  from prog.cpp:2:
 lexer/package-arm-linux-gnueabihf/zs_expr_eng-token.hxx:1:33: error: #include nested too deeply
  #include "zs_expr_eng-token.hxx"
                                  ^

Any ideas what should be done to prevent it? I don't think this is common occurrence as it would break all Quex-based projects.

Answer 1

So it seems that this version (0.69.4) or earlier introduced quite a lot of changes and also broke support for different file extensions (--fes or --file-extension-scheme) or at least fails to complain when this option is used. After removing --fes xx option from call to quex and changing file names as required (and also changing quex::Token to name of generated class) everything works as expected. At least for now.