How to build preprocessor solution with Tatsu?

Question

I'm writing a transpiler for Tiny C code to Python code, but I need to build a preprocessor solution to replace #define and manage compilator C directive (#ifdef, #else, #define...)

I choose to use a pcpp module in Python but with no success... is a solution possible in a full Tatsu solution?

Answer 1

The parts of TatSu supporting pre-processing are, unfortunately, undocumented.

You can take a look at how Buffer.include() is implemented, replacing the original lines and line information by those of the block produced by the transformation.

def include(self, lines, index, i, j, name, block, **kwargs):
        blines, bindex = self._preprocess_block(name, block, **kwargs)
        assert len(blines) == len(bindex)
        lines[i:j] = blines
        index[i:j] = bindex
        assert len(lines) == len(index)
        return j + len(blines) - 1

Basically, you can hook into the preprocessing methods with your own Buffer class, and transform the text in whichever way. A proper list of LineIndexInfo will let the parser report errors agains original source lines.

For a preprocessor with macros, you'll probably need a parser for the preprocessor, and a parser for the main language. The trick will be making the first parser preserve the line information for the second one.

The preprocessor may run separately, but it must inject metadata enough to allow the main parser to recover the original line information.