Automatically converting .py file to .pyx using python type hints

Question

Recently I had to convert a module written in pure python to pyx in in order to compile with cython. The procedure for converting from py to pyx was very straight forward since the all the variables and functions were type hinted. So it was a just a matter of looking up the cython static type for each types in python.

What are the current options available for automating the conversion of a module written in pure Python to Cython, specifically for converting .py files to .pyx files, taking into account the use of Python type hints in the original code? Are there any existing modules or tools that can facilitate this process? if NO, is it theoretically possible to develop a module that can automatically convert Python type hints to Cython static types, and if so, what challenges may arise in the development of such a module?

Answer 1

Cython already uses Python type annotations so the chances are you didn't need to do anything.

There's a few small notes:

• the most recent Cython 3 alpha versions have better support for this than the 0.29.x branch (so upgrade if you can),
• int is assumed to to be a Python object rather than a C int to avoid overflow for large ints and keep the semantics the same as Python. Use cython.int as the annotation for a C int.

"A bit more clarification:" Cython is generally pretty conservative with type annotations. It assumes the annotations are correct but aren't for its benefit. Therefore it doesn't do things where the annotation might change the answer. int is the main example - Python ints can be infinitely large while C ints have a fixed maximum value. Therefore, Cython doesn't change an int annotation to a C int because the result could change. Obviously if you're happy with the potentially different behaviour you can make this change yourself.

---

If you want to automate a fully conversion then the builtin ast module is probably the easiest way to do it. I'd still recommend sticking with "pure Python mode" for simplicity, just because the ast module can generate Python code. But essentially what you'd do is create a visitor that looks for annotations (i.e. ast.AnnAssign, ast.Arg) and replaces it will an appropriate Cython-friendly annotation. You then use ast.dump to rewrite the modified code.