I write a language lexer/parser/compiler in python, that should run in the LLVM JIT-VM (using llvm-py
) later. The first two steps are quite straightforward for now, but (even if I didn't start the compile-task yet) I see a problem, when my code wants to call Python-Code (in general), or interact with the Python lexer/parser/compiler (in special) respectively. My main concern is, that the code should be able to dynamically load additional code into the VM at runtime and thus it must trigger the whole lexer/parser/compiler-chain in Python from within the VM.
First of all: Is this even possible, or is the VM "unmutable" once it is started?
If it is I currently see 3 possible solutions (I am open for other suggestions)
- "Break out" of the VM and make it possible to call Python functions of the main process directly (maybe by registering it as a LLVM-function, that redirects to the main process somehow). I didn't found anything about this and anyway I am not sure, if this is a good idea (security and such).
- Compile the runtime (statically or dynamically at runtime) into LLVM-Assembly/-IR. This requires, that the IR-code is able to modify the VM it runs in
- Compile the runtime (statically) into a library and load it directly into the VM. Again it must be able to add functions (etc) to the VM it runs in.
You can call external C functions from LLVM JIT-ed code. What else do you need?
These external functions will be found in the executing process, meaning that if you link Python into your VM you can call Python's C API functions.
The "VM" is probably less magic than you think it is :-) In the end, it's just machine code that gets emitted at runtime into a buffer and executed from there. To the extent that this code has access to other symbols in the process in which it's running, it can do everything any other code in that process can do.