How to load a module from a byte sequence without saving it to a file in python version 3.11?

24 Views Asked by At

My loader receives part of the program from the server as .pyc files (the files themselves are NOT saving in client pc's filesystem, the byte sequence is stored in a variable).

I need to create a module from this byte sequence without creating a file.

I found a way to create a module from a string:

spec = importlib.util.spec_from_loader("modulename", loader=None) # creating spec
module = importlib.util.module_from_spec(spec) # creating module
exec(source, module.__dict__) # executing module
#  so that we can import it in other program files
sys.modules[name] = module
globals()[name] = module

But it doesn't work for me in my case.

The module is initially created with py_compile.compile("sorcefile.py", "destfile.pyc"). The answer to a similar question (How to load compiled python modules from memory?) suggests using marshal.loads(bytes[8:]) and indeed, looking at the source code of py_compile.compile function (https://svn.python.org/projects/python/trunk/Lib/py_compile.py) I saw that it uses marshal.dump(). However, as far as I understand, this code is deprecated, because looking at the source code of this function in my Python instance (3.11) (https://github.com/python/cpython/blob/3.11/Lib/py_compile.py), I saw that it no longer uses marshall.

This explains why this error appears in the following situation:

import marshal
import py_compile


py_compile.compile("source.py", "compiled.pyc")
with open("compiled.pyc", "rb") as f:
    module_bytes = f.read()

module = marshal.loads(module_bytes[8:])

Output:

Traceback (most recent call last):
  File "C:\test.py", line 9, in <module>
    module = marshal.loads(module_bytes[8:])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: bad marshal data (unknown type code)

(In this case, the pyc file is created only to emulate the data received from the server.)

So, is it possible to create a module from a sequence of bytes without creating a file in python 3.11 (as in the "string" method).

0

There are 0 best solutions below