Folder structure inside Python "embedded" packaging

Question

I distribute a ready-to-run software for Windows written in Python by:

• shipping the content of an embedded version of Python, say python-3.8.10-embed-amd64.zip
• adding a myprogram\ package folder (= the program itself)
• simply run pythonw.exe -m myprogram to start the program

It works well (and is a simple alternative to cxfreeze and co, but this part is out of topic here).

The tree structure is:

main\
    _asyncio.pyd
    _bz2.pyd
    ... + other .pyd files
    libcrypto-1_1.dll
    ... + other .dll files
    python.exe
    pythonw.exe    
    python38._pth
    python38.zip
    ...
    myprogram\     # here my main program as a module
    PIL\           # dependencies
    win32\
    numpy\
    ...            # many other folders for dependencies

Is there a way to move all the dependencies folders to a subfolder and still have Python (embedded version) be able to locate them? How to do so? More precisely, like this:

main\
    python.exe
    pythonw.exe    
    python38._pth
    python38.zip
    ...
    myprogram\     # here my main program as a module
    dependencies\
       PIL\         # all required modules 
       win32\
       numpy\
       ...      
       _asyncio.pyd   # and also .pyd files
       ...          

NB: the goal is here to use an embedded Python, which is totally independent from the system's global Python install. So this is independent from any environment variable such as PYTHONPATH etc.

Answer 1

Is there a way to move all the dependencies folders to a subfolder and still have Python (embedded version) work?

Yes. Dependencies can in principle be installed anywhere, as long as the folder that contains them is on the PYTHONPATH (see: PYTHONPATH).

The OP later added:

NB: the goal is here to use an embedded Python, which is totally independent from the system's global Python install. So this is independent from any environment variable such as PYTHONPATH etc.

In that case the 3d party dependencies can still be installed in any subfolder. You should be able to add the folder to the ._pht file, according to a Python Core dev:

The embeddable package is intended for environments that you don’t control, so you need the isolation. If you are adding a site-packages folder that you want imported, just add it to the ._pth file and it will be included in sys.path (https://discuss.python.org/t/add-a-command-line-parameter-to-add-folders-to-sys-path-needed-for-embedded-python/28426/10)

This information can also be found in the Python docs:

For those who want to bundle Python into their application or distribution, the following advice will prevent conflicts with other installations:

Include a ._pth file alongside your executable containing the directories to include. This will ignore paths listed in the registry and environment variables, and also ignore site unless import site is listed.

(https://docs.python.org/3.12/using/windows.html#windows-embeddable)

But if the intended usage is to simply run pythonw.exe -m myprogram then this is not running in an isolated, sandboxed environment, and it's not clear why a special embeddable Python version would be used in that case. It's definitely not the typical usecase for embeddable Pythons.