Where is the top level module in python import tree? How to define it?

Question

I have been working on a project that involves multiple files and have been having some issues with imports. I have read about 20 articles and stack overflow questions, but none seems to explain this behavior.

If I have a folder system that looks like this:

\package
    \folder_a
        __init__.py
        b.py
    __init__.py
    a.py

with a simple hello function in b.py. In a.py I have from package.folder_a.b import hello. When I run a.py, I get the error "ModuleNotFoundError: no module named 'package'".

Why does this happen? When a.py is run, isn't package the top level directory?

Now, I have found a fix for this by simply changing the code in a.py to from folder_a.b import hello, but this comes with it another set of issues. let us say that I am working with a slightly more complicated file structure(witch I am) such as:

\package
    \folder_a
        __init__.py
        b.py
    \folder_b
        __init__.py
        c.py
    __init__.py
    a.py

Here I want to import hello into c.py. BOTH from package.folder_a.b import hello AND from folder_a.b import hello reult in the same "ModuleNotFoundError: no module named 'package/folder_a(repectivly)'". I presume that this is because the file does not add thoes folders to the namespace when the file is executed (importing c into a when running a.py allows the import to work). Is there any way to force \package to be the top level module that does not involve running a script from a file in it's own directory? I hope this makes sense. Thanks.

Answer 1

Python treats the directory where the script being executed resides as the top level directory, as an example when executing b.py in directory folder_b, /folder_b would be considered as the top level directory and not the /package. to tackle your situation, you can use relative imports. Here's how you can adjust your imports in a.py and c.py:

file a.py -> from .folder_a.b import hello

file c.py -> from ..folder_a.b import hello

This allows Python to correctly resolve the module paths regardless of the current working directory or the location of the script being executed.

Answer 2

Let's rename package to avoid confusion, because the intent is not to define a package containing a module named a. You have a project that defines three things:

1. A top-level package that contains a module named b
2. A top-level package that contains a module named c
3. A script named a.py.

or you might define two things:

1. A top-level package package that contains two sub packages p1 and p2
2. A script named a.py.

So your source directory might look like

\project
    \p1
        b.py
    \p2
        c.py
    a.py

or

\project
    \package
        \p1
            b.py
        \p2
            c.py
    a.py

(Unless you actually have code to put in the __init__.py files, they haven't been necessary to define a package for a long time.)

When you install the code from a distribution package (say, with pip install myproject into a virtual environment), then a.py will go to an appropriate bin directory and the two packages p1 and p2 to an appropriate site-packages directory. For example, assuming the first alternative above,

$ python3 -m venv venv
$ venv/pip/install myproject
[...]
$ ls venv/bin/a.py
venv/bin/a.py
$ ls -d venv/lib/python3.12/site-packages/p*
venv/lib/python3.12/site-packages/p1
venv/lib/python3.12/site-packages/p1

So a.py should refer to p1 and p2 as top-level packages, for example with

from p1 import b
from p2 import c

not with relative imports as if a were a module in the same package as p1 and p2. If you opt for the second alternative,

from package.p1 import b
from package.p2 import c

---

If you do want a.py to be a module that can also be run as a script, then your source tree might look like

\project
    \package
        \p1
            b.py
        \p2
            c.py
        a.py

Now package.a is a module, along with package.p1 and package.p2, and a.py can use relative imports to refer to p1 and p2. To ensure that relative imports work when a.py is executed as a script as well, then add the following boilerplate before the first relative import:

if __name__ == '__main__' and __package__ is None:
    __package__ = 'package'