I noticed something weird when I inspect the sys.path in my python3.8 virtual env. Specifically, I have the following hierarchy in my playground:
.
├── mmdetection
├── monodepth2
├── py37
├── py38
└── qd-track
When I invoke the python environment py38 and look at the sys.path members, I noticed the following pattern:
dian@ubuntu:~/playground$ ls
mmdetection monodepth2 py37 py38 qd-track
dian@ubuntu:~/playground$ source py38/bin/activate
(py38) dian@ubuntu:~/playground$ python
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/dian/playground/py38/lib/python3.8/site-packages', '/home/dian/playground/mmdetection', '/home/dian/playground/py38/lib/python3.8/site-packages/terminaltables-3.1.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/mmpycocotools-12.0.3-py3.8-linux-x86_64.egg', '/home/dian/playground/qd-track', '/home/dian/playground/py38/lib/python3.8/site-packages/motmetrics-1.2.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/mmcv-1.1.5-py3.8-linux-x86_64.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/xmltodict-0.12.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/scipy-1.5.3-py3.8-linux-x86_64.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/pytest_benchmark-3.2.3-py3.8.egg']
Interestingly, mmdetection and qd-track, which are on the same level of py38, got picked up by sys.path, while monodepth2 and py37 weren't. I changed the name of mmdetection to mmdetection2, as well as created a couple of new empty folders detectron, detectron2, which made the current folder look like:
.
├── detectron
├── detectron2
├── mmdetection2
├── monodepth2
├── py37
├── py38
└── qd-track
Then I tried to print sys.path again, and got:
(py38) dian@ubuntu:~/playground$ python
Python 3.8.0 (default, Oct 28 2019, 16:14:01)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/dian/playground/py38/lib/python3.8/site-packages', '/home/dian/playground/detectron2', '/home/dian/playground/py38/lib/python3.8/site-packages/terminaltables-3.1.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/mmpycocotools-12.0.3-py3.8-linux-x86_64.egg', '/home/dian/playground/qd-track', '/home/dian/playground/py38/lib/python3.8/site-packages/motmetrics-1.2.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/mmcv-1.1.5-py3.8-linux-x86_64.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/xmltodict-0.12.0-py3.8.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/scipy-1.5.3-py3.8-linux-x86_64.egg', '/home/dian/playground/py38/lib/python3.8/site-packages/pytest_benchmark-3.2.3-py3.8.egg']
This time, mmdetection2 disappeared, detectron2 got picked up while detectron was not. This is so weird, since the paths seem to be detected by python on the fly instead of being read from some fixed indexes. And exactly how they are detected is confusing to me. I inspected PYTHONPATH and it's empty.
What's more interesting, I tried switching to the py37 environment, which sits side-by-side to py38, and nothing described above happened:
(py38) dian@ubuntu:~/playground$ deactivate
dian@ubuntu:~/playground$ source py37/bin/activate
(py37) dian@ubuntu:~/playground$ python
Python 3.7.5 (default, Nov 7 2019, 10:50:52)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/home/dian/playground/py37/lib/python3.7/site-packages', '/home/dian/playground/py37/lib/python3.7/site-packages/terminaltables-3.1.0-py3.7.egg']
So, I guess this might due to some new changes to the loading mechanics starting from python3.8? And, what exactly behavior should I expect when I make use of sys.path? The official documentation doesn't give an exact list of items to be included in sys.path.
By the way I'm using Ubuntu 18.04.3 LTS; py37 and py38 are created via virtualenv.