Why can't twistd import a module from it's current working directory?

Question

Please consider the following test case.

The project directory is structured thusly:

foo
├── foo
│   ├── __init__.py
│   └── bar.py
└── test.tac

bar.py contains a trivial class definition:

# bar.py
class Bar:
    pass

test.tac, the Twisted Application Configuration file, contains a single import statement:

#test.tac
from foo.bar import Bar

When running twistd -ny test.tac, I obtain the following error:

$ twistd -ny test.tac
Unhandled Error
Traceback (most recent call last):
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/application/app.py", line 674, in run
    runApp(config)
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/scripts/twistd.py", line 25, in runApp
    runner.run()
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/application/app.py", line 381, in run
    self.application = self.createOrGetApplication()
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/application/app.py", line 453, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/application/app.py", line 464, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/application/service.py", line 416, in loadApplication
    application = sob.loadValueFromFile(filename, 'application')
  File "/Users/lthibault/.pyenv/versions/3.6.2/lib/python3.6/site-packages/twisted/persisted/sob.py", line 177, in loadValueFromFile
    eval(codeObj, d, d)
  File "test.tac", line 1, in <module>
    from foo.bar import Bar
builtins.ModuleNotFoundError: No module named 'foo'


Failed to load application: No module named 'foo'

This is very surprising because python test.tac produces no error. In order to further debug, I modified test.tac as follows:

from sys import path
print(path)
from foo.bar import Bar

This revealed that running python <filename> prepends the current working directory to the path whereas running twistd <filename> does not.

My question is twofold:

1. Is this a bug, or is there a good reason for twistd to behave this way?
2. What is the recommended way of fixing this in twistd. Is there some sort of option I can set or does it need to be done manually?

Answer 1

this snippet will add directory containing the tac file to python path

import os
import sys

sys.path.append(os.path.abspath(os.path.dirname(__file__)))

from foo.bar import Bar

Answer 2

There is no rule in Python saying the working directory should be on the path. Rather, by default, Python puts the directory of the main script on the path. When you run python test.tac, that directory is the working directory, but when you run twistd, the main Python script is some part of Twisted, which does not lie in the working directory.

As I am unfamiliar with Twisted, I cannot say what combination of code organization and Twisted configuration you're supposed to use to make this import work.