ImportError: No module named bs4 when called from launchd

Question

I am running the following python imports in a script that works fine from the command line in terminal and from the ipython notebook.

#!/usr/bin/python
import os
import re
import urllib
import urllib2 as ul
import sys
from bs4 import BeautifulSoup as bs

When it's called from a .plist file via Mac launchd I get the following error:

11/24/15 1:17:05 PM com.jerry.sat_images[668]   Traceback (most recent call last):
11/24/15 1:17:05 PM com.jerry.sat_images[668]     File "/Users/jerrykallam/python_practice/sat_image.py", line 6, in <module>
11/24/15 1:17:05 PM com.jerry.sat_images[668]       import bs4
11/24/15 1:17:05 PM com.jerry.sat_images[668]   ImportError: No module named bs4
11/24/15 1:17:05 PM com.apple.launchd.peruser.501[165]  (com.jerry.sat_images[668]) Exited with exit code: 1

From the command line and ipython bs4 imports and the script works fine. This is the .plist code that seems to work correctly. Not sure why the script fails to import bs4 only when called by launchd.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.jerry.satimages</string>
        <key>ProgramArguments</key>
        <array>
           <string>python</string>
            <string>/Users/jerrykallam/python_practice/sat_image.py</string>
        </array>
        <key>StartInterval</key>
        <integer>360</integer>
        <key>RunAtLoad</key>
        <true/>
    </dict>
</plist> 

Answer 1

This is likely because launchd runs its daemons as root, and bs4 is likely not installed in a location in the super user's PYTHONPATH. To fix this, you can add an EnvironmentVariables key, and set the value of your PYTHONPATH there. To figure out what your PYTHONPATH is, you can run echo $PYTHONPATH.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.jerry.satimages</string>
        <key>ProgramArguments</key>
        <array>
           <string>python</string>
            <string>/Users/jerrykallam/python_practice/sat_image.py</string>
        </array>
        <!-- ADDED THIS -->
        <key>EnvironmentVariables</key>
        <dict>
            <key>PYTHONPATH</key>
            <string>/your/python/path</string>
        </dict>
        <key>StartInterval</key>
        <integer>360</integer>
        <key>RunAtLoad</key>
        <true/>
    </dict>
</plist> 

Answer 2

Look for differences in sys.path between the CLI and when it's run in from the plist. If the bs4 module isn't in the right place relative to the python path it's not going to find it.

 #!/usr/bin/env python

 import sys
 print sys.path 

With the plist version you might have to:

fd=open('/var/tmp/fred.txt','w') 

ted = sys.path

fd.writelines(ted) 
fd.close() 

instead of just "print sys.path".

Basically I'm guessing that the .plist file is getting run before your unix(ish) environment is fully set up.

Answer 3

I ran into this exact problem (OS X 10.11.3 El Cap). Adding PYTHONPATH to EnvironmentVariables did not work. What did work is calling the specific path for python.

So, in ProgramArguments instead of:

   <string>python</string>

... this:

   <string>/your/python/path</string>