A recent update (slightly unwitting) from Ubuntu 16.04 to Ubuntu 18.04 server seems to have broken our production CKAN server, which was originally installed in 16.04 from source, version 2.7.2, following the standard installation docs. I can run the server on port 5000 with these commands, from the end of those docs:-
cd /usr/lib/ckan/default/src/ckan
paster serve /etc/ckan/default/development.ini
and all seems to work well in a browser. But the main production server is giving me an Error 500 Internal Server Error. If I curl -I 127.0.0.1:8080, from the server then in /var/log/apache2/ckan_default.error.log I get the following errors each time:-
Error loading 'pyutilib.component' entry points: ''NoneType' object is not callable'
mod_wsgi (pid=9027): Target WSGI script '/etc/ckan/default/apache.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=9027): Exception occurred processing WSGI script '/etc/ckan/default/apache.wsgi'.
Traceback (most recent call last):
File "/etc/ckan/default/apache.wsgi", line 5, in <module>
from paste.deploy import loadapp
File "/usr/local/lib/python2.7/dist-packages/paste/deploy/__init__.py", line 3, in <module>
from paste.deploy.loadwsgi import *
File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 8, in <module>
import pkg_resources
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3251, in <module>
@_call_aside
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3235, in _call_as
ide
f(*args, **kwargs)
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3279, in _initial
ize_master_working_set
for dist in working_set
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 3279, in <genexpr
>
for dist in working_set
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2785, in activate
declare_namespace(pkg)
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2284, in declare_
namespace
_handle_ns(packageName, path_item)
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 2217, in _handle_
ns
loader.load_module(packageName)
File "/usr/lib/python2.7/pkgutil.py", line 243, in load_module
mod = imp.load_module(fullname, self.file, self.filename, self.etc)
File "/usr/local/lib/python2.7/dist-packages/pyutilib/component/__init__.py", line 15, in <module>
core.PluginGlobals.add_env("pca")
AttributeError: type object 'PluginGlobals' has no attribute 'add_env'
mod_wsgi (pid=9027): Target WSGI script '/etc/ckan/default/apache.wsgi' cannot be loaded as Python module.
mod_wsgi (pid=9027): Exception occurred processing WSGI script '/etc/ckan/default/apache.wsgi'.
Traceback (most recent call last):
File "/etc/ckan/default/apache.wsgi", line 5, in <module>
from paste.deploy import loadapp
File "/usr/local/lib/python2.7/dist-packages/paste/deploy/__init__.py", line 3, in <module>
from paste.deploy.loadwsgi import *
File "/usr/local/lib/python2.7/dist-packages/paste/deploy/loadwsgi.py", line 8, in <module>
import pkg_resources
File "/usr/lib/ckan/default/lib/python2.7/site-packages/pkg_resources/__init__.py", line 79, in <module>
from . import py31compat
ImportError: cannot import name py31compat
I'm confused about why we seem to have references to both /usr/lib/ckan/default/lib/python2.7 and also /usr/local/lib/python2.7 in the logs. But I've checked both of these have package versions that match with those in the requirements.txt for CKAN 2.7 (the one exception is psycopg2, which we had to upgrade to 2.7.3.2 to cope with a postgresql version issue described elsewhere). I've also tried various restarts/reboots.
Any ideas on how to fix/where to debug next would be very appreciated.
CKAN needs to be running under python 2.7, make sure your ubuntu is using python 2.7 if not you need to set it to use it, i had to move from ubunto to centos 7 in order to get rid of this issues easily.