How can I fix this python dependency?

Question

I just downloaded greenplum for os x. When I try to start the db, I get this error -- which I take to mean that it is missing gppylib.

/usr/local/greenplum$ bin/gpstart
Traceback (most recent call last):
  File "bin/gpstart", line 9, in <module>
    from gppylib.mainUtils import *
ImportError: No module named gppylib.mainUtils

Here is the line from the file that is causing the error:

from gppylib.mainUtils import *

I try pip install gppylib and pip install gppylib.mainUtils but neither can find the package. I can't find the package on google.

Am I correctly understanding that python is looking for a package called gppylib? If so, where can I find it?

Answer 1

I was able to reproduce the error, it seems the path isn't set to include /usr/local/greenplum-db-4.2.2.4/lib/python/gppylib/mainUtils.py, which is where the package is in my installation. When I did export PYTHONPATH=/usr/local/greenplum-db-4.2.2.4/lib/python/ I got another error message:

Error: unable to import module: dlopen(/usr/local/greenplum-db-4.2.2.4/lib/python/pygresql/_pg.so, 2): no suitable image found.  Did find:
    /usr/local/greenplum-db-4.2.2.4/lib/python/pygresql/_pg.so: mach-o, but wrong architecture

I suspect this is because the pygresql is compiled for a 32 bit architecture, whereas I'm using x86_64 (unrelated to your issue of not finding the gppylib in the first place):

$ lipo -info /usr/local/greenplum-db-4.2.2.4/lib/python/pygresql/_pg.so
Non-fat file: /usr/local/greenplum-db-4.2.2.4/lib/python/pygresql/_pg.so is architecture: i386

Note also that the readme says:

---

REQUIRED OS SYSTEM SETTINGS FOR GREENPLUM DATABASE

---

These must be set on all Greenplum hosts (master and segments). Make sure you restart your system after adding or changing kernel parameters.

Add all of the following to /etc/sysctl.conf:

kern.sysv.shmmax=2147483648

kern.sysv.shmmin=1

kern.sysv.shmmni=64

kern.sysv.shmseg=16

kern.sysv.shmall=524288

kern.maxfiles=65535

kern.maxfilesperproc=65535

net.inet.tcp.msl=60

If using DHCP, add the following line to /etc/hostconfig:

HOSTNAME=""

You would then use this host name when exchanging ssh keys and when initializing your Greenplum Database system.

---

POST INSTALLATION STEPS

---

These are the high-level steps to configure and initialize a new Greenplum Database instance. For detailed instructions, please see the 'Greenplum Database Installation Guide' available for download from http://powerlink.emc.com

1. Allocate a 'gpadmin' OS user to own and run your installation. This user must exist on all Greenplum hosts.

2. Source the greenplum_path.sh file in your 'gpadmin' user profile (.bashrc) of your master host. This sets the environment variables needed by Greenplum Database.

3. Use the gpseginstall utility to install and configure the Greenplum software on all hosts.

4. Create your data directory locations on all Greenplum hosts.

5. Use the gpinitsystem utility on the master host to initialize and start your Greenplum Database system. This utility requires a configuration file and a host file. For example:

   gpinitsystem -c gpinitsystem_config -h hostfile_gpinitsystem

   A sample multi-node initialization configuration file can be found in $GPHOME/docs/cli_help/gpconfigs/gpinitsystem_config. Edit the example files to reflect your desired Greenplum Database configuration.

---

ABOUT YOUR INSTALLATION

---

The Greenplum Database installer installs the following files and directories:

• greenplum_path.sh - Greenplum Database environment variables.

• GPDB-LICENSE.txt - Greenplum license agreement.

• LICENSE.thirdparty - Licenses for third-party tools

• bin - Greenplum Database server programs, client programs, and management tools.

• demo - Greenplum Database demonstration programs.

• docs - Greenplum Database documentation.

• etc - Sample configuration file for OpenSSL.

• ext - Bundled programs (such as Python) used by some Greenplum Database utilies.

• include - Greenplum Database and PostgreSQL header files.

• lib - Greenplum Database and PostgreSQL library files.

• sbin - Supporting/Internal scripts and programs.

• share - PostgreSQL sample files and templates.

---

GREENPLUM DATABASE DOCUMENTATION

---

The following documentation is available for download from http://powerlink.emc.com:

(All Greenplum Database Installations)

• GPDB--README.pdf - Release notes for this version of Greenplum Database

• GPInstallGuide.pdf - Greenplum Database Installation Guide

• GPAdminGuide.pdf - Greenplum Database Administrator Guide

• GPPerfmonAdminGuide.pdf - Greenplum Performance Monitor Administrator Guide

Answer 2

This worked for me:

source $GPHOME/greenplum_path.sh