Unable to install pycocotools on Google AI-Platform (CMLE)

Question

I'm getting this error while trainer package installation on AI-Platform,

Traceback (most recent call last): File "", line 1, in File "/tmp/pip-install-_u8thvm6/pycocotools/setup.py", line 2, in from Cython.Build import cythonize ImportError: No module named 'Cython'

Although I've included 'Cython' in setup.py.

setup.py:

import setuptools

NAME = 'trainer'
VERSION = '1.0'
REQUIRED_PACKAGES = [
    'Cython', # Cython, mentioned before pycocotools
    'tensorflow-gpu',
    'google-cloud-storage',
    'gcsfs',
    'pycocotools'
]

setuptools.setup(
    name=NAME,
    version=VERSION,
    packages=setuptools.find_packages(),
    install_requires=REQUIRED_PACKAGES,
    include_package_data=True,
    description='Trainer package')

Answer 1

You need to install cython before running setup.py. The problem is that cython is needed at build time, not runtime, and there’s no guarantee about which order the packages you listed in install_requires get installed. So when pip tries to install pycocotools it hasn’t yet installed cython and aborts.

Answer 2

By adding these lines in setup.py the error is solved,

import setuptools

# Install cython before setup
import os                                       # <--- This line added
os.system('pip3 install --upgrade cython')      # <--- This line added

NAME = 'trainer'
VERSION = '1.0'
REQUIRED_PACKAGES = [
    'Cython', # Cython, mentioned before pycocotools
    'tensorflow-gpu',
    'google-cloud-storage',
    'gcsfs',
    'pycocotools'
]

setuptools.setup(
    name=NAME,
    version=VERSION,
    packages=setuptools.find_packages(),
    install_requires=REQUIRED_PACKAGES,
    include_package_data=True,
    description='Trainer package')