Connecting to AI Platform Pipelines using the Kubeflow Pipelines SDK with Python and PyCharm

Question

Using windows 10 and Python 3.9 with PyCharm IDE

I am trying to list the pipelines using the following code:

import kfp
client = kfp.Client(host='.......bb92ea05-dot-us-central2.pipelines.googleusercontent.com')
    
client.list_pipelines()

I know the host variable is correct because I copied it from the AI platform Pipeline. I am getting the following problem:

FileNotFoundError: [WinError 2] The system cannot find the file specified

I think that the file can't be found since the program can't connect to the GCP's AI PLatform Pipeline.

I have installed Kubeflow Pipeline SDK by doing the following commands in Pycharm terminal:

• conda create --name mlpipeline python=3.7
• conda activate mlpipeline
• pip3 install kfp --upgrade --user

When I run:

kfp diagnose_me

I get the following command error:

Google Cloud SDK is not installed, gcloud, gsutil and kubectl are required for this app to run. Please follow instructions at https://cloud.google.com/sdk/install to install the SDK

I have tried

• Installing Google Cloud SDK and initializing gcloud(Although when I run a command it says 'Python not found')
• Installing kubectl and gsutil with gcloud commands

Answer 1

I transitioned over to Ubuntu and from there the problem was fixed.

What I did:

• Installed Google Cloud SDK and configured it: https://cloud.google.com/sdk/docs/quickstart

  Also install gsutil and kubectl using gcloud components install component_id command

• Install KFP SDK: Run the following commands in terminal:

   apt-get update; apt-get install -y wget bzip2
   wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
   bash Miniconda3-latest-Linux-x86_64.sh
  
   conda create --name mlpipeline python=3.7   
   conda activate mlpipeline
  
   pip3 install kfp --upgrade
  

Answer 2

I ran into the exact same problem. I fixed it by using Ubuntu OS rather than Windows OS. Use a VM and run Ubuntu in the VM. Ubuntu environment is way easier to set up and manage.