Enable web access or interactive shell for PipelineJob tasks for Vertex AI

Question

I am trying to debug a PipelineJob that I launch on Vertex AI. Is there a way to enable web access on the components like you can when you launch Custom Jobs? This way, I could ssh into the running task and do a bunch of debugging.

Here is a simplified version of my pipeline code:

import kfp.v2.dsl as dsl

from google.cloud import aiplatform
from kfp.v2 import compiler
from kfp.v2.dsl import (
    component,
    Input,
    Output,
    Dataset,
    Metrics,
    Model,
    Artifact,
    graph_component
)
from copy import copy
from kfp.v2.google.client import AIPlatformClient
from typing import Optional, Dict, Union, List

@component(
    packages_to_install=['google-cloud-aiplatform']
)
def hello_world():
    import time
    print("Hello world")
    time.sleep(300)

@dsl.pipeline(
    name = "dataprep"
)
def train_model_pipeline(style: int):
    # Set Up Training and Test Data
    hello_op = hello_world()

I expected to be able to set enable_web_access(True) on the task, but that doesn't seem like an option because it's part of the CustomJob spec and not the PipelineTask.

Answer 1

As of now, there is no feature or any workaround to enable web access to kfp components.

There is already a filed feature request regarding this issue filed by @Optimus.

Posting the answer as community wiki for the benefit of the community that might encounter this use case in the future.

Feel free to edit this answer for additional information.