I have this issue while trying to import my csv file located on my local machine : FileNotFoundError: [Errno 2] No such file or directory: '/python/ml/mnist27_train.csv', I find it in the logs of kubeflow pipeline. It's seen that i need a volume to be mounted on kubernetes configuration. Please do you have any idea to fix this issue ?
Here is my code to running my kubeflow pipeline :
import pandas as pd
from kfp import dsl
from kfp.dsl import component
import os
# Define a simple component to read a CSV file and print the first few rows
@component
def read_csv(file_path: str = '/python/ml/mnist27_train.csv'):
try:
import pandas as pd
except ImportError:
print("Installing pandas...")
# Install pandas using pip within the component
import subprocess
subprocess.run(["pip", "install", "pandas"])
import pandas as pd
df = pd.read_csv(file_path)
print(df.head())
# Define a pipeline using the component
@dsl.pipeline(
name='Read CSV Pipeline',
description='A pipeline to read a CSV file'
)
def read_csv_pipeline():
file_path = '/python/ml/mnist27_train.csv'
if os.path.exists(file_path):
print(f"File exists at {file_path}")
else:
print(f"File not found at {file_path}")
read_csv(file_path=file_path) # Pass file_path as a keyword argument
# Compile and run the pipeline
if __name__ == '__main__':
import kfp.compiler as compiler
compiler.Compiler().compile(read_csv_pipeline, 'read_csv_pipeline.yaml')
run = kfp.Client().create_run_from_pipeline_func(read_csv_pipeline, arguments={})
I try without success to create a persistent volume call 'persvol.yaml' with this content :
apiVersion: v1
kind: PersistentVolume
metadata:
name: data-pv-volume
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteMany
hostPath:
path: "/python/ml/mnist27_train.csv"
and make a :
kubectl apply -f persvol.yaml
any help will be very appreciated