Internal error occurred: failed calling webhook "v1.vseldondeployment.kb.io" while deploying Seldon yaml file on minikube

Question

I am trying to follow the instruction on Seldon to build and deploy the iris model on minikube. https://docs.seldon.io/projects/seldon-core/en/latest/workflow/github-readme.html#getting-started I am able to install Seldon with Helm and Knative using YAML file. But while I am trying to apply this YAML file to deploy the Iris model, I am having the following error:

Internal error occurred: failed calling webhook "v1.vseldondeployment.kb.io": Post "https://seldon-webhook-service.seldon-system.svc:443/validate-machinelearning-seldon-io-v1-seldondeployment?timeout=30s": dial tcp 10.107.97.236:443: connect: connection refused

I used kubectl apply YAML on other files such as knative and broker installation they don't have this problem, but when I kubectl apply any SeldonDeployment YAML file this error comes up, I also tried the cifar10.yaml for cifar10 model deploy and mnist-model.yaml for mnist model deploy they have the same problem.

Has anyone experienced similar kind of problem and what are the best ways to troubleshoot and solve the problem?

My Seldon is 1.8.0-dev, minikube is v1.19.0 and kubectl Server is v1.20.2 Here is the YAML file:

kind: SeldonDeployment
    metadata:
      name: iris-model
      namespace: seldon
    spec:
      name: iris
      predictors:
      - graph:
          implementation: SKLEARN_SERVER
          modelUri: gs://seldon-models/sklearn/iris
          name: classifier
        name: default
        replicas: 1

Error Code

Answer 1

Make sure that the Seldon core manager in seldon-system is running ok: kubectl get pods -n seldon-system.

In my case, the pod was in CrashLoopBackOff status and was constantly restarting.

Turns out the problem had been while installing the seldon. Instead of having

helm install seldon-core seldon-core-operator \
 — repo https://storage.googleapis.com/seldon-charts \
 — set usageMetrics.enabled=true \
 — set istio.enabled=true \
 — namespace seldon-system

try once:

helm install seldon-core seldon-core-operator \
    --repo https://storage.googleapis.com/seldon-charts \
    --set usageMetrics.enabled=true \
    --namespace seldon-system \
    --set ambassador.enabled=true

Reference

P. S. When reinstalling you can just delete all the namespaces (which shouldn't be a problem since ur just doing a tutorial) with kubectl delete --all namespaces.