I'm attempting to run a dask hello world program on a Kubernetess cluster to which I have (limited) access. I'm running this in Windows 10. Using kubectl in powershell, I can successfully create a pod from a yaml file that pulls and runs a docker container. This works because I've also previously set up my config in ~/.kube/config to specify the namespace and my authorization.
To do the same programmatically (I hope):
import os
from dask_kubernetes import KubeCluster, KubeConfig
from dask.distributed import Client
import dask.array as da
path_to_src = os.path.dirname(os.path.realpath(__file__))
cluster = KubeCluster(os.path.join(path_to_src, 'pod-spec.yaml'))
client = Client(cluster)
# Create a large array and calculate the mean
array = da.ones((1000, 1000, 1000))
print(array.mean().compute()) # Should print 1.0
But this leads to a permissions error:
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User \"<not_my_kubectl_config_username>\" cannot create resource \"pods\" in API group \"\" in the namespace \"default\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
The error suggests that it is not using my kubectl config because the user name is not the one I use for kubectl, and the namespace should not be default. The documentation suggests that KubeCluster will use the kubectl config file if it's in the default location (~/.kube/config), so I'm not sure where to go from here.