Kubernetes: NFS with StatefulSets

Question

I'm trying to use existing NFS with StateFulSets.

Creation of PresistentVolumeClaim seems automatic using the volumeClaimTemplates.

Problem:

But, since the PresistentVolumeClaim claims a entire PresistentVolume. I have to create PresistentVolume manually for all the replicas. Is there a way to dynamically provision NFS persistent volumes in Kubernetes ?

Note: NFS Server itself is static, just need to create volumes in K8s dynamically, not the NFS Server itslef.

I'm using mongo statefulset example:

---
apiVersion: v1
kind: Service
metadata:
  name: mongo
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    role: mongo
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: mongo
spec:
  serviceName: "mongo"
  replicas: 3
  template:
    metadata:
      labels:
        role: mongo
    spec:
      terminationGracePeriodSeconds: 10
      containers:
        - name: mongo
          image: mongo
          command:
            - mongod
            - "--replSet"
            - rs0
            - "--smallfiles"
            - "--noprealloc"
          ports:
            - containerPort: 27017
          volumeMounts:
            - name: mongo-data
              mountPath: /data/db
        - name: mongo-sidecar
          image: cvallance/mongo-k8s-sidecar
          env:
            - name: MONGO_SIDECAR_POD_LABELS
              value: "role=mongo"
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 5Gi
      selector:
        matchLabels:
          type: nfs

It needs 3 PresistentVolumeClaim, so I have to create 3 PV for it to use. Can this be dynamically added in the NFS like other dynamic provisioners like aws-ebs

Is this the proper way to get a StatefulSet with NFS Persistent Volume.

Answer 1

This is a work in progress I came back to just yesterday (my solution) but my advice if it suits your purposes (or anyone finding this later) is to check out GlusterFS and Heketi.

Information is included below but the TLDR is that GlusterFS is your NFS, Heketi can autoprovision the rest. My github repo automates the setup... it's ugly... but it works for me and I'll be making it less ugly with what I know now.

---

GlusterFS is a free and open source scalable network filesystem. https://github.com/gluster/glusterfs https://www.gluster.org/

RESTful based volume management framework for GlusterFS https://github.com/heketi/heketi

---

https://github.com/stevenaldinger/gke-glusterfs-heketi

Answer 2

The good news is that yes Kubernetes now supports the dynamic creation of Persistent volumes as described in the following posting:

• http://blog.kubernetes.io/2017/03/dynamic-provisioning-and-storage-classes-kubernetes.html

The bad news is that this feature does not appear to support NFS... Which means a certain amount of sense considering each volume needs to be manually created even without Kubernetes involved :-(

Having said that Google found the following link that suggests there might be an unofficial way to provision NFS dynamically.

• https://github.com/kubernetes-incubator/external-storage/tree/master/nfs

Hope these links help.