kube-Prometheus-stack - Grafana pod stuck at "CrashloopBackoff"

Question

Trying to install kube-prometheus-stack (39.8.0) and everything went well, now there is a requirement - where grafana pod needs to have a persistent vol with oci-fss as a storage class, below is my values.yaml file.

grafana:
  initChownData:
    enabled: false
  persistence:
    enabled: true
    type: pvc
    storageClassName: oci-fss
    accessModes:
    - ReadWriteMany
    size: 50Gi
    finalizers:
    - kubernetes.io/pvc-protection

Grafana pod status is:

pod/prometheus-grafana-54cdc8774f-blfqx                      2/3     CrashLoopBackOff

Grafana pod logs:

ubectl logs -f  pod/prometheus-grafana-54cdc8774f-blfqx -n prometheus
Defaulted container "grafana-sc-dashboard" out of: grafana-sc-dashboard, grafana-sc-datasources, grafana
{"time": "2022-11-11T08:07:40.625012+00:00", "level": "INFO", "msg": "Starting collector"}
{"time": "2022-11-11T08:07:40.625190+00:00", "level": "WARNING", "msg": "No folder annotation was provided, defaulting to k8s-sidecar-target-directory"}
{"time": "2022-11-11T08:07:40.625329+00:00", "level": "INFO", "msg": "Loading incluster config ..."}
{"time": "2022-11-11T08:07:40.626083+00:00", "level": "INFO", "msg": "Config for cluster api at 'https://10.96.0.1:443' loaded..."}
{"time": "2022-11-11T08:07:40.626199+00:00", "level": "INFO", "msg": "Unique filenames will not be enforced."}
{"time": "2022-11-11T08:07:40.626283+00:00", "level": "INFO", "msg": "5xx response content will not be enabled."}

Answer 1

it was filesystem storage issue, Grafana container was not able to write it's configuration to the storage, we can mark this as resolved