I have a AKS cluster (k1) with 3 nodes. I have deployed ECK operator using all_in_one.yaml. I have another AKS cluster(k2) which is running my application stack , i have installed filebeats in this cluster (k2) and it is shipping the logs to elasticsearch to another k8s cluster (k1). My k2 cluster configuration is 3 nodes (4 vCPU, 16 GiB RAM , 32GiB Temp Storage). My elasticsearch.yaml:
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
name: dev-elasticsearch
spec:
version: 7.9.2
http:
# Disable https
tls:
selfSignedCertificate:
disabled: true
service:
spec:
type: LoadBalancer
nodeSets:
- name: default
count: 3
config:
node.master: true
node.data: true
node.store.allow_mmap: false
xpack.security.enabled: true
discovery.seed_hosts:
- 10.x.x.x
- 10.x.x.x
- 10.x.x.x
Note : When i start the start es and kibana , everything works fine for about 30 mins and then i have service unavailable error in es
{"error":{"root_cause":[{"type":"master_not_discovered_exception","reason":null}],"type":"master_not_discovered_exception","reason":null},"status":503}
Another thing i noticed is elastic-internal-init-filesystem is in Terminated state.
- Can someone help me resolve this issue ?
- Also how can i write the logs to a az storage with more space instead on default PV ?
Thanks for your help !!