while I scale down EC2 node count into zero (basically EC2 gets terminated)under each ASG of EKS cluster and scale up again , then dynamically provisioned EBS(PV) volume gets deleted , though "Delete on termination" flag is set to "No" in case of EBS(PV). why its happening ? suppose if my cluster has currently 3 nodes and if I down 1 node or two nodes then EBS volume persists. But when I down all 3 nodes under ASG then EBS volume gets deleted. Need to know the root cause of it . Thanks in advance.
I tried by checking different scenario like terminating one EC2 /two EC2 instances , but EBS volume still there. Checked PV reclaim policy was mentioned as Delete, but PV lifecycle doesn't depend on Nodes termination so I ignored this logic.
The above scenario is happening when I scale up EKS cluster after 18 to 24 hours later from scale down process . If I scale up after 1 or 2 hour or so from scale down process , then EBS volume is not getting deleted.
This should not be the case, I myself downscale our worker nodes to 0 many times and never have the PVs got deleted, can you check the
Volumesin AWS EC2 dashboard and see if they are actually getting deleted from there or not?