as stated in the title I have a general question about securely retrieving secrets from Azure KeyVault.
Currently most of my secrets are stored in Kubernetes Secrets. As Secrets are only base64 encoded it is possible for a malicious user or even admins to read the secret using tools like lens.
To enhance the security of my application(s) I would like to use a more secure storage like Azure KeyVault to hold my secrets, certificates and alike.
Currently I am thinking about two available versions
- using CSI Driver
- querying data from KV using a REST call on container start and keeping data only in memory (my favorite)
Somehow I do not trust the approach using a CSI driver for following reasons:
If, for any reason, some malicious user is able to connect to my container, he/she/it will be able to read the values from the "file" generated by the CSI driver.
This should be quite trivial to do so assuming one connect into the container.
Altough having a malicious package installed in my app bundle, it might be possible to read those data...
On the other hand, having secrets only in memory seems to me a bit more secure as the attacker must have the chance to read the memory (I hope there is something in place to prevent such actions) as well as one has to know the exact memory location where to find the data.
Any advice, ideas or clarification is highly appreciated.
CSI together with KeyVault is a clear security upgrade to the solution you are running now. We use it all the time with our customers. It is a well adopted approach and i think it is also the k8s default for secrets now.
If you follow along the container best practices you should be good to go. There are also good steps like no shell inside the container (scratch) and shift left security to the build phase (Code scanning, Image scanning, build your own images).
Another good step is to use runtime security like Falco.
You can add another security layer by additionally restrict access to the AKS API to certain IPs or better use a private cluster. This also goes hand in hand with the KeyVault, make it private and restrict the access to it.
I think there is no 100% safe solution, but all the the things i mentioned reduce the attack scenario drastically.