We have a worker process running in IIS (Windows Server 2019) that makes an https request to a kafka resource in AWS in the background. In order for this process to work, we have to import certificates from Amazon into the local computer cert store so that the ssl connection can be established and it will trust the certificate presented from the resource in AWS. We have imported the cert chain and gotten this to work with the following caveats: The service account specified for the AppPool identity has to be a local admin OR We have to run some process, ANY process (other than w3wp.exe that's already running), as the service account.
I've dealt quite a bit with certs, permissions, etc, but what is really odd about this one is that if I open notepad, for example, as the service account, then as long as notepad is running the process works - meaning it reads the cert and is able to establish the ssl connection. If I close notepad, the process then fails, and I get the following error:
ssl://b-2.kafkamsk.abc123.c2.kafka.us-west-2.amazonaws.com:9094/bootstrap: SSL handshake failed: error:14090086:SSL routines:ssl3_get_server_certificate:certificate verify failed: broker certificate could not be verified, verify that ssl.ca.location is correctly configured or root CA certificates are installed (add broker's CA certificate to the Windows Root certificate store) (after 72ms in state CONNECT)
(text in the URL replaced with abc123 for obfuscation)
Note that we don't need to access the private key of the cert, we simply need to read the cert from the cert store. The CA cert has been imported into "Trusted Root Certification Authorities." The intermediate cert has been imported into "Intermediate Certification Authorities" and the leaf cert has been imported into the personal store - all of these in the Local Computer cert store. And again, it all works if we simply login and run some process as the service account, even without it being a local admin.
I've run sysinternals procmon while executing the process that fails, and see several access denied attempts on HKEY_USERS.DEFAULT\Software\Microsoft\SystemCertificates, and those do not appear if I have notepad running as the service account when attempting the process. I've given explicit permissions to the service account on those registry keys, but it has not helped.
For obvious security reasons, we don't want our service account to be a local admin on the box, and for operations/administration reasons we don't want to rely on having some other process running as this account just to make this work.
It probably works when notepad is run as its triggering the user profile to be loaded, terminating notepad causes the profile to be unloaded. If you were to set
LoadUserProfileto true on the app pool it would probably work - BUT this is not a good idea.I think the proper solution is related to the permissions for
IIS AppPool\AppPoolNameon the certificate itself (which boils down to a file on disk with an ACL somewhere). Grant the app pool rights via the Certificates MMC or powershell and it should work.