Databricks CLI: SSLError, can't find local issuer certificate

Question

I have installed and configured the Databricks CLI, but when I try using it I get an error indicating that it can't find a local issuer certificate:

$ dbfs ls dbfs:/databricks/cluster_init/
Error: SSLError: HTTPSConnectionPool(host='dbc-12345678-1234.cloud.databricks.com', port=443): Max retries exceeded with url: /api/2.0/dbfs/list?path=dbfs%3A%2Fda
tabricks%2Fcluster_init%2F (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer
 certificate (_ssl.c:1123)')))

Does the above error indicate that I need to install a certificate, or somehow configure my environment so that it knows how to find the correct certificate?

My environment is Windows 10 with WSL (Ubuntu 20.04) (the command above is from WSL/Ubuntu command line).

The Databricks CLI was installed into an Anaconda environment including the following certificates and SSL packages:

$ conda list | grep cert
ca-certificates           2020.6.20            hecda079_0    conda-forge
certifi                   2020.6.20        py38h32f6830_0    conda-forge
$ conda list | grep ssl
openssl                   1.1.1g               h516909a_1    conda-forge
pyopenssl                 19.1.0                     py_1    conda-forge

I get a similar error when I attept to use the REST API with curl:

$ curl -n -X GET https://dbc-12345678-1234.cloud.databricks.com/api/2.0/clusters/list
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Answer 1

This problem can be solved by disabling the SSL certificate verification. In Databricks CLI you can do so by specifying insecure = True in your Databricks configuration file .databrickscfg.

Answer 2

I established trust to my Databricks instance by setting the environment variable REQUESTS_CA_BUNDLE.

➜ databricks workspace list
Error: SSLError: HTTPSConnectionPool(host='HOSTNAME.azuredatabricks.net', port=443): Max retries exceeded with url: /api/2.0/workspace/list?path=%2F (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)')))
➜ export REQUESTS_CA_BUNDLE=path/to/ca-bundle
➜ databricks workspace list
Users
Shared
Repos

From GitHub Issue:

Download the root CA certificate used to sign the Databricks certificate. Determine the path to the CA bundle and set the environment variable REQUESTS_CA_BUNDLE. See SSL Cert Verification for more information.

Answer 3

There is a similar issue in GitHub for Azure CLI. The solution is practically the same. Combining that with the Erik's answer:

1. Download the certificate using your browser and save it to disk

   • Open you Chrome and go to the Databricks website
   • Press CTRL + SHIFT + I to open the dev tools
   • Click Security tab
   • Click View certificate button
   • Click Details tab
   • On the Certification Hierarchy, (the top panel), click the highest node in the tree
   • Click Export the selected certificate
   • Choose where you want to save (eg. /home/cert/certificate.crt)

2. Use the SET command on Windows or the export on Linux to create a env variable called REQUESTS_CA_BUNDLE and point it to the downloaded file in the Step 1. (keep in mind that this need to be done in the same machine as you are trying to use the dbfs not in the cluster) For instance:

   Linux

   export REQUESTS_CA_BUNDLE=/home/cert/certificate.crt
   

   Windows

   set REQUESTS_CA_BUNDLE=c:\temp\cert\certificate.crt
   

3. Try to run your command dbfs ls dbfs:/databricks/cluster_init/ again

   $ dbfs ls dbfs:/databricks/cluster_init/
   

It should work!