How to generate a databricks privileged token which is valid more than 48 hours

Question

Would like to run a local Jupiter notebook connecting to Azure databricks cluster, and need to use dbutils to get secrets. This requires to save a privileged token locally and it is only lasting for 2 days. Is there any way to generate token longer than that or to keep using dbutils locally longer?

Answer 1

I suppose you followed this tutorial to make Jupyther work with Azure Databricks via Databricks Connect.

And no, as it says here, there is no

way to generate token longer than that or to keep using dbutils locally longer.

A token expires after 48 hours.

Answer 2

Note: Due to security restrictions, calling dbutils.secrets.get requires obtaining a privileged authorization token from your workspace. This is different from your REST API token, and starts with dkea.... The first time you run dbutils.secrets.get, you are prompted with instructions on how to obtain a privileged token. You set the token with dbutils.secrets.setToken(token), and it remains valid for 48 hours.

There are two types of databricks secrets:

• Databricks-backed scopes

• Azure Key Vault-backed scopes

This is possible by configuring secrets with Azure Key vault.

To reference secrets stored in an Azure Key Vault, you can create a secret scope backed by Azure Key Vault. You can then leverage all of the secrets in the corresponding Key Vault instance from that secret scope. B

Reference: https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#akv-ss

https://learn.microsoft.com/en-us/azure/databricks/security/secrets/example-secret-workflow

Hope this helps.