Azure Data Factory Manged Identity connection to Databricks

1.8k Views Asked by At

I've just created a new Azure Databricks and Azure Data Factory services inside my subscription.

For ADF, I've also created a SystemAssigned (Managed Identity) via TerraForm. Then, I've added this managed identity to owners of Databricks workspace and I've also added the service principal to admins inside Databricks workspace (tried both via TerraForm and via SCIM).

When I try adding a Databricks linked service to Data Factory, I always receive the error:

<title>Error 403 User not authorized.</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /api/2.0/clusters/get. Reason:
<pre> User not authorized.</pre></p>

SCIM is saying that my application ID is in the admins group: SCIM response

What am I doing wrong?

1

There are 1 best solutions below

1
On

If one attempts to set up a linked service to a Databricks workspace to, without the correct role assignment set up, it will fail.

To grant the correct role assignment:

  • Grant the contributor role to the managed identity.
  • The managed identity in this instance will be the name of the Data Factory that the Databricks linked service will be created on.

The following diagram shows how to grant the “Contributor” role assignment via the Azure Portal.

enter image description here

This should resolve your issue.