I've just created a new Azure Databricks and Azure Data Factory services inside my subscription.
For ADF, I've also created a SystemAssigned (Managed Identity) via TerraForm. Then, I've added this managed identity to owners of Databricks workspace and I've also added the service principal to admins inside Databricks workspace (tried both via TerraForm and via SCIM).
When I try adding a Databricks linked service to Data Factory, I always receive the error:
<title>Error 403 User not authorized.</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /api/2.0/clusters/get. Reason:
<pre> User not authorized.</pre></p>
SCIM is saying that my application ID is in the admins group: SCIM response
What am I doing wrong?
If one attempts to set up a linked service to a Databricks workspace to, without the correct role assignment set up, it will fail.
To grant the correct role assignment:
The following diagram shows how to grant the “Contributor” role assignment via the Azure Portal.
This should resolve your issue.