Call Databricks API from an ASP.NET Core web application

Question

Our application is a web application which is registered in Entra Id. The applications uses OIDC for authentication. We have added Databricks scope (user_impersonation) in API permissions in Entra. The Databricks scope (<databrics_app_id>/.default) is added as scope to OpenIdConnect client.

When the user logs in, the access token returned by Entra doesn't have the user_impersonation scope. When the application makes the call to Databricks using the access token, we get an http 401 UnAuthorized error.

I am using OidcClient to login using OIDC:

var options = new OidcClientOptions()
{
    Authority = "https://login.microsoftonline.com/<tenant_id>",
    ClientId = "<client_id>,
    ClientSecret = "<client_secret>",
    Scope = "<databricks_id>/.default openid",
    Browser = new WpfEmbeddedBrowser(),
    RedirectUri = "https://localhost:<port_no>/callback",
}

options.Policy.Discovery.ValidateEndpoints = false;
options.Policy.Discovery.ValidateIssuerName = false;
var _oidcClient = new OidcClient(options);
var loginResult = await _oidcClient.LoginAsync();

I also tried

services.AddAuthentication(options =>
{
    options.DefaultScheme = "Cookies";
        options.DefaultChallengeScheme = "oidc";
        
})
.AddOpenIdConnect("oidc", options =>
{
    options.BackchannelHttpHandler = new HttpClientHandler
    {
        ServerCertificateCustomValidationCallback = (message, cert, chain, errors) => true
    };

    options.Authority = "https://login.microsoftonline.com/<tenant_id>";

    options.Scope.Add("openid");   
    options.Scope.Add("<databricks_id>/.default");
    //options.Scope.Add("<databricks_id>/user_impersonation");
    options.CallbackPath = "/callback";
    options.ResponseType = "code";
    options.SaveToken = true;
    options.ClientId = "<client_id>";
    options.ClientSecret = "<client_secret>";
}