Can't Read XML with Service Principal

7 Views Asked by At

I have the following spark configuration in my Databricks Cluster.

    fs.azure.account.oauth2.client.secret {{secrets/YYY/ZZZ}}
    fs.azure.account.auth.type OAuth
    spark.databricks.delta.preview.enabled true
    fs.azure.account.oauth2.client.endpoint https://login.microsoftonline.com/YY/oauth2/token
    fs.azure.account.oauth.provider.type org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider
    fs.azure.account.oauth2.client.id BBB

Reading the XML file as text works:

spark.read.text('abfss://[email protected]/abc.xml')

Reading the XML file as xml fails with Failure to initialize configurationInvalid configuration value detected for fs.azure.account.key:

spark.read.format("xml").load("abfss://[email protected]/abc.xml")

How can I read a XML file without using an account key?

0

There are 0 best solutions below