Authenticating on AKS for deploying a Helm release with Terraform

Question

I am trying to do a Helm chart deployment through Terraform code on AKS. The TF code that I have will create a resource in Datadog from which I will grab an output value that will be passed to my Helm release to be deployed on my cluster. It only has to create two resources, one of which is the Helm chart.
The problem that I am having is with authentication against my Kubernetes cluster, I am using a data source to bring the credentials from the cluster and then pass them in my kubernetes and helm providers.

My Terraform state for the AKS cluster is stored inside a Blob in a Azure Storage account.

I have tried updating the Helm chart versions, using different methods to access the data such as ${} around my variables. Tried changing from username = data.azurerm_kubernetes_cluster.credentials.kube_config.0.username to use the admin configuration username = data.azurerm_kubernetes_cluster.credentials.kube_admin_config.0.username Tried

Terraform version: 1.1.7

A data source is setup to bring the credentials for the AKS cluster in main.tf

data "azurerm_kubernetes_cluster" "credentials" {
  name                = var.aks_cluster_name
  resource_group_name = var.aks_cluster_resource_group_name
}

This is versions.tf and what is being used to setup the connections to AKS.

terraform {
  required_providers {
    datadog = {
      source = "DataDog/datadog"
    }
  }
  backend "azurerm" {
  }
}

provider "azurerm" {
  features {}
}

provider "helm" {
  debug = true
  kubernetes {
    username               = data.azurerm_kubernetes_cluster.credentials.kube_config.0.username
    password               = data.azurerm_kubernetes_cluster.credentials.kube_config.0.password
    host                   = data.azurerm_kubernetes_cluster.credentials.kube_config.0.host
    client_certificate     = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.client_certificate)
    client_key             = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.client_key)
    cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.cluster_ca_certificate)
  }
}

provider "kubernetes" {
  username               = data.azurerm_kubernetes_cluster.credentials.kube_config.0.username
  password               = data.azurerm_kubernetes_cluster.credentials.kube_config.0.password
  host                   = data.azurerm_kubernetes_cluster.credentials.kube_config.0.host
  client_certificate     = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.client_certificate)
  client_key             = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.client_key)
  cluster_ca_certificate = base64decode(data.azurerm_kubernetes_cluster.credentials.kube_config.0.cluster_ca_certificate)
}

Error that I am seeing when running terraform apply, which will report that it can't find the elements in the collection for any of the attributes specified in my provider:

╷
│ Error: Invalid index
│ 
│   on versions.tf line 26, in provider "helm":
│   26:     host                   = data.azurerm_kubernetes_cluster.credentials.kube_admin_config.0.host
│     ├────────────────
│     │ data.azurerm_kubernetes_cluster.credentials.kube_admin_config has a sensitive value
│ 
│ The given key does not identify an element in this collection value.
╵
[ ... ]
╷
│ Error: Invalid index
│ 
│   on versions.tf line 27, in provider "helm":
│   27:     username               = data.azurerm_kubernetes_cluster.credentials.kube_admin_config.0.username
│     ├────────────────
│     │ data.azurerm_kubernetes_cluster.credentials.kube_admin_config has a sensitive value
│ 
│ The given key does not identify an element in this collection value.

I am unsure on how to change my Terraform code such that this authentication works, given that the methods mentioned above have yielded no results. If needed I can provide the TF code for the deployment of the resources.

Answer 1

I'm using kubelogin to identify myself:

data "azurerm_client_config" "current" {
}


provider "helm" {
  kubernetes {
    host = azurerm_kubernetes_cluster.aks.kube_config.0.host
    cluster_ca_certificate = base64decode(azurerm_kubernetes_cluster.aks.kube_config.0.cluster_ca_certificate)

    exec {
      api_version = "client.authentication.k8s.io/v1beta1"
      args = [
        "get-token",
        "--environment", "AzurePublicCloud",
        "--server-id", "6dae42f8-4368-4678-94ff-3960e28e3630", # The AAD server app ID of AKS Managed AAD is always 6dae42f8-4368-4678-94ff-3960e28e3630 in any environments.
        "--client-id", "${yamldecode(azurerm_kubernetes_cluster.aks.kube_config_raw).users[0].user.auth-provider.config.client-id}", 
        "--tenant-id", data.azurerm_client_config.current.tenant_id,
        "--login", "devicecode"
      ]
      command = "kubelogin"
    }
  }
}