Unable to upload the local folder containing sub-folders and files to the blob storage in azure cloud using terraform

Question

I have been trying to create a terraform file that will create a resource group, storage account, and blob storage container and then upload a local folder containing sub-folders and files to the given blob storage container but I'm unsuccessful in doing so. I'm going to add the main.tf code as well as the error code which is being displayed. Please help me out.

//this informs the terraform to use azurerm provider
terraform {
  required_providers {
    azurerm = {
      source = "hashicorp/azurerm"
      version = "3.78.0"
    }
  }
}
// configuration option needs credential for the authentication
#which will execute the command
provider "azurerm" {
   subscription_id = "sub_id"
   client_id = "client_id"
   client_secret = "TxK8Q~uFOk03shH0tnzKfPBcqzaxEuhKT0hYuaRN"
   tenant_id = "79edd2b6-7c69-4515-be18-b5e923054c77"
  features {}
}
//resource block to create a resource group
resource "azurerm_resource_group" "app_terraform" {
  location = "North Europe"
  name     = "app-terraform"
}

resource "azurerm_storage_account" "storage_account" {
  name                     = "terraformdemotf"
  resource_group_name      = azurerm_resource_group.app_terraform.name
  location                 = azurerm_resource_group.app_terraform.location
  account_tier             = "Standard"
  account_replication_type = "LRS"

  tags = {
    environment = "staging"
  }
}
resource "azurerm_storage_container" "data1" {
  name                 = "data"
  storage_account_name = azurerm_storage_account.storage_account.name
  container_access_type = "blob"
}
resource "null_resource" "m06sparkbasics" {
  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "azcopy copy '${var.local_folder_path}' 'https://${azurerm_storage_account.storage_account.name}.blob.core.windows.net/${azurerm_storage_container.data1.name}/${var.blob_prefix}?sv=2022-11-02&ss=bfqt&srt=sco&sp=rwdlacupiytfx&se=2023-11-01T16:29:56Z&st=2023-11-01T08:29:56Z&spr=https&sig=Ysu7bIPcLVhf4tqUf%2BQSibAXDKTG%2B4zqrW8takl8sLA%3D' --recursive=true --from-to=LocalBlob"
  }
}

The error produced is -> [

Answer 1

I tried to upload the local folder containing sub-folders and files to the blob storage in azure cloud using terraform and I was able to provision the requirement successfully.

The Terraform code you've provided creates an Azure resource group, an Azure storage account, and a blob storage container, and then attempts to upload files to the blob container using a null_resource with a local-exec provisioner that invokes azcopy.

However, you need to take care while achieving the requirement the local-exec provisioner does not inherently provide detailed error outputs from azcopy. You would need to ensure that error handling is correctly managed within the script.

My terraform configuration:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.78.0"
    }
  }
}

provider "azurerm" {
  features {}
  
}

variable "blob_prefix" {
  description = "The prefix for the blob storage"
  default     = "demovk" # Set a default or make sure to provide this value when running Terraform
}

resource "azurerm_resource_group" "app_terraform" {
  name     = "appvk-terraform"
  location = "North Europe"
}

resource "azurerm_storage_account" "storage_account" {
  name                     = "terraformdemotfvkstest"
  resource_group_name      = azurerm_resource_group.app_terraform.name
  location                 = azurerm_resource_group.app_terraform.location
  account_tier             = "Standard"
  account_replication_type = "LRS"
  tags = {
    environment = "staging"
  }
}

resource "azurerm_storage_container" "data1" {
  name                  = "data"
  storage_account_name  = azurerm_storage_account.storage_account.name
  container_access_type = "blob"
}

resource "null_resource" "generate_sas" {
  depends_on = [azurerm_storage_container.data1]

  provisioner "local-exec" {
    command = "az storage container generate-sas --name ${azurerm_storage_container.data1.name} --account-name ${azurerm_storage_account.storage_account.name} --permissions rw --expiry `date -u -d '30 minutes' +%Y-%m-%dT%H:%MZ` --output tsv > ${path.module}/sas_token.txt"
    environment = {
      AZURE_STORAGE_ACCOUNT_KEY = azurerm_storage_account.storage_account.primary_access_key
    }
  }
}

resource "null_resource" "upload_files" {
  depends_on = [null_resource.generate_sas]

  provisioner "local-exec" {
    when    = create
    command = <<EOT
    SAS_TOKEN=$(cat ${path.module}/sas_token.txt)
    azcopy copy "/home/bolli/vinay/sample/November" "https://${azurerm_storage_account.storage_account.name}.blob.core.windows.net/${azurerm_storage_container.data1.name}/${var.blob_prefix}?$SAS_TOKEN" --recursive=true
    EOT
  }
}

Output: