Call to function "file" failed: no file exists - Terraform

Question

I have a template_file section in my terraform code, which has a variable value to be picked from a file like below

data "template_file" "post_sql"{
    template = "${file("/home/user/setup_template.yaml")}"
    vars = {
      name1= data.azurerm_storage_account.newazure_storage_data.name           
      name2="${file("/home/user/${var.newname}loca.txt")}"
    }    
}

This file will get generated in the middle of tasks, but terraform looks for it at the starting of apply stage itself. I have even tried adding depends_on to no avail and throws the below error

Call to function "file" failed: no file exists at
/home/user/newnamerloca.txt.

How can i make this work, any help on this would be appreciated

Answer 1

Try "cat /home/user/newnamerloca.txt" and see if this file is actually in there.

Edit: Currently there is no workaround this, "data" resources are applied at the start of plan/apply thus need to be present in order to use them

Data resources have the same dependency resolution behavior as defined for managed resources. Setting the depends_on meta-argument within data blocks defers reading of the data source until after all changes to the dependencies have been applied.

NOTE: In Terraform 0.12 and earlier, due to the data resource behavior of deferring the read until the apply phase when depending on values that are not yet known, using depends_on with data resources will force the read to always be deferred to the apply phase, and therefore a configuration that uses depends_on with a data resource can never converge. Due to this behavior, we do not recommend using depends_on with data resources.

So maybe something like:

data "template_file" "post_sql"{
    template = "${file("/home/user/setup_template.yaml")}"
    vars = {
      name1= data.azurerm_storage_account.newazure_storage_data.name           
      name2="${file("/home/user/${var.newname}loca.txt")}"
    }    
    depends_on = [null_resource.example1]
}

resource "null_resource" "example1" { # **create the file here**
  provisioner "local-exec" {
  command = "open WFH, '>completed.txt' and print WFH scalarlocaltime"
  interpreter = ["perl", "-e"]
  }
}

Answer 2

The reason for the behavior you are seeing is included in the documentation for the file function:

This function can be used only with files that already exist on disk at the beginning of a Terraform run. Functions do not participate in the dependency graph, so this function cannot be used with files that are generated dynamically during a Terraform operation. We do not recommend using dynamic local files in Terraform configurations, but in rare situations where this is necessary you can use the local_file data source to read files while respecting resource dependencies.

The file function is intended for reading files that are included on disk as part of the configuration, typically in the same directory as the .tf file that refers to them, and using the path.module symbol to specify the path like this:

file("${path.module}/example.tmpl")

Your question doesn't explain why you are reading files from a user's home directory rather than from the current module configuration directory, or why one of the files doesn't exist before you run Terraform, so it's hard to give a specific suggestion on how to proceed. The documentation offers the local_file data source as a possible alternative, but it may not be the best approach depending on your goals. In particular, reading files on local disk from outside of the current module is often indicative of using Terraform for something outside of its intended scope, and so it may be most appropriate to use a different tool altogether.