I'm working on bringing BigQuery datasets and tables into Terraform using the "terraform import module" This is how the code looks like:
main.tf:
module "bigquery" {
source = "./modules/bigquery"
for_each = var.dataset_config
region = var.region
project_id = var.PROJECT_ID
bq_dataset = each.key
bq_tables_views = each.value.table_and_views
bq_tables = flatten([for k, v in each.value.table_and_views : v.table])
bigquery_labels = each.value.labels
dataset_config = var.dataset_config
env = var.ENV
view_path = "./environments/environment"
}
variables.tfvars:
dataset_config = {
"raw" = {
"table_and_views" = [],
"labels" = {}
},
"dataset1" = {
"table_and_views" = [
{ "table" = "t_tabel1", "table_schema" = "tables.table1.json", "partition_type" = "DAY", "partition_field": "ingestion_date", "views" = [], "view_inputs" = {} }
],
"labels" = {}
},
"my_dataset" = {
"table_and_views" = [
{ "table" = "t_tabel1", "table_schema" = "tables.table1.json", "partition_type" = "DAY", "partition_field": "ingestion_date", "views" = [], "view_inputs" = {} }
],
"labels" = {}
}
}
modules/bigquery/bigquery.tf
resource "google_bigquery_dataset" "dataset" {
dataset_id = var.bq_dataset
location = var.region
project = var.project_id
delete_contents_on_destroy=true
labels = merge(
var.bigquery_labels,
{
"env" = "dev"
}
)
}
I am using the following command to import the module:
terraform import -var-file="./environments/environment/variables.tfvars" google_bigquery_dataset.dataset prj-dev-2342/my_dataset
I encounter an issue where the logs display the message "ReferenceTransformer: reference not found: 'each.value'," and consequently, the import operation does not import any resources. It appears that the variables have not been properly initialized. However, when I execute Terraform normally, it successfully creates all the required resources. The problem only arises when attempting to perform an import operation, where the actual values of these variables cannot be retrieved.
I have a lot of tables and datasets so I am wondering if there any other approach to import the module with respect to my current terraform structure?
Thank you in advance