Terraform workspace or module for S3 assets?

Question

I want to deploy a series of AWS S3 buckets with Terraform. Each bucket has different contents and a different name, but they are otherwise configured the same.

The buckets are created with the aws_s3_bucket resource and are filled via the aws_s3_object resource. The contents of each bucket are sourced from a local directory with the same name as the bucket. I am using no AWS compute resources, only storage.

I can create and fill a single bucket like this:

resource "aws_s3_bucket" "bucket" {
  bucket        = "sample-bucket-name"
}

# put stuff into the bucket from a directory of the same name
resource "aws_s3_object" "textfiles" {
  for_each     = fileset(path.module, "sample-bucket-name/**/*.txt")
  bucket       = aws_s3_bucket.bucket.id
  key          = replace(each.value, "/^sample-bucket-name//", "")
  source       = each.value
  content_type = lookup(local.content_types, regex("\\.[^.]+$", each.value), null)
  etag         = filemd5(each.value)
}

How do I do this for more than one bucket? The naive approach would be to copy this HCL code, change the bucket name, and re-run it for each bucket. But, I want my code to be DRYer than that.

As I understand it (from this blog post) there are two main approaches to DRYing up this type of code: modules and workspaces.

• Workspaces, it looks like, keep all terraform code in a single directory. With this approach I would keep a single bunch of terraform code in a single place, which makes some sense to me because all my buckets are configured the same. I assume the bucket name would be interpolated from a variable.

• Modules, on the other hand, would use number of directories, one for each bucket. I guess each would contain a variables.tf file. This approach is popular on the Terraform subreddit and elsewhere. But I am not sure I understand the advantages.

Which approach is appropriate for my use case?

A lot of the advice I can find about this online is for people who are deploying different environments: dev, staging, prod, etc., and who have somewhat different configurations (like, different numbers of EC2 instances) in each environment. This isn't my situation. I want each of my modules or workspaces to have the same configuration except for the S3 bucket name and contents.

How do I make it DRY? What variables will I use? What directory structure should I use?

Answer 1

Use of Terraform module is the best for your use case. You do not need to create different directories for using modules, I am not sure why you think you would require different directories. Below is an example of using terraform module for creating multiple s3 buckets with same configuration.

module "s3_bucket" {   
    for_each = toset(["assets", "media"])
    source = "terraform-aws-modules/s3-bucket/aws"
    bucket     = "${each.key}_bucket"
    acl    = "private"
    control_object_ownership = true
    object_ownership         = "ObjectWriter"

    versioning = {
      enabled = true
    }
}

You can use a variable with list of bucket names instead of the static list that is used above.

For populating files, you can repeat the aws_s3_object block for each bucket. But I think there could be a way to do a nested loop to remove that repetition. I am not able to think of what would be the ideal way as of now for this nested loop.