Best way to generate a list of non-Terraform created resources

Question

For a couple of days I am working on a way to generate a list of non-terraform created resources. Currently I am using the tagging method but this means that AWS resources that can't be tagged should also be excluded from the AWS Config query.

The tagging part is also quite cumbersome as this must be done manually.

1. Is there a way to tell Terraform (or Terragrunt) to automatically use general tags on resources?
2. Is there a way to ask Terraform or any other tool to create a list of resources that are not available in the state files?

Answer 1

for people still stumbling upon this. It seems that a community has created a tool named Driftctl that would check environments against your Terraform state.

Answer 2

1. Is there a way to tell Terraform (or Terragrunt) to automatically use general tags on resources?

No, this is not possible. Arguably, it would go against the declarative of Terraform's conceptual model, so this would not likely be made a feature of Terraform.

Terraform Enterprise has policy enforcement which could be used to compel users to provide specific tags, but I don't think it could automatically inject tags.

2. Is there a way to ask Terraform or any other tool to create a list of resources that are not available in the state files?

No. Terraform cares about the resources in its state file. It does not care about anything else.

---

Personally I think you are approaching this problem from the wrong angle. Asking Terraform "what do you not know about" would be convenient if it were possible (which it is not) but you would get a better answer if you consult a list of what it *does know about and infer from that whether a resource is or is not part of Terraform.

I don't know the tools you are using or the workflow you are hoping for, but you could do something like this:

• Use the terraform show -json command to generate a JSON document representing all of your known infrastructure in the Terraform state file.

• Use the tool jq to parse the JSON for a list of resource IDs

• Use the aws resourcegroupstaggingapi get-resources command to generate a list of all known resources, or perhaps even a tool like aws-list-all

• Loop through each entry in the 'all resources' list; for each one, loop through each entry in the 'Terraform-managed' list; if there's no match, add the resource ID to a list of 'not-managed-by-Terraform' resources

Answer 3

I agree with Chuppa Chump's answer & will add if it's not already obvious, to ensure all Terraform created resources have a tag indicating it was created by Terraform. For example, CreatedBy = "Terraform" OR Automation = "Terraform". And then any resource that doesn't have the Automation or CreatedBy Tag, was not created by terraform.

Answer 4

Since there is no good way to know if a resource is managed by terraform or not, I would try the following approach:

Use terraformer to import all your resources (not all resources are supported by terraformer, so this is not a complete solution), and after you have this list just substract all the resources that are managed by your terraform states from it. When running terraform plan you can see your resources ids.

If you have tags for all your terraform managed resources you can filter them with terraformer