I have two "twin applications" that requires monitoring. There is a way to create a multi alert monitor to them via Terraform? It can be done via interface but I need to migrate them to Terraform.
Being more specific, I have this query, in my monitor created using the Datadog interface, to check when the latency of an action is too high:
logs("service:(service-a OR service-b) "Action finished"").index("*").rollup("pc99", "@latency").by("@service").last("15m") > 90000
And it automatically separates alerts to "service-a" and "service-b".
If i can't create a multi alert like this, can I use something like a "for" loop to not need to create two almost identical monitors in terraform? I will have to do this to a lot of monitors and didn't want to duplicate everything.
Another requirement is to set customizable tags to each service.
I searched in the Datadog provider for a resource to do what I want but didn't find anything to multi alerts. I expect to not have to write two monitors changing only the name of the service. I also have others multi alerts monitors to a lot of controllers of other applications and wold be terrible have 4+ of the same terraform code.
Datadog Terraform actually handles alerting separate occurences based on the query given. In the GUI, they call this a Multi Alert, alerting for each separate instance.
For your example, the beginning query would look something like this:
Another thing I found while carrying Datadog infrastructure over to Terraform is that the API guide is much more useful than the Terraform docs. Most of the content written within Terraform is just in API format. Here are the docs I referenced to answer this: https://docs.datadoghq.com/api/latest/monitors/#create-a-monitor