I have a metric which sends data only in case of a specific event. In other words, it will not have regular continuous telemetry every 15 seconds, but it will be there for a single data point. Now, I configured an alert to check if the metric exists (did not include any 'for' condition in the rule as there is no need to wait), an alert should trigger.
What is happening is that when the metric becomes available in Prometheus, the rule gets activated to yellow in the prometheus UI, but instead of firing it automatically goes away within 15 seconds. Is there some additional setting I have to do to alert on event based metric which lasts for only 1 single data point or 15s ?
Thanks, Arnav
I used count_over_time for 1m and that worked out. Even though the data was there for only 15s, the query made it last for 1m and the alert triggered correctly.