We have been utilizing Grafana with a specific alerting configuration, based on Prometheus query language (PromQL). Now, we are transitioning to Stackdriver, Google Cloud Platform's native monitoring solution, which utilizes Metric Query Language (MQL) instead of PromQL. Consequently, we need to convert the existing Grafana alert query to MQL.
Below is the converted MQL query, which I believe is the appropriate conversion:
{
fetch k8s_container::kubernetes.io/container/memory/used_bytes
| filter metric.memory_type == 'non-evictable'
| group_by 5m, [value_used_bytes_mean: mean(value.used_bytes)]
| every 5m
;
fetch k8s_container::kubernetes.io/container/memory/limit_bytes
}
| filter
(resource.cluster_name == 'k8s-cluster'
&& resource.namespace_name == 'test1'
&& resource.pod_name =~ '^api.*' )
| join | div | mul 100
| condition val() > cast_units(75, '%')
While the converted query is functional in generating alerts, we are encountering an issue when attempting to save the alerting policy in Stackdriver, as an error "Error: Unable to save alerting policy. Request contains an invalid argument." appears.
To resolve the error, we need to investigate the root cause further. The error may stem from certain invalid characters in the query, unsupported aggregations, or other factors. It is vital to carefully review the query for syntax correctness, check for any inappropriate characters or symbols, and ensure compatibility with Stackdriver's supported functionalities
I have tried the multiple combinations for conditions, because it seems that the problem is occuring in conditional statement,
condition val() > cast_units(75, '%')
condition val() > 75 '%'
condition val() > 75 'By'
something like this and many more.