GCP MQL: Calculate ratio from 3 metrics

Question

I have 3 metrics created in my GCP 1) success request count 2) redirected request count and 3) failed request count. All the 3 metrics are created from log-based metric on Kubernetes log.

Here's basically a Terraform resource definition of one metric "Success requests.

resource "google_logging_metric" "success_requests" {
  filter      = <<-EOT
    resource.type="k8s_container"
    jsonPayload.message:"Request success"
  EOT
  name        = "success_requests"
  project     = var.gcp_project_id

  metric_descriptor {
    metric_kind = "DELTA"
    unit        = "1"
    value_type  = "INT64"
  }
}

The other 2 basically just filter jsonPayload.message with Request redirected and Request failed.

I want to create a dashboard that shows the success ratio with this formula

success_rate = success_requests / (success_requests + redirected_requests + failed_requests)

I am successful creating a dashboard that shows the three metrics using below MQL.

k8s_container | { 
    t_0:
        metric logging.googleapis.com/user/success_requests;
    t_1: 
        metric logging.googleapis.com/user/redirected_requests;
    t_2: 
        metric logging.googleapis.com/user/failed_requests
} | union

But how do I create a dashboard that display the success rate with formula as mentioned before?

Answer 1

To perform some arithmetic operations using metrics, you need to work with the time series using Arithmetic Computation

To sum two time series, configure your query to fetch two tables of time series, join those results, and then call the add function. The following example illustrates a query that computes the sum of the number of bytes read from, and written to, Compute Engine instances:

  fetch gce_instance
  | { metric 'compute.googleapis.com/instance/disk/read_bytes_count'
    ; metric 'compute.googleapis.com/instance/disk/write_bytes_count' }
  | outer_join 0
  | add

After you summarize the three time series you can use div function.

The following query uses time_shift, join, and div to compute the ratio of the mean utilization in each zone between now and one week ago.

fetch gce_instance::compute.googleapis.com/instance/cpu/utilization
| group_by [zone], mean(val())
| {
    ident
  ;
    time_shift 1w
  }
| join | div

You can review more examples of queries using MQL in this Link.

You can review more of Functions implementing the arithmetic operators in MQL in this link.

Answer 2

I believe what you need is an outer_join and not union. Note that MQL's outer_join only supports joining two metrics. But you can use two outer_join operations to join three tables as shown below.

After joining the metrics, you can simply refer to them by names like success_requests, failed_requests, etc.

Note that outer_join 0,0 performs a full outer join on two tables and substitutes a 0 if one of the input streams doesn't have a value at a given timestamp.

Example:

{
  {k8s_container :: logging.googleapis.com/user/success_requests ;
   k8s_container :: logging.googleapis.com/user/redirected_requests}
  | outer_join 0,0 ;
  k8s_container :: logging.googleapis.com/user/failed_requests
} 
| outer_join 0,0
| value [success_rate: success_requests / (success_requests + redirected_requests + failed_requests)]