Pull nuclio metrics into prometheus-operator

Question

I am currently trying to pull metrics from nuclio functions to my prometheus operator. But I don't realy understand how I am supposed to do.

So far, I have 2 namespaces :

• monitoring : for my kube-prometheus-stack
• nuclio : for everything related to nuclio (controller, dashboard and functions)

In the nuclio namespace I configured a platform based on the Nuclio Documentation .

My configmap (nuclio-platform-config) :

cronTriggerCreationMode: "kube"
logger: 
  sinks:
    stdout:
      kind: stdout
  system:
  - level : debug
    sink: stdout
  functions:
  - level : debug
    sink: stdout
metrics:
    sinks:
      PromPull:
        kind: prometheusPull
        url: 8090
        attributes:
          jobName: nuclio-pull-job
          instanceName: nuclio-pull-instance
    system:
    - PromPull
    functions:
    - PromPull

As expected, it did not scrape automaticaly my functions' metrics. But I have absolutely no idea what to do next...

If anyone could give me some guidance, I'd be delighted.

PS : I didn't know which config was relevant or not. If you need anything, I will edit my question to add it

Answer 1

To scrape metrics from Nuclio functions and make them available to Prometheus, you'll need to set up a custom Prometheus scrape configuration for the Nuclio functions in your monitoring namespace. This involves defining a new ServiceMonitor resource that specifies the target endpoints to scrape.

Here's how you can proceed:

1.Create a ServiceMonitor resource: You need to define a ServiceMonitor resource in the monitoring namespace to tell Prometheus what to scrape. This resource should include the labels that match your Nuclio functions' services.

For example, create a file named nuclio-function-monitor.yaml with the following content:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: nuclio-function-monitor
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app.kubernetes.io/instance: nuclio
  endpoints:
    - port: http-metrics

In this example, the selector field selects all services with the label app.kubernetes.io/instance: nuclio (you might need to adjust this label depending on how your Nuclio functions are labeled). The endpoints field specifies that Prometheus should scrape the http-metrics port of these services.

2.Apply the ServiceMonitor resource: Apply the nuclio-function-monitor.yaml file to create the ServiceMonitor resource in the monitoring namespace:

kubectl apply -f nuclio-function-monitor.yaml

3.Verify: Ensure that the ServiceMonitor is created successfully:

kubectl get servicemonitor -n monitoring

You should see the nuclio-function-monitor resource in the output.

4.Configure Nuclio to expose metrics: You also need to ensure that your Nuclio functions are configured to expose metrics on the http-metrics endpoint. It looks like you've already done this in your nuclio-platform-config ConfigMap.

5.Access the metrics: Now, Prometheus should automatically start scraping metrics from the Nuclio functions exposed on the http-metrics endpoint. You can check if Prometheus is scraping the targets by navigating to the Prometheus web UI (if you've installed Prometheus with kube-prometheus-stack) and checking the "Targets" page. You should see the Nuclio function endpoints listed there.