Monitoring TCP endpoint using BlackBox exporter

Question

I am trying to probe my LDAP server using the blackbox exporter using the tcp_connect module. The source is my k8s cluster. From within my cluster, I can definitely reach the ldap server

debug@debug:~$ telnet global.ldap 636
Trying 10.27.20.7...
Connected to global.ldap.
Escape character is '^]'.
^]

My Prometheus config looks like

- job_name: 'ldap_check'
          scrape_interval: 5m
          metrics_path: /probe
          params:
            module: [tcp_connect]
          static_configs:
            - targets:
              - 'ldaps://global.ldap:636' # Also tried 'global.ldap:636' without the 'ldaps://'
          relabel_configs:
            - source_labels: [__address__]
              target_label: __param_target
            - source_labels: [__param_target]
              target_label: instance
            - target_label: __address__
              replacement: monitoring-prometheus-blackbox-exporter:9115

But my Prometheus /targets page shows me "server returned HTTP status 400 Bad Request" Is there something I have missed ?

Answer 1

Try this:

- job_name: ldap_check
  metrics_path: /probe
  static_configs:
    - labels:
        module: tcp_connect
      targets:
        - global.ldap:636
  relabel_configs:
  ...

Answer 2

Use TCP module for that.

tcp_connect:
    prober: tcp
    timeout: 5s

like this. telnet 10.10.10.1 host for 123 port.

curl 'http://localhost:9115/probe?target=10.10.10.1:123&module=tcp_connect&debut=true'

This post cover that in detail. How to setup Prometheus BlackBox exporter

Answer 3

The most likely reason of 400 Bad Request with blackbox exporter is the absence of the requested module (tcp_connect in this case). Since tcp_connect is one of default modules, I suppose somebody modified blackbox.yml configuration and forgot to include the default modules in it.

There is a way to run a debug query and find out for sure what went wrong. Just curl http://blackbox_exporter_ip:9115/probe with the same URL parameters Prometheus does plus debug=true. For example:

❯ curl 'http://localhost:9115/probe?debug=true&module=tcp_connect2&target=localhost:9115' -v
...
< HTTP/1.1 400 Bad Request
...
Unknown module "tcp_connect2"