Get failed service name inside Consul watch handler

Question

I'm using Consul to monitor services health status. I use Consul watch command to fire a handler when some service is failed. Currently I'm using this command:

consul watch -type=checks -state=passing /home/consul/health.sh

This works, however I'd like to know inside health.sh the name of the failed service, so I could send a proper alert message containing failed service name. How can I get failed service name there?

Answer 1

Your script could get all the required information by reading from stdin. Information will be sent as JSON. You can easily examine those events by simply adding cat - | jq . into your handler.

Answer 2

The check information outputted by consul watch -type=check contains a ServiceName field that contains the name of the service the check is associated with.

[
  {
    "Node": "foobar",
    "CheckID": "service:redis",
    "Name": "Service 'redis' check",
    "Status": "passing",
    "Notes": "",
    "Output": "",
    "ServiceID": "redis",
    "ServiceName": "redis"
  }
]

(See https://www.consul.io/docs/dynamic-app-config/watches#checks for official docs.)

Checks associated with services should have values in both the ServiceID and ServiceName fields. These fields will be empty for node level checks.

The following command watches changes in health checks, and outputs the name of a service when its check transitions to a state other than "passing" (i.e., warning or critical).

$ consul watch -type=checks -state=passing "jq --raw-output '.[] | select(.ServiceName!=\"\" and .Status!=\"passing\") | .ServiceName'"