Converting New Relic drop filters to OTel Collector log filters

50 Views Asked by At

I would like to replicate some behavior configured in New Relic in Grafana.

We have drop filters based on some logs that come in to remove overly verbose successful messages. Here are a couple:

(drop health checks)

SELECT * FROM Log, LogExtendedRecord
WHERE message LIKE '%ELB-HealthChecker%'
or message LIKE '%Health endpoint requested%'

and (drop 2xx responses)

SELECT * FROM Log
WHERE `message` LIKE '{"http_x_forwarded_for":%'
AND `message` LIKE '%"status": "20%'

We have the OTel collector handling logs before going to Loki and want to reduce the size of the data we are managing. I have these processors in the OTel config:

  attributes/remove_elb_health_checks:
    exclude:
      match_type: strict
      attributes:
       - key: '' #message content
         value: "ELB-HealthChecker"
       - key: '' #message content
         value: "Health endpoint requested"

  attributes/remove_2xx_responses:
    exclude:
      match_type: strict
      attributes:
       - key: '' #message content
         value: "{\"http_x_forwarded_for"
       - key: '' #message content
         value: "\"status\": \"20%"

I have these services:

services:
    traces:
      receivers: [otlp]
      processors:
        - attributes/remove_2xx_responses
        - attributes/remove_elb_health_checks

But I imagine there is more to it than this. The messages key is unclear to me in the filter, and the and/or operator is missing in the OTel configuration which is defined in NRQL.

0

There are 0 best solutions below