Parse multiline logs as a single event in Fluentd

Question

We have EFK implemented on Openshift Container Platform version 4.3.

Issue: Multiline logs such as Java Stack trace, SQL queries are not getting parsed as a single event in Fluentd and because of this we are getting multiple entries in Kibana. We need to parse multiline logs as a single event, so that we get a single entry in Kibana for an exception or SQL query.

Answer 1

Easy: switch application logging format to json.

Hard: https://docs.fluentd.org/parser/multiline. Usually format_firstline and format1 is enough - everything else is appended until the next format_firstline match.