Grafana loki regex pipeline to parse a log line from text into json

1.3k Views Asked by At

I have this log entry in Grafana Loki I would like to parse the logline from being a text formatted some how to be json formatted and to extract some fields from it.

I added the regex to the query as apipeline or at leaset this is what I am trying to do but it is not being applied.

the regex matches the lines outside when I tested it in the https://regex101.com/

Query

{container="fluent-bit"} | regexp "\\[(?P<lineindex>\\d+)\\] (?P<filetag>\\S.+)[:] \\[(?P<timestamp>\\d+.\\d+)\\, \\{\\\"(?P<filename_label>\\w+)\\\"\\=\\>\\\"(?P<filename>)\\/\\S+\\/\\S.\\S+\\\"\\, \\\"(?P<log_label>\\S+)\\\"\\=\\>\\\"(?P<log>)[[:ascii:]]+\\\"\\}\\]\\\n*"

enter image description here

This is how the log line looks right now

log "[1] kube.fluent.log.localhost.2022-06-10.log: [1654861302.011451067, {\"filename\"=>\"/fluent/log/localhost.2022-06-10.log\", \"log\"=>\"10-Jun-2022 11:41:42.010 INFO [main] org.apache.catalina.core.ApplicationContext.log SessionListener: contextInitialized()\"}]\n"

enter image description here

and I wish to have its fields extracted as per the regex to be like this

lineindex: 1 filetag: "kube.fluent.log.localhost.2022-06-10.log" timestamp: 1654861302.011451067 filename_label: filename filename: "/fluent/log/localhost.2022-06-10.log" log_label: log log: "10-Jun-2022 11:41:42.010 INFO [main] org.apache.catalina.core.ApplicationContext.log SessionListener: contextInitialized()"

0

There are 0 best solutions below