Get index of logstash split filter

354 Views Asked by At

I am using Logstash split filter plugin as described in this page : https://www.elastic.co/guide/en/logstash/current/plugins-filters-split.html

Let say I have data looking like that :

{"log_id": "abcd", "logs": [{"val": 3}, {"val": 4}]}

Using the split filter like this :

filter { split { field => "logs" } }

Would output documents as follow :

{"log_id": "abcd", "logs": {"val": 3}}
{"log_id": "abcd", "logs": {"val": 4}}

I would like to extract the index of the splited document to use as unique id as follow :

{"log_id": "abcd", "logs": {"val": 3}, "unique_id": "abcd-0"}
{"log_id": "abcd", "logs": {"val": 4}, "unique_id": "abcd-1"}
2

There are 2 best solutions below

0
On BEST ANSWER

This is the patch I applied to fix this :

88c88
<     splits.each do |value|
---
>     splits.each_with_index do |value, index|
97a98
>       event_split.set("_split_idx", index)
0
On

From looking at the code, the plugin does not support this feature. Please consider adding an issue there.