Logstash jdbc left outer join as subdocuments

Question

I'm using the Logstash jdbc plugin to get MySQL data into ElasticSearch. Due to a left outer join I end up with multiple 'child rows' for a single 'parent row'. Say 1 user has 1 or more documents. I tried to group_concat the text of the documents and then group by by the user id to retain 1 row per user.

However, MySQL's group_concat has a length limit of 1024...

Does anyone know a solution to overcome the group_concat altogether and deal with left outer joins as nested documents?

Thanx

Answer 1

The default value is 1024, but you may increase the allowed length for GROUP_CONCAT using

SET SESSION group_concat_max_len = 102400

or whatever length you deem necessary (more info in the official docs)

I'm using that in complex OUTER JOINs which create many child objects/documents, and it works pretty fine so far.