Elasticsearch - How to sum the values ​from each new document into a separate index?

Question

Example:

My documents:

{"_id":"1", "data_sent":"100"}
{"_id":"2", "data_sent":"110"}
{"_id":"3", "data_sent":"120"}

I would like to get value of 'data_sent' for every new document and sum it up to another index, lets say

index_name: 'data_sum'
field: 'total_data_sent'='330'

Bonus: I would like to create new indexes automatically for specified time period (for example /week)

I know that aggregations can be used here, but as I understand they are performed when the request is sent and for big data it could last for a while. I need to receive those data very fast when its needed.

Is there anything in Elastic that could help in my case?

Answer 1

I have figured it out by diving deeper into documentation.

'Transforms' was that I was looking for.

https://www.elastic.co/guide/en/elasticsearch/reference/7.9/transform-overview.html