How can I automate data deletion in jaeger and elasticsearch?

Question

I want to automate the process of removing old data from elastic search container.

I have written a docker-compose file to run elastic search, jaeger collector, agent, and query containers and then configured jaeger with elastic search.

Now I want that whenever the data exceeds a specific limit elastic search deletes the old data automatically.

The contents of the docker compose file are as follows:

elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.0
    networks:
      - elastic-jaeger
    ports:
      - "127.0.0.1:9200:9200"
      - "127.0.0.1:9300:9300"
    restart: on-failure
    environment:
      - transport.host=127.0.0.1
    volumes:
      - ./.docker-volumes/esdata:/usr/share/elasticsearch/data

  jaeger-collector:
    image: jaegertracing/jaeger-collector
    ports:
      - "14269:14269"
      - "14268:14268"
      - "14267:14267"
      - "9411:9411"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--log-level=error"
    ]
    depends_on:
      - elasticsearch

  jaeger-agent:
    image: jaegertracing/jaeger-agent
    hostname: jaeger-agent
    command: ["--collector.host-port=jaeger-collector:14267"]
    ports:
      - "5775:5775/udp"
      - "6831:6831/udp"
      - "6832:6832/udp"
      - "5778:5778"
    networks:
      - elastic-jaeger
    restart: on-failure
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
    depends_on:
      - jaeger-collector

  jaeger-query:
    image: jaegertracing/jaeger-query
    environment:
      - SPAN_STORAGE_TYPE=elasticsearch
      - no_proxy=localhost
    ports:
      - "16686:16686"
      - "16687:16687"
    networks:
      - elastic-jaeger
    restart: on-failure
    command: [
      "--es.server-urls=http://elasticsearch:9200",
      "--span-storage.type=elasticsearch",
      "--log-level=debug"
    ]
    depends_on:
      - jaeger-agent

networks:
  elastic-jaeger:
    driver: bridge

Answer 1

Elasticsearch itself provides a functionality called Index Rollover that allows managing the size of indices.

https://www.jaegertracing.io/docs/1.45/deployment/#elasticsearch-rollover