I am collecting logs from two different paths:
- /var/log/containers/*.log
- /var/log/agents/*.log
I wanted a that files from each of the paths will be sent to different indexes in elasticsearch. I tried to define the filebeat configuration as follow:
output.elasticsearch:
protocol: http
hosts: ["elasticsearch:9200"]
compression_level: 1
indices:
- index: "agent-logs"
when:
contains:
log.file.path: "/var/log/agents/*.log"
- index: "container-logs"
when:
contains:
log.file.path: "/var/log/containers/*.log"
I also tried:
output.elasticsearch:
protocol: http
hosts: ["elasticsearch:9200"]
compression_level: 1
indices:
- index: "agent-logs"
when.contains:
log.file.path: "/var/log/agents/*.log"
- index: "container-logs"
when.contains:
log.file.path: "/var/log/containers/*.log"
and:
output.elasticsearch:
protocol: http
hosts: ["elasticsearch:9200"]
compression_level: 1
indices:
- index: "agent-logs"
when.equals:
log.file.path: "/var/log/agents/*.log"
- index: "container-logs"
when.equals:
log.file.path: "/var/log/containers/*.log"
But nothing seems to be working. Please help!
thank you
Thank you so much for your answer! But I have another issue... logs from /var/log/agents/*.log are not even passing to elasticsearch although they exist in the path, you know what might be the reason?
this is the full config file:
and I also tried this config, but when I run this config I only get agent logs:
what do you think?