is it possible to set log path dynamically with fluentd?

Question

I'm making some management system, and want to manage docker container's log with fluentd.

What I really want to do is saving logs dynamically with parameter in --log-opt tag. For example, when I deploy a container, I use command like this: docker run --log-driver=fluentd --log-opt fluentd-address=some_addr --log-opt tag={task_id} some_image

What I'm trying to do is classifying logs by task_id in the log-opt's tag. In fluent.conf, I want to set path like this: /fluent/log/{task_id}/data.*.log

How can I pass variables or placeholder into fluentd conf file?

Answer 1

You can try after adding environment variable in command. PFB link for fluentd deploy(daemonset) file in YAML(kubernetes), I am passing Environment variable in Fluentd daemonset file(Fluentd Deployment) and using the same in fluentd.conf.

How to get ${kubernetes.namespace_name} for index_name in fluentd?

Pass environment variable in docker- https://stackoverflow.com/questions/30494050/how-do-i-pass-environment-variables-to-docker-containers#:~:text=Using%20docker%2Dcompose%20%2C%20you%20can,commands%20specific%20to%20the%20environment.&text=Use%20%2De%20or%20%2D%2Denv,set%20environment%20variables%20(default%20%5B%5D).