Getting Docker logs to Fluentd via SYSLOG_IDENTIFIER

Question

I have docker containers which writes all logs to Journald. I have added these two flags to the docker run command

--log-driver=journald --log-opt tag="docker.test"

When I see the logs using journalctl CONTAINER_NAME=test-docker, I see the the tag clearly which I defined in --log-opt tag="docker.{{.Name}}"

Mar 23 06:33:57 sethnpl025 "docker.test"[1534]: {"thread":"localhost-startStop-1","level":"INFO","loggerName":.......

My main aim is to get these logs from journald and write them to Google Cloud logging using Fluentd. But the problem is, when I try to filter logs using the SYSLOG_IDENTIFIER in Fluentd I do not get anything. As I read in docker documentation (if I read it correct), setting --log-opt tag="xxxx" will set the CONTAINER_TAG and SYSLOG_IDENTIFIER

My Fluentd config is

  @type systemd
  tag test
  path /var/log/journal
  <storage>
    @type local
    persistent true
    path /var/log/fluent/journal.pos
  </storage>
  matches [{ "SYSLOG_IDENTIFIER": "docker.test" }]
  read_from_head true
</source>

My main question here is how can I filter logs from fluentd using SYSLOG_IDENTIFIER or am I doing anything wrong here?

Answer 1

I have found this Community Tutorial about how to configure Google Cloud Logging Driver, seems that could be useful for you.