Severity level on docker logging to syslog

Question

I have a syslog-ng container that collects logs from other containers running on the same application, and normal logging works as intended: every container send its logs and syslog-ng saves them on separate files, as defined in syslog-ng.conf.

I'm also trying to save error logs on a different file. I was assuming that everything thrown to STDERR from inside another container should be seen by syslog-ng as an error, so when I started a container like this:

docker run -dit --log-driver syslog --log-opt syslog-address=tcp://my_syslog:601 alpine echo "boom" > /dev/stderr

then I expected syslog-ng to understand it to be an error, but it does not.

This is an extract of my syslog-ng.conf inside the syslog-ng container:

@version: 3.25
source s_net {
  network(transport(tcp) port(601));
  network(transport(udp) port(514));
};
destination d_error_file {
  file("/tmp/errors");
};
filter f_errors {
  level(err .. emerg);
};
log {
  source(s_net);
  filter(f_errors);
  destination(d_error_file);
};

How can I tell syslog-ng to treat STDERR messages as errors?

Answer 1

My original reply to set up a /dev/stderr source was misinformed. I thought that syslog-ng was running in the host.

As syslog-ng is running in the container, the docker daemon is collecting the logs from the running container and sending it over the network to the syslog server.

This is why the filtering has to be performed on the network source which is just as you have done.

I setup the following project to demonstrate an example.

.
├── data
│   └── syslog-ng
│       ├── conf
│       │   └── syslog-ng.conf
│       └── logs
│           ├── errors
│           └── syslog
├── docker-compose.yml
└── hack.sh

docker-compose.yml

version: '3.8'

services:
  hack:
    image: alpine
    command: /hack.sh
    logging:
      driver: syslog
      options:
        syslog-address: "tcp://host.docker.internal:601"
    depends_on:
      - syslog
    volumes:
      - '$PWD/hack.sh:/hack.sh'
  syslog:
    container_name: syslog-ng
    image: balabit/syslog-ng:latest
    volumes:
      - '$PWD/data/syslog-ng/conf/syslog-ng.conf:/etc/syslog-ng/syslog-ng.conf'
      - '$PWD/data/syslog-ng/logs:/var/log'
    ports:
      - 514:514
      - 601:601

hack.sh: A shell script that logs to stderr and stdout

#!/bin/sh

while :
do
    echo "boom" >/dev/stderr;
    echo "no boom";
    sleep 2;
done

./data/syslog-ng/conf/syslog-ng.conf: filter info to /var/log/syslog and error to /var/log/error

@version: 3.29
@include "scl.conf"

source s_net {
   udp(
     ip("0.0.0.0")
   );

  network(
    ip("0.0.0.0")
    transport(tcp) 
    port(601)
  );

 };
  
  filter f_errors {
    level(err .. emerg);
  };
  filter f_info {
    level(info);
  };

    destination d_file {
      file("/var/log/syslog");
    };
    destination d_error_file {
      file("/var/log/errors");
    };

      log {
        source(s_net);
        filter(f_info);
        destination(d_file); 
      };
      log {
        source(s_net);
        filter(f_errors);
        destination(d_error_file);
      };

Other than creating a different filter for non-error messages, there is little difference between the above syslog-ng configuration and the one you have provided.