Adding new files to rsyslogd with wildcards

Question

We're got a pre-existing rsyslog config file which is working for papertrail e.g.

/etc/rsyslog.d/20-papertrail.conf which has

*.*          @logs4.papertrailapp.com:44407

However we've got a couple of NGINX websites on the server so would like to have it also monitor their error logs.

The paths to them are:

/var/log/nginx/www.website-one.com-error.log
/var/log/nginx/www.website-two.com-error.log
/var/log/nginx/www.website-three.com-error.log

However this /var/log/nginx also contains a bunch of .log files which we do not want to monitor e.g.

/var/log/nginx/error.log
/var/log/nginx/access.log
/var/log/nginx/error.log1
/var/log/nginx/nginx.log

In my head we need to add something like...

/var/log/nginx/*-error.log

And make sure they pipe to the papertrail url as well.

However I'm struggling to decipher the rsyslog documentation to figure out how to do this.

Thanks!

Answer 1

In rsyslog documentation it seems that you can use wildcards in files.

File

The file being monitored. So far, this must be an absolute name (no macros or templates). Note that wildcards are supported at the file name level (see WildCards below for more details).

WildCards

Before Version: 8.25.0

Wildcards are only supported in the filename part, not in directory names.

/var/log/*.log works.

/var/log/*/syslog.log does not work.

Since Version: 8.25.0

Wildcards are supported in filename and paths which means these samples will work:

/var/log/*.log works.

/var/log/*/syslog.log works.

/var/log/*/*.log works.

All matching files in all matching subfolders will work. Note that this may decrease performance in imfile depending on how many directories and files are being watched dynamically.

If you want to forward your vhosts logs you can change configuration directly in NGINX vhosts configuration, you should change/add access_log and error_log policies as explained here or use custom facilities to forward your logs (using rsyslog).

HOW TO DO IT USING RSYSLOG?

Create a new custom file in /etc/rsyslog.d/nginx_custom.conf:

module(load="imfile" PollingInterval="1") #needs to be done just once

# File 1
input(type="imfile"
      File="/var/log/nginx/www.website-*.com-error.log"
      Tag="websites"
      Facility="local0")

local0.* @logs4.papertrailapp.com:44407
#Just to test that logs are forwarded, comment the line once you've tested it
local0.* /var/log/test.log

And restart rsyslog service

NOTE: Line local0.* /var/log/test.log is just to test that you can see forwarded logs into your local server, comment this line after you've tested that everything works.