Use log filenames as Log names on Google logging

Question

How could I use log filenames as log names on google logging.

I have this on google-cloud-ops-agent/config.yaml

logging:
  receivers:
    nginx_access:
      type: nginx_access
    nginx_error:
      type: nginx_error
    nginx_access_files:
      type: files
      include_paths: [/var/log/nginx/*access*.log]
  service:
    pipelines:
      nginx:
        receivers:
          - nginx_access
          - nginx_error
          - nginx_access_files
metrics:
  receivers:
    nginx:
      type: nginx
      stub_status_url: http://127.0.0.1:80/status
  service:
    pipelines:
      nginx:
        receivers:
          - nginx

The problem is that all logs from folder /var/log/nginx/access.log shows up on google logging as one entity "nginx_access_files". The folder has hundreds of access logs with the site name on them. I would like to have that log filename as the log name on the logging console too.

This I have now on Google logging console:

And I would need to set up it so that the filename shows upon logging console. Like the default, Nginx error does on OPS agent. Or is there another way to route all access logs to google logging with the filenames?

Answer 1

There is no need to configure the nginx_access_files receiver, if you are using the default path for the access.log file, then the agent automatically pulls the logs and sends them to Cloud Logging.

The agent yaml file should be something like this:

logging:
  receivers:
    nginx_access:
      type: nginx_access
    nginx_error:
      type: nginx_error
  service:
    pipelines:
      nginx:
        receivers:
          - nginx_access
          - nginx_error
metrics:
  receivers:
    nginx:
      type: nginx
      stub_status_url: http://127.0.0.1:80/status
  service:
    pipelines:
      nginx:
        receivers:
          - nginx

Under the nginx_access log name, you will see all the access logs generated and since they are generated with the instance name as a label, you can use it to filter them:

If your access.log file is not in the default path, then you can specify where it is, but there is no need to create a new receiver for that, this is an example:

logging:
  receivers:
    nginx_access:
      type: nginx_access
      include_paths: [/your/log/path/access.log]
    nginx_error:
      type: nginx_error
  service:
    pipelines:
      nginx:
        receivers:
          - nginx_access
          - nginx_error
metrics:
  receivers:
    nginx:
      type: nginx
      stub_status_url: http://127.0.0.1:80/status
  service:
    pipelines:
      nginx:
        receivers:
          - nginx

Finally, I recommend you to take a look at the NGINX Cloud Logging integration tutorial.