Goaccess, Format Errors - Verify your log/date/time format

Question

I try to parse my nginx log file, my log file mylog.log is the below

2021-06-29T08:48:47+00:00 "10.146.212.15" -> "mydns.com" - 0.000  5704 - - 4653  302 "68b069fb-6e8f-4a9c-88fd-8a947354b681" "GET /master HTTP/1.1"
2021-06-29T08:48:47+00:00 "10.146.212.15" -> "mydns.com" 0.006 0.006  726 855 337 621  302 "-" "GET /auth/admin/master/console HTTP/1.1"

I try to parse it using the command "goaccess mylog.log --log-format=COMBINED --http-method=no --http-protocol=no -o output.html"

I got this error: ==1== Format Errors - Verify your log/date/time format

In fact:

'$time_iso8601 "$remote_addr" -> "$host" $upstream_response_time $request_time  $request_length $upstream_bytes_sent $upstream_bytes_received $bytes_sent  $status "$http_x_token_userid" "$request"'

Answer 1

Using the below configuration file as goaccess-nginx.conf:

time-format %T
date-format %Y-%m-%d
log_format %dT%t+%^ "%h" -> "%v" %^ %T  %^ %^ %D %b  %s "%e" "%r"

# $time_iso8601 "$remote_addr" -> "$host" $upstream_response_time $request_time  $request_length $upstream_bytes_sent $upstream_bytes_received $bytes_sent  $status "$http_x_token_userid" "$request"
# 2021-06-29T08:48:47+00:00 "10.146.212.15" -> "mydns.com" - 0.000  5704 - - 4653  302 "68b069fb-6e8f-4a9c-88fd-8a947354b681" "GET /master HTTP/1.1"
# 2021-06-29T08:48:47+00:00 "10.146.212.15" -> "mydns.com" 0.006 0.006  726 855 337 621  302 "-" "GET /auth/admin/master/console HTTP/1.1"

You can parse the log correctly with all of the custom fields supported by goaccess

goaccess --config-file goaccess-nginx.conf stackoverflow.log

---

If you prefer not to use a goaccess config file, you can specify the correct parameters directly as flags:

goaccess --time-format %T --date-format %Y-%m-%d --log_format '%dT%t+%^ "%h" -> "%v" %^ %T  %^ %^ %D %b  %s "%e" "%r"' stackoverflow.log