I want to use Grok Pattern for filtering out this
172.20.20.88 - - [10/Nov/2018:23:49:31 +0700] "GET /id/profile.pl?user=285&device=Bg3tlX HTTP/1.1" 502 852 "-" "Go-http-client/2.0" "0.009"
I am using COMMONAPACHELOG
%{IPORHOST:clientip} %{HTTPDUSER:ident} %{USER:auth} \[%{HTTPDATE:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-)
I have tried %{URIPATH:request}
and %{URIPARAM:request}
. The result of request is still /id/profile.pl?user=285&device=Bg3tlX
. My expectation is /id/profile.pl
.
My reference is https://github.com/hpcugent/logstash-patterns/blob/master/files/grok-patterns
Your
%{NOTSPACE:request}
matches any 1 or more non-whitespace chars beforeHTTP/1.1" 502 85...
asNOTSPACE
pattern is\S+
. So, it matches the whole/id/profile.pl?user=285&device=Bg3tlX
substring.You cannot use just
URIPATH
orURIPARAM
, because you still need to match the rest of the input. You have to use both, but makeURIPARAM
optional afterURIPATH
by enclosing it within an optional non-capturing group,(?:...)?
.So, replace
%{NOTSPACE:request}
withDemo at https://grokdebug.herokuapp.com/: