How to add/remove additional fields in Graylog

1.6k Views Asked by At

Thanks for your guidance, I would like to include additional fields in the fields tab in Search result. I have took a look at the extractors and configured accordingly. For instance, this is the message to parse:

2019-03-12 10:15:51 [https-jsse-nio-8020-exec-10] INFO : ASCPA7C500611418  ab.bc.cde.efg.common.rest.endpoints.component.filter End

I added JAVACLASS pattern:

(?:[a-zA-Z$_][a-zA-Z$_0-9]*\.)*[a-zA-Z$_][a-zA-Z$_0-9]*

After, in I configured the extractor and included the following Pattern:

%{TIMESTAMP_ISO8601}(?:%{SPACE})%{SYSLOG5424SD}(?:%{SPACE})%{LOGLEVEL}*(?:%{SPACE}):*(?:%{SPACE})%{WORD}*(?:%{SPACE})%{JAVACLASS:class}%{GREEDYDATA}

This provided the following Extractor preview:

WORD
ASCPA7C500611418
TIMESTAMP_ISO8601
2019-03-12 10:15:51
MONTHNUM
03
HOUR
[10, null]
message
End
SPACE
[ , , , , ]
YEAR
2019
DATA
https-jsse-nio-8020-exec-10
MINUTE
[15, null]
SECOND
51
LOGLEVEL
INFO
JAVACLASS
ab.bc.cde.efg.common.rest.endpoints.component.filter
MONTHDAY
12
SYSLOG5424SD
[https-jsse-nio-8020-exec-10]

Java class is parsed correctly, but in the Search result ALL the fields are shown under the Fields tab:

enter image description here

However, I want just to add "class" field in the list. How can I get rid of the other fields? I already attempted to keep just JAVACLASS, but I am not getting the appropriate value for the class.

I also removed the extractor and added the following in the filter in logstash.conf :

 filter {
        grok {
            match => {
                "message" => "%{TIMESTAMP_ISO8601}(?:%{SPACE})%{SYSLOG5424SD}(?:%{SPACE})%{LOGLEVEL}*(?:%{SPACE}):*(?:%{SPACE})%{WORD}*(?:%{SPACE})%{JAVACLASS:class}%{GREEDYDATA:message}"            
            }
            add_field => [ "class","%{JAVACLASS}"]

        }        
}

but I did not get the field called class in Graylog: enter image description here

How can I get fields added such as class field in this use case?

Update When adding this fix:

add_field => { "class" => "%{JAVACLASS}" }

Getting the following logstash output:

[2019-03-14T09:52:58,025][DEBUG][logstash.outputs.gelf    ] Sending GELF event {:event=>{"short_message"=>["2019-03-07 06:06:46 [localhost-startStop-1] DEBUG:   org.springframework.beans.factory.support.DefaultListableBeanFactory Autowiring by type from bean name 'serviceClientMapper' via property 'sqlSessionFactory' to bean named 'sqlSessionFactory'", " Autowiring by type from bean name 'serviceClientMapper' via property 'sqlSessionFactory' to bean named 'sqlSessionFactory'"], "full_message"=>"2019-03-07 06:06:46 [localhost-startStop-1] DEBUG:   org.springframework.beans.factory.support.DefaultListableBeanFactory Autowiring by type from bean name 'serviceClientMapper' via property 'sqlSessionFactory' to bean named 'sqlSessionFactory', Autowiring by type from bean name 'serviceClientMapper' via property 'sqlSessionFactory' to bean named 'sqlSessionFactory'", "host"=>"{\"os\":{\"name\":\"CentOS Linux\",\"version\":\"7 (Core)\",\"codename\":\"Core\"},\"name\":\"d1tomcat\"}", "_log_file"=>{"path"=>"/apps/logs/ABC/abc-rest-api/abc-rest-api.log"}, "_source"=>"/apps/logs/ABC/abc-rest-api/abc-rest-api.log", "_meta_cloud"=>{}, "_tags"=>"beats_input_codec_plain_applied", **"_class"=>"org.springframework.beans.factory.support.DefaultListableBeanFactory, %{JAVACLASS}"**, "_beat_name"=>"d1tomcat", "_beat_hostname"=>"d1tomcat", "_component"=>"component", "level"=>6}}

Thanks for your help

1

There are 1 best solutions below

1
On

The syntax of your add_field configuration is wrong.

It should be:

add_field => { "class" => "%{JAVACLASS}" }