Elasticsearch Filebeat document type deprecated issue

Question

I am currently using ELK 5.5. It appears document_type is now deprecated in Filebeats, but I could not find any example anywhere as to how to implement the same now.

This is what I get in my log:

WARN DEPRECATED: document_type is deprecated. Use fields instead.

This is my current filebeat configuration:

- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - C:\inetpub\logs\LogFiles\*\*
  document_type: iislog

  paths:
    - C:\MyApp\logs\*
  document_type: applog

Can someone tell me how to rewrite my log when using the same Version 5.5 and get rid of this deprecation message. BTW, I do want to use the same ES index for both "document types".

Answer 1

Instead of using document_type, you can use fields like this on Filebeat:

- input_type: log

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - C:\inetpub\logs\LogFiles\*\*
  fields:
    service: iislog
  fields_under_root: true 

  paths:
    - C:\MyApp\logs\*
  fields:
    service: applog
  fields_under_root: true

Now, for Logstash output filter, instead of using [type] for calling the document_type, you can use [service]. Here is how i'm using on logstash:

output {
  if [service] == "applog" {   
    elasticsearch {
    hosts => ["localhost:9200"]
    user => <user>
    password => <pass>
    index => "applog-%{+YYYY.MM.dd}"
    }   
  }
enter code here

Check bellow for more information about custom fields on Filebeat: https://www.elastic.co/guide/en/beats/filebeat/current/migration-changed-fields.html