match_only_text fields do not support sorting and aggregations elasticsearch

Question

I would like to count and sort the number of occurred message on a field of type match_only_text. Using a DSL query the output needed to have to look like this:

{" Text message 1":615
" Text message 2":568
....}

So i tried this on kibana:

 GET my_index_name/_search?size=0
{
  "aggs": {
    "type_promoted_count": {
      "cardinality": {
        "field": "message"
      }
    }
  }
}

However i get this error:

"error" : {
    "root_cause" : [
      {
        "type" : "illegal_argument_exception",
        "reason" : "match_only_text fields do not support sorting and aggregations"
      }

I am interested in the field "message" this is its mapping:

"message" : {
          "type" : "match_only_text"
        }

This is a part of the index mapping:

"mappings" : {
      "_meta" : {
        "package" : {
          "name" : "system"
        },
        "managed_by" : "ingest-manager",
        "managed" : true
      },
      "_data_stream_timestamp" : {
        "enabled" : true
      },
      "dynamic_templates" : [
        {
          "strings_as_keyword" : {
            "match_mapping_type" : "string",
            "mapping" : {
              "ignore_above" : 1024,
              "type" : "keyword"
            }
          }
        }
      ],
      "date_detection" : false,
      "properties" : {
        "@timestamp" : {
          "type" : "date"
        }
.
.
.
        "message" : {
          "type" : "match_only_text"
        },
        "process" : {
          "properties" : {
            "name" : {
              "type" : "keyword",
              "ignore_above" : 1024
            },
            "pid" : {
              "type" : "long"
            }
          }
        },
        "system" : {
          "properties" : {
            "syslog" : {
              "type" : "object"
            }
          }
        }
      }
    }
  }
}

Please Help

Answer 1

Yes, by design, match_only_text is of the text field type family, hence you cannot aggregate on it.

You need to:

A. create a message.keyword sub-field in your mapping of type keyword:

PUT my_index_name/_mapping
{
  "properties": {
    "message" : {
      "type" : "match_only_text",
      "fields": {
        "keyword": {
          "type" : "keyword"
        }
      }
    }
  }
}

B. update the whole index (using _update_by_query) so the sub-field gets populated and

POST my_index_name/_update_by_query?wait_for_completion=false

Then, depending on the size of your index, call GET _tasks?actions=*byquery&detailed regularly to check the progress of the task.

C. run the aggregation on that sub-field.

 POST my_index_name/_search
 {
  "size": 0,
  "aggs": {
    "type_promoted_count": {
      "cardinality": {
        "field": "message.keyword"
      }
    }
  }
}