Elasticsearch completion suggester issue

Question

Issue - completion suggester with custom keyword lowercase analyzer not working as expected. We can reproduce the issue with following steps.

Not able to understand whats causing issue here. However, if we search for "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE" , it is giving result.

Create index

curl -X PUT "localhost:9200/com.tmp.index?pretty"  -H 'Content-Type: application/json' -d'{
  "mappings": {
    "dynamic": "false",
    "properties": {
      "namesuggest": {
        "type": "completion",
        "analyzer": "keyword_lowercase_analyzer",
        "preserve_separators": true,
        "preserve_position_increments": true,
        "max_input_length": 50,
        "contexts": [
          {
            "name": "searchable",
            "type": "CATEGORY"
          }
        ]
      }
    }
  },
  "settings": {
    "index": {
      "mapping": {
        "ignore_malformed": "true"
      },
      "refresh_interval": "5s",
      "analysis": {
        "analyzer": {
          "keyword_lowercase_analyzer": {
            "filter": [
              "lowercase"
            ],
            "type": "custom",
            "tokenizer": "keyword"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "1"
    }
  }
}'

Index document

curl -X PUT "localhost:9200/com.tmp.index/_doc/123?pretty" -H 'Content-Type: application/json' -d'{
    "namesuggest": {
        "input": [
            "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE LIMITED."
        ],
        "contexts": {
            "searchable": [
                "*"
            ]
        }
    }
}
'

Issue - Complete suggest not giving result

curl -X GET "localhost:9200/com.tmp.index/_search?pretty" -H 'Content-Type: application/json' -d'{
    "suggest": {
        "legalEntity": {
            "prefix": "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE LIMITED.",
            "completion": {
                "field": "namesuggest",
                "size": 10,
                "contexts": {
                    "searchable": [
                        {
                            "context": "*",
                            "boost": 1,
                            "prefix": false
                        }
                    ]
                }
            }
        }
    }
}'

Answer 1

You are facing this issue because of default value of max_input_length parameter is set to 50.

Below is description given for this parameter in documentation:

Limits the length of a single input, defaults to 50 UTF-16 code points. This limit is only used at index time to reduce the total number of characters per input string in order to prevent massive inputs from bloating the underlying datastructure. Most use cases won’t be influenced by the default value since prefix completions seldom grow beyond prefixes longer than a handful of characters.

If you enter below string which is exact 50 character then you will get response:

PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE

Now if you add one more or two character to above string then it will not resturn the result:

PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE L

You can use this default behaviour or you can updated your index mapping with increase value of max_input_length parameter and reindex your data.

{
  "mappings": {
    "dynamic": "false",
    "properties": {
      "namesuggest": {
        "type": "completion",
        "analyzer": "keyword_lowercase_analyzer",
        "preserve_separators": true,
        "preserve_position_increments": true,
        "max_input_length": 100,
        "contexts": [
          {
            "name": "searchable",
            "type": "CATEGORY"
          }
        ]
      }
    }
  },
  "settings": {
    "index": {
      "mapping": {
        "ignore_malformed": "true"
      },
      "refresh_interval": "5s",
      "analysis": {
        "analyzer": {
          "keyword_lowercase_analyzer": {
            "filter": [
              "lowercase"
            ],
            "type": "custom",
            "tokenizer": "keyword"
          }
        }
      },
      "number_of_replicas": "0",
      "number_of_shards": "1"
    }
  }
}

You will get response like below after updating index:

"suggest": {
    "legalEntity": [
      {
        "text": "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE LIMITED",
        "offset": 0,
        "length": 58,
        "options": [
          {
            "text": "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE LIMITED.",
            "_index": "74071871",
            "_id": "123",
            "_score": 1,
            "_source": {
              "namesuggest": {
                "input": [
                  "PRAXIS CONSULTING AND INFORMATION SERVICES PRIVATE LIMITED."
                ],
                "contexts": {
                  "searchable": [
                    "*"
                  ]
                }
              }
            },
            "contexts": {
              "searchable": [
                "*"
              ]
            }
          }
        ]
      }
    ]
  }