set field data = true on java elasticsearch

Question

Running a java elastic search query through Eclipse. Getting this error when I am testing it, and I can't find anywhere in the API that tells me how I can set this field data to true

IllegalArgumentException[Fielddata is disabled on text fields by default. Set fielddata=true on [created] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.]

Does anyone know how I can fix this?

Answer 1

In Elasticsearch 5.0, String fields were broken into two - Text and Keyword. If the property is supposed to be Analyzed, you'd want to migrate your String fields to Text, if not then migrate them to Keyword.

Look into your type mapping - If you had a property like this -

{
  "your_field": {
    "type" "string",
    "index": "not_analyzed"
  }
}

You'd want to convert it to this -

{
  "your_field": {
    "type" "keyword",
    "index": true
  }
}

Similarly, if your property was supposed to be analyzed and you had it like this-

{
  "your_field": {
    "type" "string",
    "index": "analyzed"
  }
}

then you'd want to convert it to this-

{
  "your_field": {
    "type" "text",
    "index": true
  }
}

Details on Elasticsearch official page

Answer 2

You must modify index mapping properties:

try (XContentBuilder jsonBuilder = XContentFactory.jsonBuilder()) {
    final XContentBuilder builder = jsonBuilder
            .startObject()
              .startObject("your_type")
                .startObject("properties")
                  .startObject("your_field")
                    .field("type", "text")
                    .field("fielddata", true)/*setting fielddata*/
                  .endObject()
                .endObject()
              .endObject()
            .endObject();

    client.admin().indices().preparePutMapping("your_index")
            .setType("your_type")
            .setSource(builder)/*also there are overloads for setSource()*/
            .get();
}

OR

String source = "{\"your_type\":{\"properties\":{\"your_field\":{\"type\":\"text\",\"fielddata\":true}}}}";
client.admin().indices().preparePutMapping("your_index)
        .setType("your_type")
        .setSource(source, XContentType.JSON)
        .get();

Result:

{
  "your_index": {
    "aliases": {},
    "mappings": {
      "your_type": {
        "properties": {
          ...
          ...
          "your_field": {
            "type": "text",
            "fielddata": true,
            "fields": {
              "keyword": {
                "type": "keyword",
                "ignore_above": 256
              }
            }
          }
          ...
          ...
        }
      }
    }
  }
}

Answer 3

I have recently encountered this issue but in my case, I faced this issue when I was performing the Sorting.

An Excerpt from the link

Most fields are indexed by default, which makes them searchable. Sorting, aggregations, and accessing field values in scripts, however, requires a different access pattern from search.

Fielddata is disabled on text fields by default.

we can solve this problem with two approaches

1. Updating the mappings by enabling fielddata on text fields as "fielddata": true

PUT your_index/_mapping
{
  "properties": {
    "my_field": { 
      "type":     "text",
      "fielddata": true
    }
  }
}

2. Appending .keyword to the field as your_field.keyword for aggregations, sorting.

In the previous approach, we have to perform reindexing which is sometimes a cumbersome process for large indexes. For such cases, the below solution would be an easier solution to implement.

ES Query:

GET /your_index/_search
{
  "query": {
    "match_all": {}
  },
   "sort":[
      {
         "your_field.keyword":{
            "order":"asc"
         }
      }
   ]
}

Java Code

SearchRequest searchRequest = new SearchRequest("your_index");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchAllQuery());
//Appending the .keyword to the sorting field
FieldSortBuilder fieldSortBuilder = SortBuilders.fieldSort("your_field"+".keyword");
fieldSortBuilder.order(SortOrder.ASC);
searchSourceBuilder.sort(fieldSortBuilder);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = restHighLevelClient.search(searchRequest, RequestOptions.DEFAULT);