Is there a way to Elasticsearch diferentiate when a field does not exists vs when a field exists but is null ({})

Question

I have 2 kinds of documents:

"doc1":{
    "dummy_attribute":"HI",
    ...
    "important_attribute":{}
}
"doc2":{
    "dummy_attribute":"HI",
    ...
}

If I search like this:

GET myindex/_search
{
  "query": {
    "bool": {
      "must_not": [{
        "exists": {
          "field": "important_attribute"
        }
      }]
    }
  }
}

it returns me both documents. Is there a way to search only for the documents where "important_attribute" actually does not exist in the document ?

Answer 1

That's unfortunately not possible since ES treats a non-existent value the same way as an empty-ish value (null, "", [] or {}).

You could use script fields to quickly check whether a document contains empty dicts but this context cannot be used when filtering:

GET myindex/_search
{
  "query": {
    "bool": {
      "must_not": [
        {
          "exists": {
            "field": "important_attribute"
          }
        }
      ]
    }
  },
  "script_fields": {
    "has_empty_dict": {
      "script": {
        "source": "params._source.containsKey('important_attribute') && params._source.important_attribute.size() == 0"
      }
    }
  }
}