Kibana scripted field error tells me I dont have any values in field while they do show up in discovery

Question

I've been trying to create a scripted field in which i want to make a URL which directs me to a specific APM timeline page. The values I want to add to this URL are filters which are needed to select te specific transaction in APM.

When i try to create this field I keep getting an error telling me there are no values in 2/4 fields.

The error is:

{
 "root_cause": [
  {
   "type": "script_exception",
   "reason": "runtime error",
   "script_stack": [
    "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
    "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
    "'https://xxx/xxx/xxx/xxxrangeFrom=now-1h&rangeTo=now&traceId=' + doc['trace.id'].value + '&transactionId=' + doc['transaction.id'].value + '&transactionName=' + doc['transaction.name'].value + '&transactionType=' + doc['transaction.type'].value",
    "                                                                                                                                                                                        ^---- HERE"
   ],
   "script": "'https://xxx/xxx/xxx/xxxrangeFrom=now-1h&rangeTo=now&traceId=' + doc['trace.id'].value + '&transactionId=' + doc['transaction.id'].value + '&transactionName=' + doc['transaction.name'].value + '&transactionType=' + doc['transaction.type'].value",
   "lang": "painless",
   "position": {
    "offset": 184,
    "start": 0,
    "end": 244
   }
  }
 ],
 "type": "search_phase_execution_exception",
 "reason": "Partial shards failure",
 "phase": "fetch",
 "grouped": true,
 "failed_shards": [
  {
   "shard": 0,
   "index": "apm-2020.47-esdec-backend-error-7.8.1",
   "node": "uvVVVbE9RRGTnxaF6_S73g",
   "reason": {
    "type": "script_exception",
    "reason": "runtime error",
    "script_stack": [
     "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.get(ScriptDocValues.java:496)",
     "org.elasticsearch.index.fielddata.ScriptDocValues$Strings.getValue(ScriptDocValues.java:503)",
     "'https://xxx/xxx/xxx/xxxrangeFrom=now-1h&rangeTo=now&traceId=' + doc['trace.id'].value + '&transactionId=' + doc['transaction.id'].value + '&transactionName=' + doc['transaction.name'].value + '&transactionType=' + doc['transaction.type'].value",
     "                                                                                                                                                                                        ^---- HERE"
    ],
    "script": "'https://xxx/xxx/xxx/xxxrangeFrom=now-1h&rangeTo=now&traceId=' + doc['trace.id'].value + '&transactionId=' + doc['transaction.id'].value + '&transactionName=' + doc['transaction.name'].value + '&transactionType=' + doc['transaction.type'].value",
    "lang": "painless",
    "position": {
     "offset": 184,
     "start": 0,
     "end": 244
    },
    "caused_by": {
     "type": "illegal_state_exception",
     "reason": "A document doesn't have a value for a field! Use doc[<field>].size()==0 to check if a document is missing a field!"
    }
   }
  }
 ]

When I check in discovery my fields definetly do have values. I can't figure out what the problem is. Anybody has any idea?

Answer 1

Try with .size() == 1 and other checks for the fields.

if (doc['field.keyword'].size() == 1 && doc['field.keyword'] != null && doc.containsKey('field')) {      
      return Double.parseDouble(doc['field.keyword'].value)       
}

This answer provides more information about the breaking changes in Elastic 7: https://discuss.elastic.co/t/scripted-fields-dont-work-after-upgrading-to-7-1/185829/2