Elasticsearch indexing error - field [data] not present as part of path [data]

2.2k Views Asked by At

I get this error: field [data] not present as part of path [data] when I attempt to ingest an attachment encoded in CBOR, in ElasticSearch 7.16. Using the ingest-attachment plugin. Files are large, so using CBOR format to prevent encoding and decoding data (example in Python included).

Example file to be ingested (sample.txt) contains:

The quick brown fox jumps over a lazy dog

which, when encoded in CBOR I get

x)The quick brown fox jumps over a lazy dog

Here is what I have tried:


Create the index abcd1234

curl -X PUT "localhost:9200/abcd1234?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index": {
      "number_of_shards": 1,  
      "number_of_replicas": 0 
    }
  },
  "mappings": {
    "properties": {
      "attachment.data": {
        "type": "binary"
      }
    }
  }
}'

Create the attachment pipeline

curl -X PUT "localhost:9200/_ingest/pipeline/cbor-attachment?pretty" -H 'Content-Type: application/json' -d'
{
  "description" : "Extract attachment information encoded in CBOR",
  "processors" : [
    {
      "attachment": {
        "description" : "Ingest attachment",
        "field": "data",
        "indexed_chars": -1
      }
    }
  ]
}'

Attempt to index the sample file

curl -X PUT "localhost:9200/abcd1234/_doc/sample?pipeline=cbor-attachment" -H 'Content-Type: application/cbor' -d'
{
  "data": "x)The quick brown fox jumps over a lazy dog"
}'

Any hints on what I might be doing wrong?

0

There are 0 best solutions below