I get this error: field [data] not present as part of path [data]
when I attempt to ingest an attachment encoded in CBOR, in ElasticSearch 7.16. Using the ingest-attachment plugin. Files are large, so using CBOR format to prevent encoding and decoding data (example in Python included).
Example file to be ingested (sample.txt) contains:
The quick brown fox jumps over a lazy dog
which, when encoded in CBOR I get
x)The quick brown fox jumps over a lazy dog
Here is what I have tried:
Create the index abcd1234
curl -X PUT "localhost:9200/abcd1234?pretty" -H 'Content-Type: application/json' -d'
{
"settings": {
"index": {
"number_of_shards": 1,
"number_of_replicas": 0
}
},
"mappings": {
"properties": {
"attachment.data": {
"type": "binary"
}
}
}
}'
Create the attachment pipeline
curl -X PUT "localhost:9200/_ingest/pipeline/cbor-attachment?pretty" -H 'Content-Type: application/json' -d'
{
"description" : "Extract attachment information encoded in CBOR",
"processors" : [
{
"attachment": {
"description" : "Ingest attachment",
"field": "data",
"indexed_chars": -1
}
}
]
}'
Attempt to index the sample file
curl -X PUT "localhost:9200/abcd1234/_doc/sample?pipeline=cbor-attachment" -H 'Content-Type: application/cbor' -d'
{
"data": "x)The quick brown fox jumps over a lazy dog"
}'
Any hints on what I might be doing wrong?