I am creating the elastic sink connector in confluent.cloud(Kafka). here, if I use transforms, it is not working. will confluent.cloud(kafka) supports transforms?.
Below is my script,
{
"name": "ElasticConnectorTest5",
"config": {
"topics": "enterprise.contact6.model",
"input.data.format": "AVRO",
"connector.class": "ElasticsearchSink",
"name": "ElasticConnectorTest5",
"kafka.api.key": "xxxxxxxxxxxxx",
"kafka.api.secret": "xxxxxxxxxxxxxxxxxx",
"connection.url": "url",
"connection.username": "**********",
"connection.password": "*********",
"type.name": "_doc",
"key.ignore": "false",
"schema.ignore": "true",
"tasks.max": "1",
"errors.tolerance": "all",
"errors.log.enable": "true",
"errors.log.include.messages": "true",
"behavior.on.malformed.documents": "warn",
"transforms": "InsertMetadata,IndexName",
"transforms.InsertMetadata.type": "org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.InsertMetadata.partition.field": "partition",
"transforms.InsertMetadata.offset.field": "offset",
"transforms.IndexName.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.IndexName.regex": "enterprise.(.*)",
"transforms.IndexName.replacement": "es.$1"
}
}
Single Message Transforms (SMTs) are now supported by Confluent Cloud (Oct 2021):
Note, there is word "most" in the announcement and some transforms are not yet supported:
As well as some specific limitations for Elasticsearch Service Sink Connector for Confluent Cloud.