kafka connect not loading manually built connectors

1.5k Views Asked by At

I downloaded aiven-kafka-connect-jdbc and built the jar manually.Placed the same jar in kafka-connect plugin directory and mentioned the plugin path in connect-standalone.properties. But when i run the connect it throws error saying "Failed to find any class that implements Connector and which name matches io.aiven.connect.jdbc.JdbcSourceConnector"

What is missing in my configuration?

connect-standalone.properties

bootstrap.servers=*******:9092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=true
value.converter.schemas.enable=true
offset.storage.file.filename=/Kafka/kafka_2.13-2.8.0/data/kafka/connect/offsets/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/Kafka/kafka_2.13-2.8.0/Plugins,/Kafka/kafka_2.13-2.8.0/libs,

connector.properties

name=aiventransconnector
connector.class=io.aiven.connect.jdbc.JdbcSourceConnector
connection.url=jdbc:oracle:thin:@****&&:1521/****?verifyServerCertificate=false&useSSL=true&requireSSL=true
connection.user=******
connection.password=******
table.whitelist= TRANSACTIONS
tasks.max= 1
auto.create= true
auto.evolve= true
mode=incrementing
incrementing.column.name=TRANSACTION_ID
numeric.mapping=precision_only
topic.prefix=AIVEN

Folder structure of kafka connect plugin

/Kafka/
  -kafka_2.13-2.8.0/
    -Plugins/
      -aiven-kafka-connect-jdbc/
        -lib/
          -aiven-kafka-connect-jdbc-6.7.0-SNAPSHOT.jar with all dependencies
2

There are 2 best solutions below

5
OneCricketeer On

Not able to reproduce...

Maybe one of these steps will help

$ cd ~/workspace
$ mkdir aiven-kafka-connect-jdbc
$ curl -kL https://github.com/aiven/aiven-kafka-connect-jdbc/releases/download/v6.6.0/aiven-kafka-connect-jdbc-6.6.0.tar | tar -xv -C ./aiven-kafka-connect-jdbc
$ grep -e '^plugin.path=' /usr/local/etc/kafka/connect-standalone.properties
plugin.path=/home/me/workspace/aiven-kafka-connect-jdbc
$ connect-standalone /usr/local/etc/kafka/connect-standalone.properties ~/workspace/sqlite-standalone.properties
...
[2021-08-18 11:07:24,260] INFO Registered loader: PluginClassLoader{pluginLocation=file:/home/me/workspace/aiven-kafka-connect-jdbc/aiven-kafka-connect-jdbc-6.6.0/} (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:269)
[2021-08-18 11:07:24,260] INFO Added plugin 'io.aiven.connect.jdbc.JdbcSourceConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:198)
[2021-08-18 11:07:24,261] INFO Added plugin 'io.aiven.connect.jdbc.JdbcSinkConnector' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader:198)
... 
[2021-08-18 11:07:26,276] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:57)
[2021-08-18 11:07:26,296] INFO JdbcConfig values:
    batch.max.rows = 100
    catalog.pattern = null
    connection.attempts = 3
    connection.backoff.ms = 10000
    connection.password = null
    connection.url = jdbc:sqlite:/home/me/workspace/test.db
0
swaggyp On

Make sure you are running on Java 11 or higher.

As stated at the top in their README, The Aiven GCS Sink Connector requires Java 11+, and your Java version is likely older than that.

You can check your current JVM by running java --version.

To download the latest version of Java visit here.