Plc4x v. 0.8.0 Kafka connector returns 'null' when pushing a new connector

Question

I recently updated the Plc4x Kafka Connect plugin from version 0.5.0 to 0.8.0 by cloning from source from Plc4x's Github page and built it using Maven, exactly as specified in the Readme. After building the source I received a uber JAR which contains all the neccesary libraries and dependencies for the Kafka Connect plugin. I then create a Connector configuration file that looks like the example from Plc4x's Github page:

{
 "name":"plc-source-test",
  "config": {
   "connector.class":"org.apache.plc4x.kafka.Plc4xSourceConnector",
   "tasks.max":"1", 
   "file":"test.sink.txt", 
   "topics":"connect-test" 
  }
}

I then push the configuration to the REST interface:

curl -X POST -H "Content-Type: application/json" --data config.json http://localhost:8083/connectors

The REST interface now responds with:

{"error_code":500,"message":null}

And this is where I'm stuck. I believe this error has to do with the following line in the config.json file:

"connector.class":"org.apache.plc4x.kafka.Plc4xSourceConnector"

because when I use a different connector class, such as:

"connector.class":"FileStreamSinkConnector"

everything works fine and I can successfully push my Connector configuration to the REST interface. This problem did not occur in the 0.5.0 version of Plc4x either. I have unpacked the uber JAR containing all the dependencies and verified that the Plc4xSourceConnector class do exist. I don't know what I'm doing wrong since I'm following the steps outlined in their Github page to build and configure everything. Has anyone else experienced this issue?

Answer 1

I managed to find a solution to my question. The connector configuration file I used was designed for PLC4x v.0.4.0 and it had been updated prior to the release of v.0.8.0. I found an example configuration file in the PLC4X Github repo and used it as layout. And from their website I found that the field:

"sources.machineX.connectionString"

must be formatted in a special way. I updated my connector configuration file to this:

{"name": "plc-source-test",
 "config": {
  "connector.class": "org.apache.plc4x.kafka.Plc4xSourceConnector",
  "default-topic": "test-topic",
  "tasks.max": "1",
  "sources": "machineA",
  "sources.machineA.connectionString": "s7:<PLC_IP>?remote-rack=0&remote-slot=0",
  "sources.machineA.jobReferences": "jobA",
  "sources.machineA.jobReferences.jobA": "job-topic",
  "jobs": "jobA",
  "jobs.jobA.interval": "500",
  "jobs.jobA.fields": "fieldA",
  "jobs.jobA.fields.fieldA": "%DB1.DBD1:REAL"
 }

and got everything working!

When using the Plc4xSourceConnector one must specify the required key:values in the connector configuration:

"default-topic" //Required. The default name for the Kafka topic

"tasks.max" //Not quite sure what this does, but it is in the example config so lets use it

"sources" //Required. It must be a comma separated list of your source

"sources.machineA.connectionString" //Required. The connection string to the machine/PLC that you want to talk to

"sources.machineA.jobReferences" //Required. A comma separated list of all the jobs that you wish to create for this machine

"sources.machineA.jobReferences.jobA" //Required. The Kafka topic name for data produced by jobA

"jobs" //Required. A comma separated list of all jobs you wish to create

"jobs.jobA.interval" //Not sure if this is required. Determines the polling rate of your created job

"jobs.jobA.fields" //Required. A comma separated list of the fields belonging to this job. A field is a machine/PLC register containing a value

"jobs.jobA.fields.fieldA" //Required. The address to the resource on your machine/PLC

The error messages that was displayed when posting a faulty connector configuration was very vague, often just some NullPointerException. So it took me some time analyzing the source code of PLC4X, especially this class, to figure out what fields were required.