CDC in Seatunnel with Phoenix DB

20 Views Asked by At

I successfully integrated Seatunnel with Phoenix DB and conducted a test on the batch mode job, which yielded positive results. However, when attempting to test the streaming job of Seatunnel with Phoenix, I encountered the same outcome as the batch mode job. Streaming is not working in phoenix. I would greatly appreciate your assistance in resolving this issue. Although no error exceptions are occurring, the streaming mode is behaving similarly to the batch mode. Specifically, it is transferring all the records from the source to the sink table, and the job is subsequently marked as completed My conf file details is mentioned below

env{
  #parallelism = 3
  job.mode = "STREAMING"
  job.name="test_hbase_source_to_hbase"
}

source {
    Jdbc {
        driver = org.apache.phoenix.jdbc.PhoenixDriver
        url = "jdbc:phoenix:instance-1:/hbase-unsecure"
        query = "select ROWID, MSISDN, ID, STATUS_FIELD from TEST_MERGE_SPLIT"
    }
}

sink{
    Jdbc {
        driver = org.apache.phoenix.jdbc.PhoenixDriver
        url = "jdbc:phoenix:instance-2:/hbase-unsecure"
        query = "upsert into TEST_MERGE_SPLIT(ROWID, MSISDN, ID, STATUS_FIELD) values(?,?,?,?)"
    }
}
0

There are 0 best solutions below