During the process of data discrepancy, I realised some events are missed and not written to kafka during CDC. These are the steps I did:
1- Setup two different debezium connector each fetch data from similar mysql table and write to two different topic of kafka.
2- Activate log on side of consumer application to have all received events in form of {"op": operation_name, "before":{...}, "after":{...}}
3-Each consumer writes the result to different tables in postgres
4-After long running test, I saw there are some records in both postgres tables which are not synced with the source mysql table and even log table doesn't receive it's even. The number of missing records are the same at the two postgres tables(pipelines)
So I concluded that missing record issue may not relate to consumer application since there are two applications working in parallel and have the same result.
I want to see if there are any cases when debezium pass over a specific position in binlog file(skip it) or are there any events that mysql binlog file can not record it for-example so binlog miss the record?
Also note that CDC read data from a slave node of mysql (not master) but master and replica is synced and the events that are missed can be found at slave node db.