I'm working on migration/integration of large on-premise Oracle monolithic app to cloud based Microservices. For a long time, microservices will need to be fed from and synchronized with the Oracle DB.
One of the alternatives is using Oracle Golden Gate for DB-to-DB(s) near-real-time replication. The advantage is that it seems to be reliable and resilient. The disadvantage is that it works on low-level CDC/DB changes (as opposed to app-level events).
An alternative is creating higher level business events from source DB by enriching data and then pushing it to something like Kafka. The disadvantage is that it puts more load on source DB, and requires durability on the source.
Anybody dealt with similar problems? Any advice is appreciated.
The biggest problem for us has been that legacy data is on the LAN, and our microservices are in the public cloud (in an attempt to avoid a "new legacy" hybrid cloud future).
Oracle Goldengate for Big Data can push change records as JSON to Kafka/Confluent. There's also the option to write your own handlers. You can find a lot of our PoC code in github.
As time has gone by, it became apparent the number of feeds was going to end up in the 300+ range, and we're now considering a data virtualisation + caching approach rather than pushing the legacy data to the cloud apps