I have two apps App 1 and App 2. Both are Rails App. App 1 collects the data from the user and these collected data need to synced to App 2. I use RabbitMQ and Sneakers (with 5 worker processes) for the syncing process between Apps. Whenever there is some update in App 1, A job gets queued to RabbitMQ server. From there, Sneakers processes these jobs and stores the data in App 2 Database.
Architecture used for storing the data in both Apps is as follows.
App 1
Author has many Books.
Author Table:
id name
1 Kruze
2 Jiht
3 Rama
Book Table:
id name author_id
1 Book A 1
2 Book B 2
3 Book C 1
App 2
Author Table:
id name original_id books
1 Kruze 1 {'1':{'id':1, 'name':'Book A'}, '3':{'id':3, 'name':'Book C'}}
2 Rama 3 {}
3 Jiht 2 {'2':{'id':2, 'name':'Book B'}}
Here original_id column is App 1 author record id and books column is a hash containing all books' information of an author.
Problem I am facing here was whenever there were multiple updates on the same author's books (create or update) on the same time by multiple users, there were multiple jobs gets en-queued to RabbitMQ server. In App 2, Sneakers processes these jobs parallelly (five jobs at a time since there were five sneakers processes running in App 2). Each sneakers process is taking different time to commit. So at the end, wrong data is present in App 2.
Enqueue Order Commit Order Process
1 1 1
2 2 2
3 5 5
4 6 1
5 4 4
6 3 3
7 7 2
8 10 5
9 9 4
10 8 3
But when there is only one Sneakers process, this issue does not occur.
My question: Is there is any way to overcome this incorrect data issue?