I have a versioned data stored in a DB. Consumer request event data for a version, as event can have multiple versions consumer keep requesting for newer version on event data.
There are 2 services, service1 is external cusumer facing proxy service gets data from service2 and service2 fetches data from db.
Currently we are using .net7, restapi with protobuf between these service call.
Issues are
- cunsumer can request a version which is not present but soon it can be available, so in service2 we are doing long polling for such requests for 15sec refetch version every 0.5sec from db. Because of which that request is being hold on service2 and service1 waits for it.
- When we deploy something or do app pool recycle on service2, service1 returning 500 error to cunsumer for requests which entered in long polling on service2
Now I have a solution on my mind that if I move polling logic to service1 then problem will solve but http requests from service1 to service2 will surely increase.
Current load on service2 is huge. We are receiving at least 8k requests per sec on each node and we have 8nodes. I suspect that after moving polling to service1 will double the requests. Like 16k req per sec.