Our infra for web application looks like this
Nodejs Web application -> GraphQL + Nodejs as middleware (BE for FE) -> Lot's of BE services in ROR -> DB/ES etc etc
We have witness the whole middleware layer of GrpahQL+Nodejs gets latent whenever any of the multiple crucial BE service gets latent and request queuing starts happening. When we tried to compare it with number of requests during the period it got latent it was <1k request which is much lower than the claimed 10k concurrent request handling of nodejs. Looking for pointers to debug this issue further. Analysis done so far from our end:
- As per Datadog and other APM which are used to to monitor system health, CPU and memory usage have shown no abnormal behaviour when the servers gets latent
- We are using various request tracking methods from top most layer to last layer, and it is confirmed that request queuing is happening on this middleware layer only.