I have an application in production. Angular UI---->UI Controller(.Net)----->Web API Controller(.Net)----->Database with dynatrace oneagent running in all servers.
I see spikes of specific calls in specific timeframes with response time more than a minute. As an example, One particular end to call takes 56 seconds instead of expected 5 seconds. On checking dynatrace method hotspots, I see more time is spent(around 55 seconds) on the UI Controller in ManualResetEventslim.wait before hitting API and database. at the end of 55 seconds, it reaches API and db which consume only 780 milliseconds to execute.
My question is: Why is it waiting in the UI Controller when it needs to simply call the API and db? 350 users use the application. On an average 45000 total web requests made per day What is causing the wait in UI controller? I cannot further drill down since dynatrace shows this method ManualResetEventslim.wait as dead end. I can dig in to my code if it highlights it but now I am stuck.
We might have been experiencing the same issues and we have a suspicion is dynatrace. Which version of OneAgent are you running? Do you have stack traces that relate to Dynatrace introspective and include this namespace: Dynatrace.OneAgent.Introspection
Our incident have been periodic and only lasted for around 10 minutes each time. We have had around 4 of such incident over a period of 4 days.
Additionally, we have seen some other very strange behaviour that indicate some sudden but permanent corruption of the running binaries whose symptoms are type violation. The exception included the following: