Troubleshoot Application Insights telemetry - a local log or have it surface exceptions?

72 Views Asked by At

I have a .NET Core application that does a number of interesting things: upon receiving a message on a Rabbit MQ bus it calls some external APIs (REST over HTTP) to obtain some data, asynchronously uploads several bigger files to Azure Storage and upon completion of one certain file upload (not necessarily last to finish uploading) it sends a completion message to a Rabbit MQ exchange and then waits for all uploads to complete and then resumes listening to incoming messages. Actually, you can freely ignore all the above, that was just to explain that events might occur in different order.

The whole application is instrumented with logging via Serilog which also has a trace sink towards the Application Insights. Additionally, the App. Insights TelemetryClient is used to wrap larger operations with App.Insights-specific telemetry and establish correlation with external calls to other services. I use the "classic" SDK (telemetryClient.StartOperation("my request event")).

What I observe is that under some very concrete combination of conditions (on a particular instance of Rabbit MQ bus, with a specific number of message headers - of all things!) the telemetry seizes to be sent to Application Insights, as if the client loses connectivity or something else bad happens. So far I flat out don't understand what is going on.

I tried to inspect the instance of TelemetryClient when I know it has "gone over to the bad side" and "visually" see no obvious corruption on it. Quite naturally, the telemetry instrumentation doesn't report any trouble in form of exceptions, either - I guess it is coded to create as little fuss as possible by itself. So, finally, the question is: how do I troubleshoot this? Can client-side instrumentation be instructed to either write a local log or to become more verbose in some other way (raise exceptions?). I am able to reproduce this situation in my local debugger while running the service in a local Docker container - anything I can do from the debugger?

0

There are 0 best solutions below