I am in the process of pushing the logs
- Application logs to 'Fluentd'
- From 'Fluentd' to Dynatrace.
The application is Spring Boot application which is deployed as Azure App Service.
In the application, Fluentd is configured with IP address and port. In logback-spring.xml the appender is configured.
<appender name="FLUENTD" class="ch.qos.logback.more.appenders.DataFluentAppender">
<remoteHost>${fluentdIP}</remoteHost>
<port>${fluentdPort}</port>
</appender>
Fluentd is serving as container in Azure Container Instance and the configuration of Fluentd output contains Dynatrace URL. fluentd.conf is the configuration file which has the output API and token details of dynatrace.
The application logs are pushed to Dynatrace successfully.
The application becomes unresponsive daily. Restarting the App Service temporarily restores functionality, suggesting a potential resource allocation problem.
My suspicion lies with the internal threads allocated by the web server to connect to Fluentd. These threads might be becoming fully occupied, preventing the server from processing new requests.
Stack Trace:
org.fluentd.logger.sender.RawSocketSender
java.net.SocketTimeoutException: Connect timed out
Any heads up would be much appreciated.