We currently utilize New Relic as the APM. However, the problem that I came across with New Relic (may be not necessarily New Relic specific) is that, I cannot create an Alert, let's say when there is steady rise in the used heap space even after a Garbage Collection. We though of using the data from New Relic to do it ourselves by polling for data at regular intervals and then used the data collected to extract the information that we need. However, we can't say when the GC triggers. I know that we can view it on the APM dashboard to see how is the collection pattern, but that is something that required manual intervention. So, instead we would like to automate this alerting based on the pattern that we see from the data gathered by New Relic.
So, before we go on and build a system like this, are there any existing tools in the market that can achieve the aforementioned goal. If it can be done by leveraging NRQL in New Relic or any other functionality within New Relic, then that would be ideal for us.
I found the answer on the New Relic discussions. The relevant minute by minute metrics like used heap, response time, throughput, Garbage collection CPU utilization, thread count, etc can be fetched using the New Relic API. This data then can be studied to establish the upward or downward trends or benchmarks for alerting. LINK to the post for further information.