I am currently sending json data from AWS Lambda to Datadog, via AWS Kinesis Firehose. The data has key value pairs, for example { "active_count": 10000, "deleted_count": 100, ...}. I have verified that the kinesis is sending the data properly to Datadog, since this data can be seen in the Logs Explorer of Datadog. This data will be sent once per day.
I now want to display these values as query widgets on the Datadog dashboard. For example, there will be a query widget with a title of Active Count, and displays a value of 10000, from the data in the log. I understand that the active_count can be turned into a facet, which can be used to create widgets out of it. However, I noticed that there is no straightforward way to display the 10000 value as a query widget. When I try to input the information to create the widget, I must put it in a query pattern, such as COUNT * of active_count. This will display 1, since there is only one unique active_count facet in the log; however, this is not what I need, I need the value associated with active_count in the json data in the log.
Thus, I need a way to display the value of the facet itself in the event attribute of the log.
One workaround I tried was, for example, to send a single json data per facet, and send that "value" times. For example, a json data of { "active_count": xxx }, where xxx can be any value, will be sent 10000 times. This way, when using the COUNT * of the query, it will count the number of logs with the active_count facet, which is 10000, and display it. This will be done for all other facets as well.
Although this method works, it is quite inefficient in terms of log buildup. In addition, I believe there are methods other than using logs to do this kind of work, but I am unfamiliar with it.
If anyone is aware of more efficient methods to do this in Datadog, please advise. Thank you.