Thingsboard - is there a way to use the rule engine to calculate percentile values?

Question

We want to calculate the percentile values from a timeseries of data from a group of devices.

Each device transmits data every 10 seconds. We average over a period of 5 minutes. For each 5 minute period we wish to know the 5,10,50,90, 95% percentile values.

I've had a look at rule engine node functions and perhaps the Analytics or Math script nodes could do this, but it's not clear.

Or if not the rule engine, can a function be implemented in a dashboard widget?

Looking for some guidance on what is possible. Thanks.

I'm trying out the rule engine node functions for Analytics aggregation, which can do average. But not clear if percentiles are possible.

Answer 1

Save that data when it comes to the system (Save timeseries).

Generator on every 5 minutes -Success-> API call rule node "Get time-series data (getTimeseries)" (look at Swagger for details) onto that device which has telemetry. -Success-> blue script rule node in which you will get your device telemetry in msg, and therefore you can do any math over it -Success-> save timeseries

It is possible to do the "same thing" on dashboard, but with much more skill and effort.

Answer 2

Yes the function can be implemented in Thingsboard widget, that's my way of calculating bombarding type of data but in the rule chain use some API node to call the data so no data will be lost and the calculation will be exact according to the data that is saved in the database.

Answer 3

You could achieve that leveraging the OriginatorTelemetry Node

The set up of the node could be like this:

Time-series key : yourVariableName
Fetch mode : ALL
Start interval : 1ms
End interval : 5min

The action of this is to create a metadata "yourVariableName" history with a structure like this:

{
    "yourVariableName" : "[{\"ts\":1703733802000,\"value\":3.5},{\"ts\":1703733819000,\"value\":3.42},{\"ts\":1703733820000,\"value\":3.77},{\"ts\":1703733821000,\"value\":3.51},...]
    // other_metadata
}

This is an scaped string, so you must apply the proper parsing to use it, and then process it as you desire. Something like

var array = JSON.parse(metadata.yourVariableName);
// Do some maths: sort array, calc percentile indexes, take each percentile value,...

Hope being of help.