Using the interpolate() function between all points

Question

I'm investigating using VictoriaMetrics for some metrics collected from remote sites with intermittent network connectivity. So the data will have gaps. But some of the metrics are counters that only increase.

For these counters, we would like to use the interpolate() function to calculate the increase per hour or day.

I've inserted some test data in my VM instance:

Time    Counter
00:30   2
01:15   5
01:30   10
01:45   12
02:30   15
04:15   22
04:45   24

When I try to query it I get this result:

Between 2:30 and 4:15 I can see the interpolate() function interpolates values. But it seems like it interpolates between 3:20 and 4:15 instead of 2:30 and 4:15. It seems the data point at 2:30 is extended until 3:20. And the same happens between 01:45 and 02:30 where no interpolation is happening.

How do I get it to interpolate between each data point?

Answer 1

VictoriaMetrics tries detecting the interval between samples (aka scrape_interval in Prometheus world) per each time series. The interval is determined as a median interval between raw samples on the selected time range. The interval is used for filling gaps smaller than 2x of the detected interval. This logic works great when the interval between samples is close to constant. The logic can fail when the interval between raw samples is irregular as in the case above.

In this case the default gap-filling logic can be adjusted by wrapping the time series selector into last_over_time() function. This function limits the gap filling with the duration specified in square brackets. For example, last_over_time(intertest[5m]) stops filling gaps bigger than 5 minutes. So, interpolate(last_over_time(intertest[5m])) should give the expected result for the case above.