Question
Problem: When querying Thanos for a graph at resolution lower than a certain threshold, Thanos will return data with missing data points when Prometheus does not (see screenshots later).
Setup: A kubernetes cluster. Thanos’s querier is configured to query a single Prometheus’s sidecar (I did this as a test, to be certain there would be no downsampling at play here.) The querier is launched without the --query.auto-downsampling
option.
Question: Why isn’t Thanos returning all the data Prometheus does?
Graphs
Thanos Query for a 12h range, with a 90s resolution
Same Query, but on Prometheus
Thanos Query for a 6h range, with a 90s resolution
(We get the same result on Prometheus)
Fixed it by upgrading Thanos from
0.19.0-rc.0
to0.20.0
…