How to query Apache Druid's __time with milliseconds precision (efficiently)?

170 Views Asked by At

This is my Apache Druid query that I ran in the Druid’s web GUI:

SELECT *
FROM my_table
WHERE __time >= '2023-10-19T09:29:58.613Z'
and __time <= '2023-10-19T09:30:13.613Z'
and my_string_field = 'value'

It returns a record where the value in the __time field is 2023-10-19T09:29:58.040Z which is less than the lower bound set in my query. The reason seems to be that strings, when parsed to Druid's time format, get truncated to seconds. I think so because the query SELECT TIME_PARSE('2023-10-19T09:29:58.613Z') results in 2023-10-19T09:29:58.000Z. So how do I filter column __time precisely up to milliseconds precision (without sacrificing the efficiency of searching by an indexed field)?

Another thing I don't understand is that if I add a field TIMESTAMP_TO_MILLIS(__time) as tms to my query it results in 1697794200930 for __time that is 2023-10-20T09:30:00.930Z, and if I add MILLIS_TO_TIMESTAMP(TIMESTAMP_TO_MILLIS(__time)) as mls I get back 2023-10-20T09:30:00.930Z, but when I add MILLIS_TO_TIMESTAMP(1697794200930) as mls_2 I get 2023-10-20T09:30:00.000Z instead of 2023-10-20T09:30:00.930Z (it gets truncated to seconds).

UPDATE:

I found out that the behavior of MILLIS_TO_TIMESTAMP was a bug and it was fixed in version 25.0.0.

1

There are 1 best solutions below

0
On BEST ANSWER

Turns out it was a bug and it was fixed. I installed version 27.0.0 locally and TIME_PARSE returns milliseconds now and filtering seems to account for milliseconds as well.