I'm using Kafka 2.2.0, so the option to configure max.compaction.lag.ms
per topic is not yet available.
Globally I have log.cleaner.max.compaction.lag.ms=86400000
(one day):
For the topic __consumer_offsets
, I'd like to delay the compaction to at least three days, to keep some short-term history.
I could set min.compaction.lag.ms
to 259200000
(three days), but it seems contrary to the smaller log.cleaner.max.compaction.lag.ms
. What would happen if I did so nonetheless?
The compaction will be performed if either happens:
min.compaction.lag
are fulfilledor
max.compaction.lag
is fulfilledSo it is OK to have
min.compaction.lag.ms
bigger thanmax.compaction.lag.ms
. Since they are used in 2 different conditions and do not contradict each other.BTW, what did you choose in the end? What were your results?