What happens when min.compaction.lag.ms is greater than log.cleaner.max.compaction.lag.ms?

3k Views Asked by At

I'm using Kafka 2.2.0, so the option to configure max.compaction.lag.ms per topic is not yet available.

Globally I have log.cleaner.max.compaction.lag.ms=86400000 (one day):

For the topic __consumer_offsets, I'd like to delay the compaction to at least three days, to keep some short-term history.

I could set min.compaction.lag.ms to 259200000 (three days), but it seems contrary to the smaller log.cleaner.max.compaction.lag.ms. What would happen if I did so nonetheless?

1

There are 1 best solutions below

3
On

The compaction will be performed if either happens:

  1. Both conditions for dirty/clean ratio and min.compaction.lag are fulfilled

or

  1. max.compaction.lag is fulfilled

So it is OK to have min.compaction.lag.ms bigger than max.compaction.lag.ms. Since they are used in 2 different conditions and do not contradict each other.

BTW, what did you choose in the end? What were your results?