could kafka-connect distributed worker work with topic 'connect-offset' with configuration 'cleanup.policy=compact,delete'

Question

I started a kafka-connect distributed worker cluster which uses the topic connect-offset for offset storage:

offset.storage.topic=connect-offset

Since the broker is provided with default policy 'cleanup.policy=delete', when creating a topic with 'cleanup.policy=compact', I will get 'cleanup.policy=compact,delete' for this topic eventually. Which cause kafka-connect worker process to throw exception:

org.apache.kafka.common.config.ConfigException: Topic 'slpe-connect-offset' supplied via the 'offset.storage.topic' property is required to have 'cleanup.policy=compact' to guarantee consistency and durability of source connector offsets, but found the topic currently has 'cleanup.policy=compact,delete'. Continuing would likely result in eventually losing source connector offsets and problems restarting this Connect cluster in the future. Change the 'offset.storage.topic' property in the Connect worker configurations to use a topic with 'cleanup.policy=compact'.

Question: is there any kafka-connect worker config allow eating this exception to keep worker process? although it's a risk but delete won't happen until it hits either retention or size limit.

Answer 1

This validation was introduced in KAFKA-9216 (PR) and affects 2.3.2, 2.6.0, 2.4.2, 2.5.1 versions:

The current change helps guard against users running Connect with topics that have delete cleanup policy enabled, which will remove all connector configurations, source offsets, and connector & task statuses that are older than the retention time. This means that, for example, the configuation for a long-running connector could be deleted by the broker, and this will cause restart issues upon a subsequent rebalance or restarting of Connect worker(s). Connect behavior requires that its internal topics are compacted and not deleted after some retention time

Therefore, it highly unrecommeded to use delete clean policy for Kafka Connect interanl topics: just recreate those topics.

There is no any config property to ignore this validation, but if you want, you can downgrade Kafka Connect to a version without it.