set config retention.ms=3600000 still data not delete from Kafka

Question

I have set the retention.ms=3600000 by below command but still there is lots of data on disk after 1 hour. My disk got full due to huge data coming to Kafka.

./bin/kafka-topics.sh --zookeeper zookeeper:2181 --alter --topic topic_1 --config retention.ms=3600000

Describe command

 ./bin/kafka-topics.sh --zookeeper zookeeper:2181 --describe --topics-with-overrides
    Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:3     Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
    Topic:topic_1    PartitionCount:3        ReplicationFactor:3     Configs:retention.ms=3600000
    Topic:topic_2    PartitionCount:3        ReplicationFactor:3     Configs:retention.ms=3600000
    Topic:topic_3    PartitionCount:3        ReplicationFactor:3     Configs:retention.ms=3600000,retention.bytes=104857600

Can anyone give advice why kafka not delete the data after 1 hours.?

Answer 1

From the describe command result, topic retention policy is set to compact which will enable log compaction instead of deleting and will keep the latest data for each key. To delete all the data older than the retention period, you need to set retention policy to delete.

./bin/kafka-topics.sh --zookeeper zookeeper:2181 --alter --topic topic_1 --config cleanup.policy=delete

Answer 2

As the documentation suggests, retention.ms controls the maximum time kafka will retain a log before it will discard old log segments to free up space if we are using the "delete" retention policy. Looks like your cleanup.policy is set to compact instead of delete

bin/kafka-configs.sh --zookeeper 2181 --entity-type topics 
--entity-name topic_1  --alter --add-config cleanup.policy=delete

PS:Altering topic configuration from the kafka-topics.sh script (kafka.admin.TopicCommand) has been deprecated. Going forward, please use the kafka-configs.sh script (kafka.admin.ConfigCommand) for this functionality.

Answer 3

Check the value of log.retention.check.interval.ms.
This value affects the Log cleaner. It will check whether any log is eligible for deletion with this interval.