We are running with Kafka 2.12-2.4.1 on Windows 2012 R2 OS and using in our application as messaging source. We found that after exact at 38hrs the Kafka Broker is getting down with below error message. We found that deleting all Kakfa temp logs and restarting the Kafka broker would solve this issue, but we are looking for permanent resolution for this as this needs to be done after every 38hrs.

Can someone help us with their expertise.

[2020-05-10 04:07:53,435] ERROR Failed to clean up log for __consumer_offsets-9 in dir C:\kafka\temp\kafka-logs due to IOException (kafka.server.LogDirFailureChannel)
java.nio.file.FileSystemException: C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.cleaned -> C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000
000000000.timeindex.swap: The process cannot access the file because it is being used by another process.
        at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
        at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
        at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
        at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
        at java.nio.file.Files.move(Unknown Source)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:795)
        at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:209)
        at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:497)
        at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2286)
        at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2286)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at kafka.log.Log.replaceSegments(Log.scala:2286)
        at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:605)
        at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:530)
        at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:529)
        at scala.collection.immutable.List.foreach(List.scala:392)
        at kafka.log.Cleaner.doClean(LogCleaner.scala:529)
        at kafka.log.Cleaner.clean(LogCleaner.scala:503)
        at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:372)
        at kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:345)
        at kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:325)
        at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:314)
        at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
Suppressed: java.nio.file.FileSystemException: C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.cleaned -> C:\kafka\temp\kafka-logs\__consumer_o
ffsets-9\00000000000000000000.timeindex.swap: The process cannot access the file because it is being used by another process.

                at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
                at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
                at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
                at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
                at java.nio.file.Files.move(Unknown Source)
                at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:792)
                ... 17 more
0

There are 0 best solutions below