A Logback FileAppender that uses a single log file and deletes old log files

Question

I'm using logback, configured in a logback.groovy file, to manage logs for my applications. I want to:

1. Create log files that are timestamped at the application launch, and retained as a single log file for the life of the application. (We might have several instances of an application running at the same time, or multiple instance run during the course of a day, and they may be running for several days.)
2. Keep a clean log file directory, such that logs that are older than a given period are deleted.

Achieving the first would suggest using a FileAppender, along the following lines - however, this does not delete the old log files:

appender("FILE", FileAppender) {
  file = "path/to/log/dir/log_file_${date}.log"
}

Achieving the second would suggest using a RollingFileAppender with a TimeBasedRollingPolicy, along the following lines - which keeps the log files for 7 days. However, this will use a single file for logging all instances of an application on a given date, regardless of when the application was run:

appender("FILE", RollingFileAppender) {
    rollingPolicy(TimeBasedRollingPolicy) {
        fileNamePattern = "path/to/log/dir/log_file_%d{yyyyMMdd}.log"
        maxHistory = 7;
    }
}

How can I have my cake and eat it - i.e. get the benefits of a single log file per application run (with startup timestamp), but with the historical clean up benefits of the RollingFileAppender/TimeBasedRollingPolicy?

Answer 1

To the extent that it may be useful, one can do this by creating a custom FileAppender, along the following lines:

public class TidyFileAppender<E> extends FileAppender<E> {

  protected File directory;

  @Override
  public void start() {
    if (conditions to determine historical files to be deleted) {
      File[] oldFiles = directory.listFiles(new FileFilter() {
        @Override
        public boolean accept(File file) {
          // return true if file is 'old'
        }
      });
      if (oldFiles != null) {
        for (File oldFile : oldFiles) {
          if (!oldFile.delete()) {
            addWarn("Failed to delete old log file: " + oldFile);
          }
        }
      }
    } else {
      addWarn("Cannot tidy historical logs...");
    }
    super.start();
  }
}

Answer 2

Expanding on the previous answer, this is a concrete and working Kotlin implementation:

import ch.qos.logback.core.FileAppender
import java.io.File

class TidyFileAppender<E>: FileAppender<E>() {
    /** Defaults to the parent dir of the current logfile. */
    var directory: String? = null
    /** How many previous files to keep, by last modified attribute. */
    var maxHistory: Int = 20
    /** Threshold for extra files that may be kept to reduce file system accesses. */
    var threshold: Int = 0
    
    override fun start() {
        if (directory == null)
            directory = File(fileName).parent
        File(directory).list()?.let { files ->
            if (files.size > maxHistory + threshold) {
                files.map { File(directory, it) }
                        .sortedBy { it.lastModified() }
                        .take(files.size - maxHistory)
                        //.also { println("Removing $it") }
                        .forEach(File::delete)
            }
        }
        super.start()
    }
}

Usage example:

  <timestamp key="time" datePattern="MM-dd'T'HHmm"/>
  <appender name="FILE" class="TidyFileAppender">
    <file>${LOG_DIRECTORY:-log}/game-server_${time}.log</file>
    <maxHistory>15</maxHistory>

    <layout class="ch.qos.logback.classic.PatternLayout">
      <Pattern>%d{HH:mm:ss} %-5level %36logger{36} - %msg%n</Pattern>
    </layout>
  </appender>