logback: TimeBasedRollingPolicy daily rolling works, but all old log files are deleted

Question

I have used RollingFileAppender with a TimeBasedRollingPolicy, so that at midnight a new logfile is created and the old logs 10 days are to be kept. At midnight a new logfile is created, but the old one will be deleted immediately, sometimes even after a few minutes.

public static void init(String logfilename,String loggername,String pattern) throws Exception {
  LoggerContext context = (LoggerContext)LoggerFactory.getILoggerFactory();

  logfilename = new File(logfilename).getCanonicalPath();

  PatternLayoutEncoder encoder = new PatternLayoutEncoder();
  encoder.setContext(context);
  encoder.setPattern(pattern);
  encoder.start();

  RollingFileAppender<ILoggingEvent> rollingFileAppender = new RollingFileAppender<>();
  rollingFileAppender.setContext(context);
  rollingFileAppender.setName(loggername);
  rollingFileAppender.setEncoder(encoder);
  rollingFileAppender.setAppend(true);
  rollingFileAppender.setFile(logfilename+".log");

  TimeBasedRollingPolicy<ILoggingEvent> rollingPolicy = new TimeBasedRollingPolicy<>();
  rollingPolicy.setContext(context);
  rollingPolicy.setParent(rollingFileAppender);
  rollingPolicy.setFileNamePattern(logfilename+".%d{yyyy-MM-dd}.log");
  rollingPolicy.setMaxHistory(10);
  rollingPolicy.setTotalSizeCap(FileSize.valueOf("100MB"));
  rollingPolicy.setCleanHistoryOnStart(false);
  rollingPolicy.start();

  rollingFileAppender.setRollingPolicy(rollingPolicy);     
  rollingFileAppender.start();

  Logger logger = (Logger)LoggerFactory.getLogger(loggername);
  logger.setAdditive(false);
  logger.setLevel(Level.INFO);
  logger.detachAndStopAllAppenders();
  logger.addAppender(rollingFileAppender);
}

I think in this case MaxHistory should mean 10 Days, but it don't work.

Answer 1

I have found the reason. It was my mistake. I had an old Process which has cleaned the directory every hour. :). After disabling, logback works as expected. You can use the code above as example. it works fine.

Answer 2

I've create a minimal running demo based on your code:

https://github.com/riskop/slf4j_logback_rollingfileappender_programatically

I've just lowered totalSizeCap to 1KB, changed fileNamePattern to roll minutely and created a main method to drive the configuration:

public static void main(String[] args) throws Exception  {

    init2("A_LOG_FILE", "A_LOGGER_NAME", "%d{HH:mm:ss.SSS} %-5level %logger - %msg%n");

    while (true) {
        org.slf4j.Logger logger = LoggerFactory.getLogger("A_LOGGER_NAME");
        logger.warn("a warning " + System.currentTimeMillis());
        Thread.sleep(1000);
    }
}

public static void init2(String logfilename,String loggername,String pattern) throws Exception {
    LoggerContext context = (LoggerContext)LoggerFactory.getILoggerFactory();

    logfilename = new File(logfilename).getCanonicalPath();

    PatternLayoutEncoder encoder = new PatternLayoutEncoder();
    encoder.setContext(context);
    encoder.setPattern(pattern);
    encoder.start();

    RollingFileAppender<ILoggingEvent> rollingFileAppender = new RollingFileAppender<>();
    rollingFileAppender.setContext(context);
    rollingFileAppender.setName(loggername);
    rollingFileAppender.setEncoder(encoder);
    rollingFileAppender.setAppend(true);
    rollingFileAppender.setFile(logfilename+".log");

    TimeBasedRollingPolicy<ILoggingEvent> rollingPolicy = new TimeBasedRollingPolicy<>();
    rollingPolicy.setContext(context);
    rollingPolicy.setParent(rollingFileAppender);
    rollingPolicy.setFileNamePattern(logfilename+".%d{yyyy-MM-dd_HH-mm}.log");
    rollingPolicy.setMaxHistory(10);
    rollingPolicy.setTotalSizeCap(FileSize.valueOf("1KB"));
    rollingPolicy.setCleanHistoryOnStart(false);
    rollingPolicy.start();

    rollingFileAppender.setRollingPolicy(rollingPolicy);     
    rollingFileAppender.start();

    Logger logger = (Logger)LoggerFactory.getLogger(loggername);
    logger.setAdditive(false);
    logger.setLevel(Level.INFO);
    logger.detachAndStopAllAppenders();
    logger.addAppender(rollingFileAppender);
}

However, it's working fine. The above code results in one or max 2 files, according to the very low capSize. If I raise the capSize to 10KB then it results in 3 - 4 files.

So I guess that your problem is caused by your environment somehow.

Could you try out the above code?

Could you try to run it within your environment (file system, directory).

If I were you, I'd also try to switch off maxHistory, totalCapSize and cleanHistory settings.