How to add zapcore.NewMultiWriteSyncer in zap.Config using golang zap log project

Question

I am working on a log package and hope to configure whether to use log rotation through switches.

Previous code did not support log rotate

// using zap.Config and cannot log rotate

// Build constructs a global zap logger from the Config and Options.
func (o *Options) Build() error {
    var zapLevel zapcore.Level
    if err := zapLevel.UnmarshalText([]byte(o.Level)); err != nil {
        zapLevel = zapcore.InfoLevel
    }
    encodeLevel := zapcore.CapitalLevelEncoder
    if o.Format == consoleFormat && o.EnableColor {
        encodeLevel = zapcore.CapitalColorLevelEncoder
    }

    zc := &zap.Config{
        Level:             zap.NewAtomicLevelAt(zapLevel),
        Development:       o.Development,
        DisableCaller:     o.DisableCaller,
        DisableStacktrace: o.DisableStacktrace,
        Sampling: &zap.SamplingConfig{
            Initial:    100,
            Thereafter: 100,
        },
        Encoding: o.Format,
        EncoderConfig: zapcore.EncoderConfig{
            MessageKey:     "message",
            LevelKey:       "level",
            TimeKey:        "timestamp",
            NameKey:        "logger",
            CallerKey:      "caller",
            StacktraceKey:  "stacktrace",
            LineEnding:     zapcore.DefaultLineEnding,
            EncodeLevel:    encodeLevel,
            EncodeTime:     timeEncoder,
            EncodeDuration: milliSecondsDurationEncoder,
            EncodeCaller:   zapcore.ShortCallerEncoder,
            EncodeName:     zapcore.FullNameEncoder,
        },
        OutputPaths:      o.OutputPaths,
        ErrorOutputPaths: o.ErrorOutputPaths,
    }
    
    logger, err := zc.Build(zap.AddStacktrace(zapcore.PanicLevel))
    if err != nil {
        return err
    }
    zap.RedirectStdLog(logger.Named(o.Name))
    zap.ReplaceGlobals(logger)

    return nil
}

Now I hope to support rotation. I checked the relevant information and found that it can be rotated in the following way

//   logrotate.log
syncer :=zapcore.NewMultiWriteSyncer(zapcore.AddSync(os.Stdout), zapcore.AddSync(lumberJackLogger))   
// lumberJackLogger encapsulate lumberjack.Logger

core := zapcore.NewCore(
        zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig()),
        syncer,
        logLevel,
    )
logger := zap.New(core, zap.AddCaller(), zap.AddStacktrace(zap.ErrorLevel))

I hope to change OutputPaths and ErrorOutputPaths into the form of logrorate.go when starting rotation based on config.

Just like the following pseudo code

if ! o.EnableRorate
  zap.Config{...OutputPaths,ErrorOutputPaths}
else
   zap.config add zapcore.NewMultiWriteSyncer

Please tell me what I need to do to achieve my expectations. Looking forward to your reply

PS: I hope that other configurations besides o.OutputPaths and o.ErrorOutputPaths can still be used under the new solution.

Answer 1

It seems to be a weird mix of concerns having a logger package do log rotation. The life cycle of an app is almost always less than the life cycle of log files on disk, and logging to disk is just one of many logger outputs. Take a look at zap.Sink. Nowhere is a file even remotely implied, it's basically just an io.Writer.

If you're not in control over the machine on which your app runs, then ask your platform person to introduce logrotate to the system. This is a pretty stable tool from the 80s, still active and used all over the place. It will allow you to write, care-free from zap, and let logrotate care about size/lines/whatever.

It could be that we have different interpretations of what "log rotation" means, and I have interpreted it to be a constraint on log file size, and some archival function (logrotate gzips by default). If this differs from your interpretation, then forgive me.

Answer 2

I searched and found that it can be achieved using zap.WrapCore.

// hack replace zap config build core with lumberjack logger
func (o *Options) replaceLogCore(conf zap.Config) zap.Option {
    var enc zapcore.Encoder
    // Copy paste from zap.Config.buildEncoder.
    switch conf.Encoding {
    case jsonFormat:
        enc = zapcore.NewJSONEncoder(conf.EncoderConfig)
    case consoleFormat:
        enc = zapcore.NewConsoleEncoder(conf.EncoderConfig)
    default:
        panic("unknown encoding")
    }

    logSyncer := zapcore.AddSync(&lumberjack.Logger{
        Filename:   o.RotateLogPath,
        MaxSize:    o.RotateMaxSize, //MB
        MaxBackups: o.RotateMaxBackups,
        MaxAge:     o.RotateMaxAge, //days
        LocalTime:  o.RotateEnableLocalTime,
        Compress:   o.RotateEnableCompress,
    })

    return zap.WrapCore(func(zapcore.Core) zapcore.Core {
        return zapcore.NewCore(enc, logSyncer, conf.Level)
    })
}

reference