Log4Perl: How do I change the logger file used from running code? (After a fork)

Question

I have an ETL process set up in perl to process a number of files, and load them to a database.

Recently, for performance reasons I set up the code to be multi-threaded, through use of a fork() call and a call to system("perl someOtherPerlProcess.pl $arg1 $arg2").

I end up with about 12 instances of someOtherPerlProcess.pl running with different arguments, and these processes each work through one directories worth of files (corresponding to a single table in our database).

The applications main functions work, but I am having issues with figuring out how to configure my logging.

Ideally, I would like to have all the someOtherPerlProcess.pl share the same $log_config value to initialize their loggers, but have each of those create a log file in the directory they are working on.

I haven't been able to figure out how to do that. I also noticed that in the directory I am calling these perl scripts from I see several files (ARRAY(0x260eec), ARRAY(0x313f8), etc) that contain all my logging messages!

Is there a simple way to change the log4perl.appender.A1.filename value from running code? Or to otherwise dynamically configure the file name we use, but use all other values from a config file?

Answer 1

I came up with a less than ideal solution for this, which is to configure my logger from someOtherPerlProcess.pl directly.

my $FORKED_LOG_CONF = "log4perl.appender.A1.filename=$directory_to_load/log.txt
log4perl.rootLogger=WARN, A1
log4perl.appender.A1=Log::Log4perl::Appender::File
log4perl.appender.A1.mode=append
log4perl.appender.A1.autoflush=1
log4perl.appender.A1.layout=PatternLayout
log4perl.appender.A1.layout.ConversionPattern=[%p] %d{yyyy-MM-dd HH:mm:ss}: %m%n";

#Logger start up
Log::Log4perl::init( \$FORKED_LOG_CONF);
my $logger = get_logger();

The $directory_to_load is the process specific portion of the logger, which works in the context of the perl process that is running and has a (local) value for that variable, but that method will fail if used in an external config file.

I would be happy to hear of any alternative solutions.

Answer 2

In your config file:

log4perl.appender.A1.filename=__LOGFILE__

In your script:

use File::Slurp;
my $log_cfg = read_file( $log_cfgfile );
my $logfile = "$directory_to_load/log.txt";
$log_cfg =~ s/__LOGFILE__/$logfile/;
Log::Log4perl::init( \$log_cfg );