I'm making my first steps in Python and have come to a point where I need a logging module. The reasons I didn't opt for a rotating file handler are:
- I wanted a new folder to be created each time the code is run, hosting the new log files.
- Using conventional filenames (
myName_X.log
, and not the defaultmyName.log.X
). - Limit log files by number of lines, and not file size (as done by the rotating file handler).
I've written such a module, using Python's built in logging module, but i have two problems:
- The new folder and file are created, and the logging data is printed into the file. However, when
main()
is run for the second time (see code below), the newly created file is locked, and cannot be deleted from the file explorer unless I close the IDE or release the lock through Process Explorer. - The IPython interpreter freezes the second time I run
main()
. If I try thepdb
module, it freezes as well.
I'm using WinPython 3.3.5 (with Spyder 2.3.0beta). I spent hours trying to find a solution to this problem. I don't know if it is a problem in my code or rather a bug with Spyder.
General coding remarks are always welcome.
main_example.py:
import myLogging
def main():
try:
myLoggerInstance = myLogging.MyLogger()
# Do stuff...
# logging example
for i in range(0, 3):
msg = 'Jose Halapeno on a stick {0}'.format(i)
myLoggerInstance.WriteLog('DEBUG', msg)
print('end of prints...')
finally:
myLoggerInstance._closeFileHandler()
print('closed file handle...')
if __name__ == "__main__":
main()
myLogging.py:
import logging
import time
import os
class MyLogger:
_linesCounter = 0
_nNumOfLinesPerFile = 100000
_fileCounter = 0
_dirnameBase = os.path.dirname(os.path.abspath(__file__))
_dirname = ''
_filenameBase = 'logfile_{0}.log'
_logger = logging.getLogger('innerGnnLogger')
_severityDict = {'CRITICAL' : logging.CRITICAL, 'ERROR': logging.ERROR, 'WARNING':
logging.WARNING, 'INFO': logging.INFO, 'DEBUG': logging.DEBUG}
@staticmethod
def __init__():
# remove file handle
MyLogger._closeFileHandler()
# create folder for session
MyLogger._dirname = MyLogger._dirnameBase + time.strftime("\\logs_%Y_%m_%d-
%H_%M_%S\\")
MyLogger._dirname = MyLogger._dirname.replace('\\\\', '/')
if not os.path.exists(MyLogger._dirname):
os.makedirs(MyLogger._dirname)
# set logger
MyLogger._logger.setLevel(logging.DEBUG)
# create console handler and set level to debug
MyLogger._hConsole = logging.StreamHandler()
MyLogger._hFile = logging.FileHandler(MyLogger._dirname + \
MyLogger._filenameBase.format(MyLogger._fileCounter))
MyLogger._hConsole.setLevel(logging.WARNING)
MyLogger._hFile.setLevel(logging.DEBUG)
# create formatter
MyLogger._formatter = logging.Formatter('%(asctime)s %(filename)s, %(funcName)s, %(lineno)s, %(levelname)s: %(message)s')
# add formatter to handlers
MyLogger._hConsole.setFormatter(MyLogger._formatter)
MyLogger._hFile.setFormatter(MyLogger._formatter)
# add handlers to logger
MyLogger._logger.addHandler(MyLogger._hConsole)
MyLogger._logger.addHandler(MyLogger._hFile)
@staticmethod
def _StartNewFileHandler():
MyLogger._closeFileHandler()
# create new file handler
++MyLogger._fileCounter
MyLogger._hFile = logging.FileHandler(MyLogger._dirname + \
MyLogger._filenameBase.format(MyLogger._fileCounter))
MyLogger._hFile.setLevel(logging.DEBUG)
MyLogger._hFile.setFormatter(MyLogger._formatter)
MyLogger._logger.addHandler(MyLogger._hFile)
@staticmethod
def WriteLog(severity, message):
if (len(MyLogger._logger.handlers) < 2):
MyLogger._StartNewFileHandler()
MyLogger._linesCounter += 1
MyLogger._logger.log(MyLogger._severityDict[severity], message)
if (MyLogger._linesCounter >= MyLogger._nNumOfLinesPerFile):
MyLogger._logger.info('Last line in file')
MyLogger._StartNewFileHandler()
MyLogger._linesCounter = 0
@staticmethod
def _closeFileHandler():
if (len(MyLogger._logger.handlers) > 1):
MyLogger._logger.info('Last line in file')
MyLogger._logger.handlers[1].stream.close()
MyLogger._logger.removeHandler(MyLogger._logger.handlers[1])