selenium chromedriver: chrome.exe open the python logging filehandler file

Question

I set the logger handlers and then initiate the chrome webdriver. It get [Error 32] when it try to rotate the log file, which because it fail to rename the log file. And I find the log file is opened by Chrome.

I change the order of set logger and initiate chrome webdriver.It will successfully rotate the log file.When my script restart the chrome, the error happens again.I have to set logger.handlers = [] every time I start a new chrome webdriver instance.

I've try --disable-logging, but It doesn't work.

Why the chrome.exe open the log file of the python logging filehandler?

Is there any solution to avoid chrome to open the log file?

Here is my script.

import time
import logging
from selenium import webdriver
from logging.handlers import TimedRotatingFileHandler

logger = logging.getLogger('m')
def set_logger():
    logger.setLevel(logging.DEBUG)
    formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s: %(message)s',
                                  datefmt='%Y-%m-%d %H:%M:%S')
    rotate_file_handler = TimedRotatingFileHandler('D:\\data\\spam.log',
                                                   when='s', interval=10, backupCount=10)
    rotate_file_handler.setLevel(logging.INFO)
    rotate_file_handler.setFormatter(formatter)
    console_handler = logging.StreamHandler()
    console_handler.setLevel(logging.DEBUG)
    console_handler.setFormatter(formatter)
    logger.addHandler(rotate_file_handler)
    logger.addHandler(console_handler)

if __name__ == '__main__':
    # change the order these two, it will work correct.
    # or set logger.handlers = [], it will work too.
    set_logger()
    b = webdriver.Chrome()

    for i in range(100):
        logger.info(i)
        time.sleep(1)

Here is the Traceback.(Windows 7 SP1 x86_64, python=2.7.10, chrome=43.0.2357.81, chromedriver=2.15.322448).

Traceback (most recent call last):
  File "C:\Python27\lib\logging\handlers.py", line 77, in emit
    self.doRollover()
  File "C:\Python27\lib\logging\handler.py", line 350, in doRollover
    os.rename(self.baseFilename, dfn)
WindowsError: [Error 32]
Logged from test.py, line 28

Answer 1

I have just run into the same problem today. After some digging, I found what the problem is.

The root cause of the problem is in python itself. Before version 3.4, including 2.x brunches, python creates inheritable file descriptors by default (for almost all file related operations as I understand), and there is no way for user to change this behavior. When selenium chromedriver calls webdriver.exe with subprocess, logging file handler is inherited by the webdriver process, which will prevent the logging rollover operation and causes the error.

This issue is resolved in version 3.4, according to pep-0446.

It seems that the only solution to this problem is to upgrade to python 3.4 or above. For python 2.7 brunch, it seems that this issue has been marked as "won't fix" state, see issue26769.

A work-around for your script is to avoid using the TimedRotatingFileHandler, so it won't rotate and trigger the error.