Selenium script that works perfectly in headed mode fails to work in headless mode (Python)

Question

I am trying to download a file using Selenium (Chrome Driver). I used a test website that contains a small sample audio file to download. The script works fine when used in a headed mode and the file downloads but fails in headless mode. I looked at some of the related questions and made sure that I used all mentioned options for the Chrome Driver but the following error pops up multiple times

I also tried the answer mentioned here but it didn't work

[0125/104422.896:INFO:CONSOLE(0)] "Refused to execute script from 'https://filesamples.com/detroitchicago/anaheim.js?gcb=2&cb=1' because its MIME type ('image/gif') is not executable.", source: https://filesamples.com/formats/mp3 (0)

My code is as follows

download_dir = "path/to/downloads/"
driver_path = "path/to/driver"

chrome_options = Options()

chrome_options.add_experimental_option("prefs", {"download.default_directory": download_dir,
"download.prompt_for_download": False,
})

chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=1920,1080")
chrome_options.add_argument("user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36")

driver = webdriver.Chrome(executable_path=driver_path,chrome_options=chrome_options)
driver.command_executor._commands["send_command"] = ("POST", '/session/$sessionId/chromium/send_command')
params = {'cmd': 'Page.setDownloadBehavior', 'params': {'behavior': 'allow', 'downloadPath': download_dir}}
command_result = driver.execute("send_command", params)

base_url = "https://filesamples.com/formats/mp3"
driver.get(base_url)
xpath = "/html/body/div[1]/main/div[2]/div[1]/a"
driver.find_elements_by_xpath(xpath)[0].click()

There's no proper traceback, but I am fairly confident that the error is because of the driver.get() method because commenting out that line did not produce it

Chrome version - 87.0.4280.141
Chrome Driver version - 87.0.4280.88

Any help would be appreciated. Thanks!

Answer 1

options.add_argument("--headless")
options.add_argument("--window-size=1920, 1080")
options.add_argument(
    "user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.88 Safari/537.36")

driver = webdriver.Chrome(options=options)

you should set window size also