Python download script works successfully in headed mode fails to work in headless mode

Question

I am trying to figure out why my code doesn't get a download file in headless mode whereas running in regular/headed mode, I do get a download file. It used to work just fine up until this week.

This script opens up a public Google drive, then applies a list view and sort directions before selecting the first download link. Here's what I've typed up and copy/pasted from the inter webs:

chrome_options = webdriver.ChromeOptions()
    
chrome_options.add_argument("--headless")
chrome_options.add_argument("--disable-infobars")
chrome_options.add_argument("--window-size=1920,1080")
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--proxy-server='direct://'")
chrome_options.add_argument("--proxy-bypass-list=*")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--disable-dev-shm-usage')
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--ignore-certificate-errors')
    
prefs = {}
prefs["profile.default_content_settings.popups"]=0
prefs["download.default_directory"]=download_path
chrome_options.add_experimental_option("prefs", prefs)
    
browser = splinter.Browser('chrome',options=chrome_options)
    
browser.visit('https://drive.google.com/drive/folders/')
logging.debug("Google Drive website opened")
time.sleep(5)
    
WebDriverWait(browser.driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '[aria-label="List view"]'))).send_keys(keys.Keys.ENTER)
logging.debug("List view requested")
time.sleep(5)
WebDriverWait(browser.driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '[aria-label="Reverse sort direction"]'))).send_keys(keys.Keys.ENTER)
logging.debug("Sort with latest on top requested")
time.sleep(5)
WebDriverWait(browser.driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, '[aria-label="Download"]'))).send_keys(keys.Keys.ENTER)
logging.debug("Download requested")
time.sleep(5)

wait_for_downloads(browser.driver, download_path, headless=True)
    
logging.debug("Download completed")
    
browser.quit()
logging.debug("Browser Quit")

Answer 1

Pls try like below.

from selenium import webdriver
import time
from selenium.webdriver.chrome.options import Options

options = webdriver.ChromeOptions()
options.add_argument("headless")
driver = webdriver.Chrome(executable_path='C:\\temp\\chromedriver_win32\\chromedriver.exe', options=options)
driver.get('http://www.google.com')

Answer 2

I have abandoned trying to use chrome in headless mode to download from google drive. I switched to Firefox and it works in headless mode using the same code/methodology. I had some issues setting it up in Splinter so here's how I did that in case anyone else stumbles on this issue and cant get Google Headless to function as non-headless does:

#Solution below from https://github.com/cobrateam/splinter/issues/378#issuecomment-87162477

prof = {}
prof['browser.download.manager.showWhenStarting'] = 'false'
prof['browser.helperApps.alwaysAsk.force'] = 'false'
prof['browser.download.dir'] = download_path
prof['browser.download.folderList'] = 2
prof['browser.helperApps.neverAsk.saveToDisk'] = 'text/csv, application/csv, text/html,application/xhtml+xml,application/xml, application/octet-stream, application/pdf, application/x-msexcel,application/excel,application/x-excel,application/excel,application/x-excel,application/excel, application/vnd.ms- excel,application/x-excel,application/x-msexcel,image/png,image/jpeg,text/html,text/plain,application/msword,application/xml,application/excel,text/x-c,application/x-7z-compressed,application/zip'
prof['browser.download.manager.useWindow'] = 'false'
prof['browser.helperApps.useWindow'] = 'false'
prof['browser.helperApps.showAlertonComplete'] = 'false'
prof['browser.helperApps.alertOnEXEOpen'] = 'false'
prof['browser.download.manager.focusWhenStarting']= 'false'

browser = splinter.Browser('firefox',profile_preferences=prof, headless=True)