Do driver.quit() and pkill chrome have the same effect in releasing memory?

560 Views Asked by At

I have a Python script that uses Selenium to do some web-page clicking and scraping. Script is running on Ubuntu, running on an EC2 instance. The basic code:

from selenium import webdriver
from bs4 import BeautifulSoup as bs
import datetime
from datetime import datetime as dt
import re
from selenium.webdriver.chrome.options import Options
from selenium.common.exceptions import ElementNotVisibleException
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import TimeoutException
    
from selenium.webdriver.common.by import By
    
#Set driver options
options = Options()
options.add_argument('--no-sandbox')
options.add_argument('--window-size=1420,1080')
options.add_argument('--headless')
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--disable-gpu')
options.add_argument("--disable-notifications")

options.add_experimental_option("excludeSwitches", ["enable-automation"])

options.add_experimental_option('useAutomationExtension', False)
    
driver = webdriver.Chrome(chrome_options=options, executable_path=r'/Applications/chromedriver_91')

# Do a bunch of stuff................

driver.quit() #invoke after web-scraping

Does driver.quit(), in this instance, essentially do the same thing as the Linux command pkill chrome? Sometimes, this script will crash because there's not enough memory. Using pkill chrome in combination with pkill -f "(chrome)?(--headless)" in the terminal itself usually kills all the processes and frees up memory, and the script will work after that.

Is driver.quit() sufficient to close all Chrome processes in headless or otherwise? Is adding something in my Python script like:

import os 
os.system("pkill chrome") 

Doing anything that driver.quit() is not already doing? I just want to minimize the chances of a crash by making sure Chrome is completely closed after Python script is run.

1

There are 1 best solutions below

3
On

driver.quit() quits (closes) only this specific driver object.
This will definitely not close any other running driver processes.
Also process involved by chromedriver appears as chromedriver or chromedriver (32 bit), not chrome. chrome process is your Chrome browser, not the Selenium webdriver.
To ensure closing the chromedriver you can use try-except-finally involving driver.quit() inside the finally block.
I'm not sure this is the optimal approach since AFAIK this may affect reporting mechanism.