I'm a total beginner to webscraping using Selenium. I am trying to open a specific google profile (since all the sites would be already signed in). I'm happy that the code is able to open the specific profile window (profile being default). However, it fails to open the site in the window and start scraping. Here's the code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
# Installed chromedriver.exe
# Changing some arguments to set Chrome profile
options = webdriver.ChromeOptions()
# options.add_argument("--headless")
options.add_argument("--no-sandbox")
options.add_argument("--disable-gpu")
# Location where Chrome stores profiles
# options.add_arguments = {"user-data-dir": r"C:\Users\Kavipriyan\AppData\Local\Google\Chrome\User Data\Default"}
options.add_argument(r"--user-data-dir=C:\Users\Kavipriyan\AppData\Local\Google\Chrome\User Data")
# Profile name
options.add_argument(r"--profile-directory=Default")
service = Service(executable_path="chromedriver.exe")
driver = webdriver.Chrome(options=options, service=service)
driver.get("https://twitter.com/home")
time.sleep(3)
driver.close()
This code on the side gave this huge error in the console too (I'm not sure which parts are the exactly required ones so I pasted whatever it printed out):
Opening in existing browser session.
Traceback (most recent call last):
File "d:\COMPUTER FILES\My Stuff\Code Programs - Python\Twitterscraping 2.0\Twitterscraper 3.2.py", line 23, in <module>
driver = webdriver.Chrome(options=options, service=service)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\chrome\webdriver.py", line 45, in __init__
super().__init__(
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\chromium\webdriver.py", line 61, in __init__
super().__init__(command_executor=executor, options=options)
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 208, in __init__
self.start_session(capabilities)
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 292, in start_session
response = self.execute(Command.NEW_SESSION, caps)["value"]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\webdriver.py", line 347, in execute
self.error_handler.check_response(response)
File "C:\Users\Kavipriyan\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\selenium\webdriver\remote\errorhandler.py", line 229, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: Chrome failed to start: exited normally.
(session not created: DevToolsActivePort file doesn't exist)
(The process started from chrome location C:\Program Files\Google\Chrome\Application\chrome.exe is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Stacktrace:
GetHandleVerifier [0x00007FF667BC7062+63090]
(No symbol) [0x00007FF667B32CB2]
(No symbol) [0x00007FF6679CEC65]
(No symbol) [0x00007FF667A00777]
(No symbol) [0x00007FF6679FB2F4]
(No symbol) [0x00007FF667A40BFB]
(No symbol) [0x00007FF667A40830]
(No symbol) [0x00007FF667A36D83]
(No symbol) [0x00007FF667A083A8]
(No symbol) [0x00007FF667A09441]
GetHandleVerifier [0x00007FF667FC25CD+4238301]
GetHandleVerifier [0x00007FF667FFF72D+4488509]
GetHandleVerifier [0x00007FF667FF7A0F+4456479]
GetHandleVerifier [0x00007FF667CA05A6+953270]
(No symbol) [0x00007FF667B3E57F]
(No symbol) [0x00007FF667B39254]
(No symbol) [0x00007FF667B3938B]
(No symbol) [0x00007FF667B29BC4]
BaseThreadInitThunk [0x00007FFDF5047344+20]
RtlUserThreadStart [0x00007FFDF56426B1+33]
Thanks in advance!