How to get the data from the elements tab in selenium?

Question

I'm scrapping a page in selenium (using the edge driver) and I've noticed discrepancies between the information under the elements tab in the dev window found here:

and the data returned from browser.page_source(), is there a way to get the data from the elements tab?

Answer 1

I think some websites block or restrict the selenium user agent.

Change user agent:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()

options.add_argument("user-agent=Something")

driver = webdriver ...

You can get random user-agent with fake-useragent module.

from fake_useragent import UserAgent

user_agent = UserAgent()
print(user_agent.random)

install:

pip install fake-useragent