How to scrape a price from this particular webpage

Question

when I go to this particular webpage you will see that there are a number of items that have particular price in Ethereum (ETH) attached to it: https://opensea.io/collection/beanzofficial?search[paymentAssets][0]=ETH&search[stringTraits][0][name]=Background&search[stringTraits][0][values][0]=Off%20White%20C&search[toggles][0]=BUY_NOW

For example, the first item in the picture below is "Bean #7474" and has a price of "1.37 ETH" How can I get that price of 1.37 ETH by python? I tried to look at the "view-source:" of the page in Chrome, but the string 1.37 ETH does not show up. Also, I have used python to do:

req = Request(pageurl, headers={'User-Agent': 'User-Agent'})

html = urlopen(req).read()

but the html text of the page does not contain the '1.37 ETH' string. Is there any way I can get '1.37 ETH' using python by getting the html text of the page (ideally) or otherwise. Thank you

Answer 1

Check the below Python code: Updated code based on feedback received in the comments

driver = webdriver.Chrome()
driver.maximize_window()
driver.get("https://opensea.io/collection/beanzofficial?search[paymentAssets][0]=ETH&search[stringTraits][0][name]=Background&search[stringTraits][0][values][0]=Off%20White%20C&search[toggles][0]=BUY_NOW")
wait = WebDriverWait(driver, 10)
product = wait.until(EC.visibility_of_element_located((By.XPATH, "(//span[@data-testid='ItemCardFooter-name'])[1]"))).text
amount = wait.until(EC.visibility_of_element_located((By.XPATH, "(//div[@data-testid='ItemCardPrice'])[1]//span//span[1]"))).text
print("Product name:" + product + "  " + "Amount:" + amount)

Imports:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Console output:

Product name:Bean #7474  Amount:1.37