Python bs4 BeautifulSoup: findall gives empty bracket

Question

when i run this code it gives me an empty bracket. Im new to web scraping so i dont know what im doing wrong.

import requests

from bs4 import BeautifulSoup

url = 'https://www.amazon.com/s/ref=nb_sb_noss_1?url=search-alias%3Daps&field-keywords=laptop'

r = requests.get(url)

soup = BeautifulSoup(r.text, 'html.parser')

container = soup.findAll('li', {'class': 's-result-item celwidget '})
#btw the space is also there in the html code

print(container)

results:

[]

What i tried is to grab the html code from the site, and to soup trough the li tags where all the information is stored so I can print out all the information in a for loop.

Also if someone wants to explain how to use BeautifulSoup we can always talk. Thank you guys.

Answer 1

So a working code that grabs product and price would could look something like this.

import requests
from bs4 import BeautifulSoup

url = 'https://www.amazon.com/s/ref=nb_sb_noss_1?url=search-alias%3Daps&field-keywords=laptop'
r = requests.get(url, headers={'User-Agent': 'Mozilla Firefox'})
soup = BeautifulSoup(r.text, 'html.parser')
container = soup.findAll('li', {'class': 's-result-item celwidget '})
for cont in container:
    h2 = cont.h2.text.strip()

    # Amazon lists prices in two ways. If one fails, use the other
    try:
        currency = cont.find('sup', {'class': 'sx-price-currency'}).text.strip()
        price = currency + cont.find('span', {'class': 'sx-price-whole'}).text.strip()
    except:
        price = cont.find('span', {'class': 'a-size-base a-color-base'})
    print('Product: {}, Price: {}'.format(h2, price))

Let me know if that helps you further...