Adapting Craigslist Scraper Python

Question

I am trying to adapt a python 2.7 craigslist scraper i found online to work with python 3.6.

But every time I run the python script it doesn't return anything. Is it because i'm not targeting the correct html tags? and if so how would i target the correct html tags?

I'm assuming it's this portion of the code here:

    for listing in soup.find_all('p',{'class':'result-row'}):
    if listing.find('span',{'class':'result-price'}) != None:

Full script is below.

Thank you in Advance.

import requests
from bs4 import BeautifulSoup
from urllib.parse import urljoin

URL = 'https://vancouver.craigslist.ca/search/sss?query=Vespa'
BASE = 'https://vancouver.craigslist.ca/'

response = requests.get(URL)

soup = BeautifulSoup(response.content,"html.parser")
for listing in soup.find_all('p',{'class':'result-row'}):
    if listing.find('span',{'class':'result-price'}) != None:
        price = listing.text[2:6]
        price = int(price)
        if price <=3600 and price > 1000:
            print (listing.text)
            link_end = listing.a['href']
            url = urljoin(BASE, link_end)
            print (url)
            print ("\n")
print('test')

Answer 1

You're right about this being the likely issue:

 for listing in soup.find_all('p',{'class':'result-row'}):
    if listing.find('span',{'class':'result-price'}) != None:

This piece has to be edited for the particular web page you're scraping. Have you looked at the page's HTML and verified these two rows? If not, right-click on the page and select "view page source". Then you have to find the particular data you want to scrape.

If I want to grab something from a webpage that looks like this in the html:

<div class='what'>hello</div>

I'd change that code above to this:

for listing in soup.find_all('div',{'class':'what'}):
     # do something