How to scrape good quality images from a site? (Python, Selenium)

Question

I'm trying to use a web scrapper that gets images from a site.

However, my methods cause a severe drop in quality.

I'm trying to scrape the images from this site.

I have tried to download the image and use the image link I got from the href tag. Both of these attempts cause a considerable drop in image quality. I am now considering taking a screenshot and cropping the image, however, I feel like this would be a roundabout way to do it. Does anyone know of a method to get around the quality drop? or any useful libraries maybe?

(edit) The easiest way seems to be to download the images themselves using a combination of request and web frameworks; might pursue this but I'm going to give it a little more time to stew. Does anyone know of a way to use the images without outright downloading them?

Answer 1

Manually getting the data through requests and writing it to a file of the same extension should work.

import requests

url = "https://img.ltwebstatic.com/images3_pi/2023/04/22/1682132328cfa167169f129c340da4fc854d5587b4_thumbnail_600x.jpg"
img_data = requests.get(url).content
with open('image.jpg', 'wb') as f:
   f.write(img_data)

Answer 2

Here's how you can save/download all the available images of the given product.

import requests
from bs4 import BeautifulSoup
from PIL import Image

def save_image(img_url):
    filename = img_url.split('/')[-1]
    img = Image.open(requests.get(img_url, stream=True).raw)
    img.save(f"{filename}")
    print(f"saved image: {filename}\n")


url = "https://us.shein.com/Ditsy-Floral-Print-Knot-Cuff-Blouse-p-14155075-cat-1733.html?src_identifier=fc%3DWomen%60sc%3DCLOTHING%60tc%3D0%60oc%3D0%60ps%3Dtab01navbar05%60jc%3DitemPicking_001121425&src_module=topcat&src_tab_page_id=page_home1687813063116&mallCode=1"
data = requests.get(url).text
soup = BeautifulSoup(data, 'lxml')

images = soup.select_one('div.product-intro__main').select('img')
print(f"https:{images[0].get('src')}")
save_image(f"https:{images[0].get('src')}")

for image in images[1:]:
    print(f"https:{image.get('data-src')}")
    save_image(f"https:{image.get('data-src')}")

output:

https://img.ltwebstatic.com/images3_pi/2023/04/22/1682132328cfa167169f129c340da4fc854d5587b4_thumbnail_600x.jpg
saved image: 1682132328cfa167169f129c340da4fc854d5587b4_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/16821323962950631a3b7546fcb3f7beea67915bf4_thumbnail_600x.jpg
saved image: 16821323962950631a3b7546fcb3f7beea67915bf4_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/16821323996e9b06f5553e4a23876b767e12051b7a_thumbnail_600x.jpg
saved image: 16821323996e9b06f5553e4a23876b767e12051b7a_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/1682132401e3e59b7a6a6b5fdd1e6b2585313ba4b1_thumbnail_600x.jpg
saved image: 1682132401e3e59b7a6a6b5fdd1e6b2585313ba4b1_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/168213240373037078a11f0977ef94ae97e16ebfe6_thumbnail_600x.jpg
saved image: 168213240373037078a11f0977ef94ae97e16ebfe6_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/1682132408646e8ffe0b156bad84ef44d34c3d4f88_thumbnail_600x.jpg
saved image: 1682132408646e8ffe0b156bad84ef44d34c3d4f88_thumbnail_600x.jpg

https://img.ltwebstatic.com/images3_pi/2023/04/22/1682132410a5bb9b65bc0abe32ec5182c35fa2a640_thumbnail_600x.jpg
saved image: 1682132410a5bb9b65bc0abe32ec5182c35fa2a640_thumbnail_600x.jpg

Answer 3

Ended up following pineconee's advice, iterating through the images I wanted to get downloading them, applying them to where I wanted them to go and then deleting them using the OS module.

However there was an extra caveat to my problem which required me to step out of the Selenium module because the file selection was local and not through the browser, solved that by using the pyautogui. The time.sleeps() seem to be a must, if the automation goes too fast the file doesn't seem to catch and gets stuck on the file explorer.

def loadImages(driver, images):
try:
    for image in images:
        filename = image.split('/')[-1].split('.')[0] + ".jpg"
        imgData = requests.get(image).content

        with open(filename, "wb") as f:
            f.write(imgData)

        filepath = os.path.abspath(filename)

        boxx = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[@class='dn_ht']")))
        boxx.click()

        time.sleep(1)
        pyautogui.write(filepath)
        time.sleep(1)
        pyautogui.press('enter')
        time.sleep(5)
        os.remove(filename)

except Exception as e:
    print(e)