Python3 Connection reset by peer

Question

urllib.error.URLError urlopen error [Errno 54] Connection reset by peer

I got this error by when try to fetch notino.com . I guess the guy used some clever way to prevent the screen scraper . I tried to add header and cookie but this doesn't work

from urllib.request import urlopen
url = "https://www.notino.com"
html = urlopen(url)

Answer 1

An auto-bot detection mechanism is most likely dropping your connection. You should provide a User-Agent header to fake a browser visit - worked for me:

>>> import requests
>>> response = requests.get(url, headers={'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.95 Safari/537.36'})
>>> response.status_code
200

Using requests module in this example.