Webscraping issue with cloudscraper and proxies

1.7k Views Asked by At

I have problem with usage proxies servers with cloudscraper.

Simple program:

proxy_list = {"http": "http://username:[email protected]:port"}
scraper = cloudscraper.create_scraper()
print(scraper.get("https://api.ipify.org/", proxies=proxy_list))
exit()

This simple program always showing my IP address not used proxy server. Why? I try multiple (working) proxy servers, no errors when running, and always not working. Any suggestions?

2

There are 2 best solutions below

1
juhacz On

I change:

proxy_list = {"http": "http://username:[email protected]:port"}
to
proxy_list = {"https": "http://username:[email protected]:port"}

and all now working fine, but I don know why.

0
M. Serseg On

You have to set proxy for both http and https

proxy_list = {"http": "http://username:[email protected]:port", "https": "http://username:[email protected]:port"}