I usually scrape web pages with rvest::read_html(); however, this time I want to read an html page using a residential proxy:
Proxy username: username
Proxy password: password
Proxy hostname: geo.iproyal.com (my residential proxy service)
Proxy port: port
Let's also assume that I'm trying to read the following web page: https://ipv4.icanhazip.com
The website for my residential proxy service provide the following curl request which works perfectly:
$ curl -v -x http://username:[email protected]:port -L https://ipv4.icanhazip.com
where $ refers to a bash shell.
My goal is to reproduce this curl request using {httr} or {httr2}. Here is my (failed) attempt:
req <- httr2::request(base_url = url) |>
httr2::req_proxy(
url = "geo.iproyal.com",
port = port,
username = username,
password = password,
auth = "basic"
)
This returns a "403 forbidden" error message.
Thanks in advance for your help.