Mechanize instance can't .open a specific website

101 Views Asked by At

I'm trying to use the mechanize .open method on a URL so I can eventually login and scrape this website. I've tried using this code on other websites and they work, but for some reason I'm getting the error only for this site: "mechanize._response.httperror_seek_wrapper: HTTP Error 404: Not Found". I'm new to Python but here's my code:

import mechanize
from bs4 import BeautifulSoup
import html2text
import http.cookiejar

# Browser
br = mechanize.Browser()

# Cookie Jar
cj = http.cookiejar.CookieJar()
br.set_cookiejar(cj)

br.set_handle_equiv(True)
br.set_handle_gzip(True)
br.set_handle_redirect(True)
br.set_handle_referer(True)
br.set_handle_robots(False)
br.set_handle_refresh(mechanize._http.HTTPRefreshProcessor(), max_time=1)
br.addheaders = [('User-agent', 'Chrome')]

br.open('https://onlinebusiness.icbc.com/webdeas-ui/login;type=driver')

Thanks a lot in advance.

0

There are 0 best solutions below