Use python SSL to download google.com.au page

74 Views Asked by At

The following code is capable of retrieving the source code for the main page of a secure website, such as www.stgeorge.com.au

import socket, ssl
context = ssl.create_default_context()
context.verify_mode = ssl.CERT_OPTIONAL
context.check_hostname = True
context.load_default_certs()
host = 'www.stgeorge.com.au'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ssl_sock = context.wrap_socket(s, server_hostname=host) 
ssl_sock.connect((host, 443)) 
ssl_sock.send('GET https://' + host + ' HTTP/1.0\n\n')

while True:
    data = ssl_sock.recv(1024)
    if not data:
        break
    print data
ssl_sock.close()

However, if I replace www.stgeorge.com.au with www.google.com.au, pyton reports error "SSL: CERTIFICATE_VERIFY_FAILED"

What do I need to do to get the source code from google.com.au?

Regards, Nick

1

There are 1 best solutions below

1
On

After using the requests library, as suggested by skyline75489, the code looks like this:

import requests
r = requests.get('https://www.google.com.au')
f = open('test.txt', 'w')
f.write((r.text).encode('utf8'))
f.close()
print((r.text).encode('utf8'))

Thanks for help