.Net WebClient DownloadString Not Working With Mixed Content

Question

I am working on scraper in which i need to scrape data from one site. I have tired very simple webclient downloadString to get the data, which seems to be working with other sites, but having issue with the one which is i have trying in the following code. Any help will be appreciated.Thanks in Advance. Following code in VB.Net,but i can happy to have a working solution in c# as well.

Private Function GetHtml() As String
    Dim mData As String = ""
    Try
        'ServicePointManager.ServerCertificateValidationCallback = New Security.RemoteCertificateValidationCallback(AddressOf ValidateServerCertificate)
        'ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12

        With mWC
            mData = .DownloadString("https://www.adorama.com/brands")
        End With
    Catch ex As Exception
        Debug.Print(ex.Message)

        'With CertificateValidationCallback
        'The remote server returned an error: (403) Forbidden.

        'Without CertificateValidationCallback
        'The request was aborted: Could not create SSL/TLS secure channel.
    End Try

    Return mData
End Function

Private Shared Function ValidateServerCertificate(ByVal sender As Object, ByVal certificate As X509Certificate, ByVal chain As X509Chain, ByVal sslPolicyErrors As Net.Security.SslPolicyErrors) As Boolean
    If sslPolicyErrors = Net.Security.SslPolicyErrors.None Then
        Return True
    End If

    Return True
End Function

Answer 1

You have to add User-Agent for this website before you use .DownloadString() method.

mWC.Headers.Add("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:55.0) Gecko/20100101 Firefox/55.0")