Access csv data from internet with c# and System.Net.Http

59 Views Asked by At

I am trying to download historical time series from Yahoo! Finance (I have the same issue with other sites). I have embedded the url address in a function to get a csv file. I am using System.Net and I am really not familiar with the protocols.

The address I am seeking is:

https://query1.finance.yahoo.com/v7/finance/download/AAPL?&period1=1662422400&period2=1693958400&interval=1d&ignore=.csv&events=history

It works well when I directly use it in my internet navigator: the csv file is correctly downloaded.

Now, I am trying to call it through the following routine:

using System;
using System.IO;
using System.Net;

namespace Space
{
    public class HistData
    {
        private WebClient connector;

        public string GetData(string address)
        {
            string csvData = null;
            using (connector = new WebClient())
            {
                using (Stream strm = connector.OpenRead(address))
                {
                    using (StreamReader strd = new StreamReader(strm))
                    {
                        csvData = strd.ReadToEnd();
                    }
                }
            }
            return csvData;
        }
    }
}

I have an exception on the statement

connector.OpenRead(address)

that says 'connector.OpenRead(address)' threw an exception of type 'System.Net.WebException': "The underlying connection was closed: An unexpected error occurred on a send".

The inner exception gives me the message: "Authentication failed because the remote party has closed the transport stream".

Is there a way to bypass or sort out this problem?


Addendum


Following the comments, I moved to Windows 10 and VS 2015.

I changed the code to this.

using System;
using System.Collections.Generic;
using System.IO;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

namespace Space
{
    public class HistData
    {
        private static string address1 ="https://query1.finance.yahoo.com/v7/finance/download/AAPL?&period1=1694563200&period2=1694736000&interval=1d&ignore=.csv&events=history";
        private static string address2 ="https://www.marketwatch.com/investing/stock/AAPL/downloaddatapartial?&startdate=8/15/2023&enddate=9/15/2023&frequency=p1d&csvdownload=true";
        
        private static HttpClient client = new HttpClient();
        private static string csvData;

        public string GetCsvData(string address)
        {
            Task<bool> awaiter = GetCsvData(address);
            if (awaiter.Result)
            {
                return csvData;
            }
            return null;
        }

        public async Task<bool> GetCsvData(string address)
        {
            if (address != null && address != "")
            {
                using (client = new HttpClient())
                {
                    using (Stream strm = await client.GetStreamAsync(address))
                    {
                        using (StreamReader strd = new StreamReader(strm))
                        {
                            csvData = strd.ReadToEnd();
                        }
                    }
                }
                return true;
            }
            return false;
        }
    }
}

Whenever I request the data from the internet browser for both Yahoo! Finance and Market Watch addresses, I get the expected csv file.

If I run the request for Yahoo! Finance under the routine, it still works. However, if I run the request for Market Watch under the routine, it fails. I have the following exceptions.

"System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.\r\n
   at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)\r\n
   at System.Net.PooledStream.EndWrite(IAsyncResult asyncResult)\r\n
   at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)\r\n
   --- End of inner exception stack trace ---\r\n
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)\r\n
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)\r\n
   --- End of inner exception stack trace ---\r\n
   --- End of inner exception stack trace ---\r\n
---> (Inner Exception #0) System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Authentication failed because the remote party has closed the transport stream.\r\n
   at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)\r\n
   at System.Net.PooledStream.EndWrite(IAsyncResult asyncResult)\r\n
   at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)\r\n
   --- End of inner exception stack trace ---\r\n
   at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)\r\n
   at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)\r\n
   --- End of inner exception stack trace ---<---\r\n"

I have the same question as before: is there a way to prevent or circumvent this sort of problem?

0

There are 0 best solutions below