C# WebSocketSharp.WebSocketException: Not a WebSocket handshake response

Question

I'd like to scrape real-time data from a website and i decided to use webSocket - sharp library. My problem is that with the same code i can parse the data from a specific website and i can't from another one.

The program throws this exception: WebSocket.connect:0|WebSocketSharp.WebSocketException: Not a WebSocket handshake response.

using (var wss = new WebSocket("wss://..."))
{
    wss.SslConfiguration.EnabledSslProtocols = System.Security.Authentication.SslProtocols.Tls12;
    wss.Origin = "https://www.blabla.com";
           
    wss.CustomHeaders = new Dictionary<string, string>
    {
        { "Accept-Encoding", "gzip, deflate, br" },
        { "Accept-Language", "el-GR,el;q=0.9,en;q=0.8" },
        { "Cache-Control", "no-cache" },
        { "Connection", "Upgrade" },
        { "Host", "blabla.com" },
        { "Origin", "https://www.bla.com" },
        { "Pragma", "no-cache" },
        //{ "Sec-WebSocket-Key", secWebSocketKey },
        //{ "Sec-WebSocket-Protocol", "zap-protocol-v1" },
        { "Sec-WebSocket-Extensions", "permessage-deflate; client_max_window_bits" },
        { "Sec-WebSocket-Version", "13" },
        { "Upgrade", "websocket" },
        { "User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36" }
     };

     //wss.OnOpen += Ws_OnOpen;
     wss.OnMessage += (sender, e) => Console.WriteLine($"Server: {e.Data}");
     wss.OnError += (sender, e) => Console.WriteLine($"Error: {e.Message}");

     wss.Connect();

     Console.ReadKey();
 }

I tried with or without custom headers.

What have i do to make a valid handshake?

(P.S: I can parse the data without custom headers from the first website)

UPDATE

In the URL there is a uid parameter wss://blabla.com/zap/?uid=5829062969032768

This uid changes in every refresh of webpage. I think it's necessary for the handshake. Is there any way to reproduce it?

Answer 1

This uid changes every time the page loads. I found that this site uses code obfuscation so it was too difficult for me to underastand the js code so i used selenium 4 devtools and finally scrape real-time data.

First have to initialize chrome devtools

public async static Task<DevToolsSession> InitializeChromeDevTools(IWebDriver driver)
{
    var devTools = driver as IDevTools;
    var output = devTools.CreateDevToolsSession();
    await output.Network.Enable(new OpenQA.Selenium.DevTools.Network.EnableCommandSettings());

    return output;
}

And then

var session = await ChromeDriverSettings.InitializeChromeDevTools(driver);
session.Network.WebSocketFrameReceived += Network_WebSocketFrameReceived; 

private static void Network_WebSocketFrameReceived(object sender, OpenQA.Selenium.DevTools.Network.WebSocketFrameReceivedEventArgs e)
{
    var message = e.Response.PayloadData;
}