netTimeout when navigating more than once with RSelenium

Question

I'm getting this netTimeout error when I try to navigate more than once with RSelenium. The first time I use navigate after loading the browser, it always works. The second time I use navigate, I always get the timeout error.

The example below is using the docker image: selenium-standalone-firefox.

# Start selenium docker container
system('docker restart selenium-standalone-firefox') 

remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost", port = 4446L, browserName = "firefox")
remDr$open()

# navigate to first url - always works 
url <- "https://investors.alvotech.com/news-events/news-releases?page=1"
remDr$navigate(url)

# navigate to second url - always fails

url <- "https://investors.alvotech.com/news-events/news-releases?page=2"
remDr$navigate(url)

Selenium message:Reached error page: about:neterror?e=netTimeout&u=https%3A//investors.alvotech.com/news-events/news-releases%3Fpage%3D6&c=UTF-8&d=The%20server%20at%20investors.alvotech.com%20is%20taking%20too%20long%20to%20respond.
Build info: version: '4.8.3', revision: 'b19b418e60'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.10-76060610-generic', java.version: '11.0.18'
Driver info: driver.version: unknown

Error:   Summary: UnknownError
     Detail: An unknown server-side error occurred while processing the command.
     class: org.openqa.selenium.WebDriverException
     Further Details: run errorDetails method

# Restart browser now it works again:
remDr <- RSelenium::remoteDriver(remoteServerAddr = "localhost", port = 4446L, browserName = "firefox")
remDr$open()
remDr$navigate(url)

Below, I tried increasing timeouts but I get the same error:

# try increase timeouts - same error
remDr$setTimeout(type = "page load", milliseconds = 50000)
remDr$setTimeout(type = "script", milliseconds = 50000)
remDr$setTimeout(type = "implicit", milliseconds = 50000)
remDr$navigate(url)
Selenium message:Reached error page: about:neterror?e=netTimeout&u=https%3A//investors.alvotech.com/news-events/news-releases%3Fpage%3D6&c=UTF-8&d=The%20server%20at%20investors.alvotech.com%20is%20taking%20too%20long%20to%20respond.
Build info: version: '4.8.3', revision: 'b19b418e60'
System info: os.name: 'Linux', os.arch: 'amd64', os.version: '6.6.10-76060610-generic', java.version: '11.0.18'
Driver info: driver.version: unknown

Error:   Summary: UnknownError
     Detail: An unknown server-side error occurred while processing the command.
     class: org.openqa.selenium.WebDriverException
     Further Details: run errorDetails method

Answer 1

I was able to fix this by deleting cookies between page loads:

url <- "https://investors.alvotech.com/news-events/news-releases?page=1"
remDr$navigate(url)

remDr$deleteAllCookies()

url <- "https://investors.alvotech.com/news-events/news-releases?page=2"
remDr$navigate(url)

I have no idea why I need to delete cookies. I don't have to do this if I'm navigating to google searches, but if I am doing pagination on many different websites I had to do it.

If someone can come up with a more complete answer, please let me know. I'm really curious why I had to delete cookies.