Firefox could not load to a certain webpage with remote webdriver

Question

I am doing selenium test running on dockers whit multi browser. It did run successfully at the beginning with all three browsers. after one month, it just works fine with chrome and edge, but fails with firefox. I have check with the local firefox browser and webdriver, it still works. So now it just doesn't work when runs on dockers.

The process of the test is:

1. load to https://theCompany.com, then click on the "loging" button
2. webpage load to login.microsoftonline.com, and the user type in account and password, and then a confirm button.
3. webpage load to the https://theCompany.com again.

The problem is with the second step. Errors are showed different: sometimes it is said "we're having trouble finding that site. we cannot connect to the server at login.microsoftonline.com. if that address is correct, here are three other things you can try: try again later. check your network connection. if you are connected but behind a firewall, check that firefox has permission to access the web." And sometimes it just stayed on the first step (I saw that via VNC viewer),could not load to login.microsoftonline.com, but the error is: could not find the element By.id("logonIdentifier") which should be on the second step(login.microsoftonline.com).

The firefox remote webdriver i set up with a login test.

@BeforeTest
@Throws(Exception::class)
fun setup(browser: String) {
        val cap = DesiredCapabilities()
        val options = FirefoxOptions()
        val profile = FirefoxProfile()
        profile.setPreference("browser.download.dir", folder!!.absolutePath)
        profile.setPreference("browser.download.folderList", 2)
        profile.setPreference(
            "browser.helperApps.neverAsk.saveToDisk",
            "image/jpeg, application/pdf, application/octet-stream"  )
        profile.setPreference("browser.download.manager.showWhenStarting", false)
        profile.setPreference("pdfjs.disabled", true)

        options.setLogLevel(FirefoxDriverLogLevel.TRACE)
        options.profile = profile
        cap.setCapability(FirefoxOptions.FIREFOX_OPTIONS, options)
       
       //driver = FirefoxDriver(cap)
       driver = RemoteWebDriver(URL("http://localhost:4444/wd/hub"), cap);           
       driver.manage().deleteAllCookies()
       driver.manage().window().maximize()
 }
@Test
@Throws(WebDriverException::class)
fun login() {
    driver.get(baseurl)
    sleep(2000)
    val element = driver.findElement(By.xpath("//label[contains(text(),'Sign in)]"))
    val executor = driver as JavascriptExecutor
    executor.executeScript("arguments[0].click();", element)
    WebDriverWait(driver, 60).until(ExpectedConditions.elementToBeClickable(By.id("logonIdentifier")))
}

And the docker-compose file I used to set up docker containers:

version: "3"
services:
  chrome:
    image: selenium/node-chrome:4.0.0-20211013
    container_name: chrome
    shm_size: 2gb
    depends_on:
      - selenium-hub
    volumes:
      - ./target:/home/seluser/Downloads
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6900:5900"
  edge:
    image: selenium/node-edge:4.0.0-20211013
    container_name: edge
    shm_size: 2gb
    depends_on:
      - selenium-hub
    volumes:
      - ./target:/home/seluser/Downloads
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6901:5900"
  firefox:
    image: selenium/node-firefox:4.0.0-20211013
    container_name: firefox
    shm_size: 2gb
    depends_on:
      - selenium-hub
    volumes:
      - ./target:/home/seluser/Downloads
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
      - SE_NODE_GRID_URL=http://localhost:4444
    ports:
      - "6902:5900"
  selenium-hub:
    image: selenium/hub:4.0.0-20211013
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

And the log is:

1638531872036   Marionette  DEBUG   0 -> [0,8,"WebDriver:FindElement",{"using":"css selector","value":"#logonIdentifier"}]
2021-12-03T11:44:32.037730000Z 1638531872037    Marionette  TRACE   [17] MarionetteCommands actor created for window id 2147483651
2021-12-03T11:45:02.047816700Z 1638531902047    Marionette  DEBUG   0 <- [1,8,{"error":"no such element","message":"Unable to locate element: #logonIdentifier","stacktrace":"WebDriverError@chrome:// ... te/content/shared/webdriver/Errors.jsm:393:5\nelement.find/</<@chrome://remote/content/marionette/element.js:305:16\n"},null]
2021-12-03T11:45:02.048075900Z 1638531902047    webdriver::server   DEBUG   <- 404 Not Found {"value":{"error":"no such element","message":"Unable to locate element: #logonIdentifier","stacktrace":"WebDriverError@chrome://remote/content/shared/webdriver/Errors.jsm:181:5\nNoSuchElementError@chrome://remote/content/shared/webdriver/Errors.jsm:393:5\nelement.find/</<@chrome://remote/content/marionette/element.js:305:16\n"}}
2021-12-03T11:45:02.410007400Z 1638531902409    webdriver::server   DEBUG   -> DELETE /session/e954f16b-4d17-43e9-be81-65c4bd1cab72 
2021-12-03T11:45:02.411098100Z 1638531902410    Marionette  DEBUG   0 -> [0,9,"Marionette:Quit",{"flags":["eForceQuit"]}]
2021-12-03T11:45:02.414650600Z 1638531902414    Marionette  INFO    Stopped listening on port 45613
2021-12-03T11:45:02.575774300Z 1638531902575    Marionette  TRACE   Received observer notification quit-application
2021-12-03T11:45:02.575994900Z 1638531902575    Marionette  TRACE   Received observer notification quit-application
2021-12-03T11:45:02.576081200Z 1638531902575    Marionette  DEBUG   Marionette stopped listening
2021-12-03T11:45:02.585735700Z 1638531902585    RemoteAgent DEBUG   Resetting recommended pref apz.content_response_timeout
2021-12-03T11:45:02.586393900Z 1638531902585    RemoteAgent DEBUG   Resetting recommended pref browser.contentblocking.introCount
2021-12-03T11:45:02.586443200Z 1638531902585    RemoteAgent DEBUG   Resetting recommended pref browser.download.panel.shown
2021-12-03T11:45:02.586744200Z 1638531902586    RemoteAgent DEBUG   Resetting recommended pref browser.pagethumbnails.capturing_disabled
2021-12-03T11:45:02.587027400Z 1638531902586    RemoteAgent DEBUG   Resetting recommended pref browser.tabs.disableBackgroundZombification
2021-12-03T11:45:02.587565400Z 1638531902587    RemoteAgent DEBUG   Resetting recommended pref browser.tabs.unloadOnLowMemory
2021-12-03T11:45:02.587855700Z 1638531902587    RemoteAgent DEBUG   Resetting recommended pref browser.tabs.warnOnCloseOtherTabs
2021-12-03T11:45:02.587993800Z 1638531902587    RemoteAgent DEBUG   Resetting recommended pref browser.toolbars.bookmarks.visibility
2021-12-03T11:45:02.588222400Z 1638531902587    RemoteAgent DEBUG   Resetting recommended pref browser.urlbar.suggest.searches
2021-12-03T11:45:02.593716200Z 1638531902593    RemoteAgent DEBUG   Resetting recommended pref datareporting.policy.dataSubmissionPolicyAccepted
2021-12-03T11:45:02.593821000Z 1638531902593    RemoteAgent DEBUG   Resetting recommended pref dom.file.createInChild
2021-12-03T11:45:02.593969900Z 1638531902593    RemoteAgent DEBUG   Resetting recommended pref dom.push.connection.enabled
2021-12-03T11:45:02.594384800Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref extensions.getAddons.cache.enabled
2021-12-03T11:45:02.594597500Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref extensions.getAddons.discovery.api_url
2021-12-03T11:45:02.594740000Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref network.http.prompt-temp-redirect
2021-12-03T11:45:02.595122800Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref privacy.trackingprotection.enabled
2021-12-03T11:45:02.595170600Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref security.notification_enable_delay
2021-12-03T11:45:02.595184300Z 1638531902594    RemoteAgent DEBUG   Resetting recommended pref signon.autofillForms
2021-12-03T11:45:02.608444800Z 1638531902607    RemoteAgent DEBUG   Resetting recommended pref browser.topsites.contile.enabled
2021-12-03T11:45:02.608747300Z 1638531902608    RemoteAgent DEBUG   Resetting recommended pref browser.contentblocking.features.standard
2021-12-03T11:45:02.608850200Z 1638531902608    RemoteAgent DEBUG   Resetting recommended pref network.cookie.cookieBehavior
2021-12-03T11:45:02.609426200Z 1638531902609    RemoteAgent DEBUG   Resetting recommended pref dom.disable_beforeunload
2021-12-03T11:45:02.622667000Z 1638531902621    RemoteAgent TRACE   Received observer notification quit-application
2021-12-03T11:45:02.623740400Z 1638531902623    RemoteAgent DEBUG   Resetting recommended pref browser.contentblocking.features.standard
2021-12-03T11:45:02.623797600Z 1638531902623    RemoteAgent DEBUG   Resetting recommended pref network.cookie.cookieBehavior
2021-12-03T11:45:02.636600700Z 1638531902625    Marionette  DEBUG   0 <- [1,9,null,{"cause":"shutdown","forced":false}]
2021-12-03T11:45:02.701636900Z 1638531902701    webdriver::server   DEBUG   Teardown session
2021-12-03T11:45:02.701710900Z JavaScript error: resource:///modules/Interactions.jsm, line 201: NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIUserIdleService.removeIdleObserver]
2021-12-03T11:45:02.721095000Z 1638531902720    Marionette  DEBUG   Closed connection 0
2021-12-03T11:45:03.803159500Z 1638531903802    geckodriver::browser    DEBUG   Browser process stopped: exit status: 0
2021-12-03T11:45:03.820153700Z 1638531903819    webdriver::server   DEBUG   <- 200 OK {"value":null}

Could anyone has any suggestions? Thanks a lot.