Windows Jenkins slaves are disconnecting and tunneling port stopped accepting requests

166 Views Asked by At

I’m using Jenkins with (Jenkins version is 2.375.2) (Java version is 55.0) (https.protocols TLSv1.2) The master node is deployed on an ubuntu server (4.15.0-142-generic #146~16.04.1-Ubuntu) And connected to agents through tunneling on port 50000, agents are windows and connected to the master using the jnlp command. Now, all of a sudden some slaves disconnected from the master and when trying to reconnect gives me this error.

May 14, 2023 12:33:47 PM hudson.remoting.jnlp.Main$CuiListener status                                                   INFO: Connecting to <jenkins>:50000 (retrying:7)
java.io.IOException: Failed to connect to <jenkins>:50000 
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:248)                                     at hudson.remoting.Engine.connectTcp(Engine.java:900) 
at hudson.remoting.Engine.innerRun(Engine.java:784) 
at hudson.remoting.Engine.run(Engine.java:543)
Caused by: java.net.ConnectException: Connection refused: connect
at java.base/sun.nio.ch.Net.connect0(Native Method) 
at java.base/sun.nio.ch.Net.connect(Net.java:483) 
at java.base/sun.nio.ch.Net.connect(Net.java:472)
at java.base/sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:692) 
at java.base/java.nio.channels.SocketChannel.open(SocketChannel.java:194)
at org.jenkinsci.remoting.engine.JnlpAgentEndpoint.open(JnlpAgentEndpoint.java:206) 
... 3 more     

The master logs are

May 12, 2023 11:34:33 PM INFO jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/<ip>:50000 remote=<served>]] / Computer.threadPoolForRemoting [#903] for vsb-5 terminated: java.nio.channels.ClosedChannelException
May 12, 2023 11:41:15 PM INFO jenkins.slaves.DefaultJnlpSlaveReceiver channelClosed
IOHub#1: Worker[channel:java.nio.channels.SocketChannel[connected local=/<ip>:50000 remote=<server>]] / Computer.threadPoolForRemoting [#908] for vsb-4 terminated: java.nio.channels.ClosedChannelException
May 12, 2023 11:43:11 PM INFO hudson.slaves.ChannelPinger$1 onDead
Ping failed. Terminating the channel vsb-3.
java.util.concurrent.TimeoutException: Ping started at 1683927551978 hasn't completed by 1683927791979
    at hudson.remoting.PingThread.ping(PingThread.java:132)
    at hudson.remoting.PingThread.run(PingThread.java:88)

The UI node logs after disconnecting are

Connection was broken

java.nio.channels.ClosedChannelException
    at org.jenkinsci.remoting.protocol.NetworkLayer.onRecvClosed(NetworkLayer.java:155)
    at org.jenkinsci.remoting.protocol.impl.NIONetworkLayer.ready(NIONetworkLayer.java:180)
    at org.jenkinsci.remoting.protocol.IOHub$OnReady.run(IOHub.java:789)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:30)
    at jenkins.security.ImpersonatingExecutorService$1.run(ImpersonatingExecutorService.java:70)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)

I checked everything related to the network, and I found that the port is listening on the host but is not listening inside the container. The server metrics (CPU, RAM) are also good.

0

There are 0 best solutions below