Why can't my Docker container ping my Jenkins sidecar container by hostname instead of IP?

Question

I'm attempting to use the paradigm of Jenkins sidecar containers so that I can have one container running a postgres database while another container makes connections to it: https://www.jenkins.io/doc/book/pipeline/docker/#running-sidecar-containers

The problem I'm having is that my second container can't reach the postgres sidecar container by hostname. They can only communicate via IP address which is much more inconvenient.

I've reduced my job down to a simple scripted pipeline example which just uses two alpine linux containers and having container2 try to ping container1.

node {
    // create my-network if it does not already exist
    sh 'docker network inspect my-network >/dev/null 2>&1 || docker network create --driver bridge my-network'

    docker.image('alpine:latest').withRun('--name container1 --network my-network', 'tail -f /dev/null') { c ->
        sh 'docker ps --filter network=my-network' // shows that container1 running is on my-network

        // create container2 and ping container1 from inside
        docker.image('alpine:latest').inside('--name container2 --user root:root --network my-network') {

            sh """
              timeout 10s sh -c "until ping container1; do sleep 1 ; done"
            """
        }
    }
}

Here are the Jenkins logs of running this pipeline:

Started by user jewbix.cube
Replayed #7
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins-ABC in /opt/jenkins-work/workspace/ABC/test-docker-comms
[Pipeline] {
[Pipeline] sh
+ docker network inspect my-network
[Pipeline] isUnix
[Pipeline] sh
+ docker run -d --name container1 --network my-network alpine:latest tail -f /dev/null
time="2023-01-17T14:02:58-08:00" level=error msg="failed to move the rootless netns slirp4netns process to the systemd user.slice: exec: \"dbus-launch\": executable file not found in $PATH"
[Pipeline] sh
+ docker ps --filter network=my-network
CONTAINER ID  IMAGE                            COMMAND            CREATED                 STATUS                     PORTS       NAMES
ded0a24218e3  docker.io/library/alpine:latest  tail -f /dev/null  Less than a second ago  Up Less than a second ago              container1
[Pipeline] isUnix
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker inspect -f . alpine:latest
.
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] withDockerContainer
Jenkins-ABC does not seem to be running inside a container
$ docker run -t -d -u 1001:1002 --name container2 --user root:root --network my-network -w /opt/jenkins-work/workspace/ABC/test-docker-comms -v /opt/jenkins-work/workspace/ABC/test-docker-comms:/opt/jenkins-work/workspace/ABC/test-docker-comms:rw,z -v /opt/jenkins-work/workspace/ABC/test-docker-comms@tmp:/opt/jenkins-work/workspace/ABC/test-docker-comms@tmp:rw,z -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** -e ******** alpine:latest cat
$ docker top 0493f85dd30172a147cfefc423f2f3555e4100d403ceb6a8266602d895760830 -eo pid,comm
[Pipeline] {
[Pipeline] sh
+ timeout 5s sh -c 'until ping container1; do sleep 1 ; done'
ping: bad address 'container1'
ping: bad address 'container1'
ping: bad address 'container1'
ping: bad address 'container1'
ping: bad address 'container1'
Terminated
[Pipeline] }
$ docker stop --time=1 0493f85dd30172a147cfefc423f2f3555e4100d403ceb6a8266602d895760830
$ docker rm -f 0493f85dd30172a147cfefc423f2f3555e4100d403ceb6a8266602d895760830
[Pipeline] // withDockerContainer
[Pipeline] withEnv
[Pipeline] {
[Pipeline] sh
+ docker stop ded0a24218e33e8db18dc5d73308a40541a61f2c9847f24784a8bff69fe879d0
time="2023-01-17T14:03:16-08:00" level=warning msg="StopSignal SIGTERM failed to stop container container1 in 10 seconds, resorting to SIGKILL"
ded0a24218e33e8db18dc5d73308a40541a61f2c9847f24784a8bff69fe879d0
+ docker rm -f ded0a24218e33e8db18dc5d73308a40541a61f2c9847f24784a8bff69fe879d0
ded0a24218e33e8db18dc5d73308a40541a61f2c9847f24784a8bff69fe879d0
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 143
Finished: FAILURE

If I instead ping by IP address, which I obtain using the following code in the withRun block:

ip = sh(returnStdout: true, script: "docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' ${c.id}").trim()

Then the result is the following:

+ timeout 5s sh -c 'until ping 10.89.0.2; do sleep 1 ; done'
PING 10.89.0.2 (10.89.0.2): 56 data bytes
64 bytes from 10.89.0.2: seq=0 ttl=64 time=0.090 ms
64 bytes from 10.89.0.2: seq=1 ttl=64 time=0.072 ms
64 bytes from 10.89.0.2: seq=2 ttl=64 time=0.063 ms
64 bytes from 10.89.0.2: seq=3 ttl=64 time=0.062 ms
64 bytes from 10.89.0.2: seq=4 ttl=64 time=0.088 ms
Terminated
64 bytes from 10.89.0.2: seq=5 ttl=64 time=0.066 ms

One of the main reasons why this is made more difficult is that the documentation for Jenkins sidecar containers makes use of the deprecated --link flag which has since been removed from Docker completely, it seems. However as I understand, the alternative of creating a user-defined network and connecting both containers to it should be enough to allow pinging by hostname.

You may have also noticed that container2 is running with the --user root:root flag. If I don't set this flag, then any sh commands will just hang and never run. So I figured out that running as root inside the container is the only way I can actually run any sh scripts. Just pointing that out in case it's contributing to the issue.