Cannot use existing network in Compose: ERROR: Network declared as external, but could not be found

Question

I'm trying to create a service that must join the existing stack, so I force the compose to use the same network.

Surely, my network is persists

 docker network ls
   NETWORK ID          NAME                             DRIVER              SCOPE
   oiaxfyeil72z        ELK_default                      overlay             swarm
   okhs1e1wu73y        ELK_elk                          overlay             swarm

My docker-compose.yml

version: '3.3'
services: 
  logstash:
    image: docker.elastic.co/logstash/logstash-oss:7.5.1
    ports:
      - "5000:5000"
      - "9600:9600"
    volumes:
      - '/share/elk/logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro'
      - '/share/elk/logstash/pipeline/:/usr/share/logstash/pipeline/:ro'
    environment:
      LS_JAVA_OPTS: "-Xmx512m -Xms256m"
    networks:
      - elk
    deploy:
      mode: replicated
      replicas: 1
networks:
  elk:
    external: 
       name: ELK_elk

the other services was created with

version: '3.3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.5.1
    ports:
      - "9200:9200"
      - "9300:9300"
    volumes:
      - '/share/elk/elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml:ro'
    environment:
      ES_JAVA_OPTS: "-Xmx512m -Xms256m"
      ELASTIC_PASSWORD: changeme
      discovery.type: single-node
    networks:
      - elk
    deploy:
      mode: replicated
      replicas: 1
      
  kibana:
    image: docker.elastic.co/kibana/kibana:7.5.1
    ports:
      - "5601:5601"
    volumes:
      - '/share/elk/kibana/config/kibana.yml:/usr/share/kibana/config/kibana.yml:ro'
    networks:
      - elk
    deploy:
      mode: replicated
      replicas: 1

networks:
  elk:
    driver: overlay

check with docker stack services

docker stack services ELK
ID                  NAME                MODE                REPLICAS            IMAGE                                                 PORTS
c0rux6mdvzq3        ELK_kibana          replicated          1/1                 docker.elastic.co/kibana/kibana:7.5.1                 *:5601->5601/tcp
j824fd0blxdp        ELK_elasticsearch   replicated          1/1                 docker.elastic.co/elasticsearch/elasticsearch:7.5.1   *:9200->9200/tcp, *:9300->9300/tcp

Then trying to bring the service up with docker-compose up -d. The service doesn't create but produce the error

docker-compose up -d
WARNING: Some services (logstash) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
WARNING: The Docker Engine you're using is running in swarm mode.

Compose does not use swarm mode to deploy services to multiple nodes in a swarm. All containers will be scheduled on the current node.

To deploy your application across the swarm, use `docker stack deploy`.

Removing tmp_logstash_1
Recreating bbf503fc3eaa_tmp_logstash_1 ... error

ERROR: for bbf503fc3eaa_tmp_logstash_1  Cannot start service logstash: Could not attach to network ELK_elk: rpc error: code = PermissionDenied desc = network ELK_elk not manually attachable

ERROR: for logstash  Cannot start service logstash: Could not attach to network ELK_elk: rpc error: code = PermissionDenied desc = network ELK_elk not manually attachable
ERROR: Encountered errors while bringing up the project.

Answer 1

The issue is due to the fact that the elk network is defined as an "overlay" network. It's a docker swarm feature so docker-compose does not know how to deal with it.

Instead of using docker-compose up you need to deploy a docker swarm stack:

docker stack deploy -c docker-compose.yml <service_name>

You can refer to the Docker documentation for more info: https://docs.docker.com/network/

Answer 2

For some reason non manager nodes only see networks with active containers using it (Run on non manager node):

docker run --rm -d --name dummy busybox # Run a dummy container

docker network connect [OVERLAY_NETWORK] dummy # Connect to overlay network

now network is available on non manager node and you can run:

docker compose -f compose.yaml -p project up -d

docker stop dummy # Remove dummy container

Compose file:

networks:
  db:
    external: true
    driver: overlay