How do I set up confluent docker to be accessed externally, based on confluent docker quickstart

Question

the following is based on this quickstart guide here: http://docs.confluent.io/current/cp-docker-images/docs/quickstart.html

In there they demonstrate various kafka/confluent components in their own docker containers, each started with the

--net=host

flag and accessed via

localhost:port

No matter what I do, I am unable to access this from outside the ubuntu server itself, neither via ip nor domain, which they state should work e.g. for the control center. But on the same host, my ubuntu box, everything works fine.

Any idea what the issue could be here? Stuck on this for hours already

Answer 1

Is the problem that you can't access the port (ie telnet not possible) or that you can't make the server works (ie no request are answered by the server)?

There is the -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:29092 \, this is the address kafka broker communicates to client for communication (and except on localhost, this obviously won't work...) after initial connexion. You can replace localhost with the ip of your server, it should work properly.

To be more specific, there is listeners config (default 0.0.0.0:9092) and advertised.listeners (default PLAINTEXT://locahost:9092) The client initialize connexion on (bootstrap-server or broker-list) via the listener binding. Once this initial connexion is done, the broker will return all advertised.listeners from all brokers in cluster, and this is what is used for later exchanges)

From github repo, confluent assign to listener the value of advertised listener, changing the host to 0.0.0.0 (so it will be accessible from remote sure) You can't set 0.0.0.0 to advertised.listeners though, it must be a unique reachable interface

Answer 2

Answering myself now, a stupidly simple firewall / docker thing:

careful when using UFW (firewall) and docker

see http://blog.viktorpetersson.com/post/101707677489/the-dangers-of-ufw-docker

I used a simple nodeJS hello world example on port 3000 and was able to connect to it from the outside as long as I used default/bridge networking and published the port (-p 3000:3000).

Using the host network I was not able to connect at all.

In both cases the firewall (UFW on ubuntu) did not explicitly allow port 3000 so Docker must be doing some hidden magic here of rewriting iptables without UFW noticing to allow bridged and published ports to go through.

=> solved by explicitly opening kafka ports in UFW