Circus/ZeroMQ "socket in use" error

Question

I'm running a Flask app and internally using a library written in Node.js, which I access through ZeroRPC (the actual node process is managed by Circus). This works fine on its own; I can unit test with no issues. But when starting the Flask app as a listening process, and calling into a REST api which calls this libary, the program throws an exception when trying to start the process. The code to start the service is as follows:

from circus.watcher import Watcher
from circus.arbiter import ThreadedArbiter
from circus.util import (DEFAULT_ENDPOINT_DEALER, DEFAULT_ENDPOINT_SUB,
     DEFAULT_ENDPOINT_MULTICAST)
class Node(object):
    {... omitted code that initializes self._arbiter and self._client ...}
    def start(self):
        if self._arbiter and self._client:
            return
        port = 'ipc:///tmp/inlinejs_%s' % os.getpid()
        args = 'lib/server.js --port %s' % port
        watcher = Watcher('node', '/usr/local/bin/node', args,
            working_dir=INLINEJS_DIR)
        self._arbiter = ThreadedArbiter([watcher], DEFAULT_ENDPOINT_DEALER,
            DEFAULT_ENDPOINT_SUB, multicast_endpoint=DEFAULT_ENDPOINT_MULTICAST)
        self._arbiter.start()
        self._client = zerorpc.Client()
        self._client.connect(port)

This function returns, but shortly afterwards in a separate thread, I get this error:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/python/lib/python2.7/site-packages/circus/_patch.py", line 21, in _bootstrap_inner
    self.run()
  File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 647, in run
    return Arbiter.start(self)
  File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
    return func(self, *args, **kw)
  File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 456, in start
    self.initialize()
  File "/python/lib/python2.7/site-packages/circus/util.py", line 319, in _log
    return func(self, *args, **kw)
  File "/python/lib/python2.7/site-packages/circus/arbiter.py", line 427, in initialize
    self.evpub_socket.bind(self.pubsub_endpoint)
  File "socket.pyx", line 432, in zmq.core.socket.Socket.bind (zmq/core/socket.c:4022)
  File "checkrc.pxd", line 21, in zmq.core.checkrc._check_rc (zmq/core/socket.c:5838)
ZMQError: Address already in use

I have no idea why this is happening, especially since it doesn't happen in unit tests. Can anyone shed any light?

Answer 1

In general, when you get this type of "Address in use" error, it means that your program is trying to bind on an IP port number but something else got there first.

I am not familiar with this library, but since the error is caused by "evpub_socket.bind", I am going to guess that you have a conflict with the port number specified by the constant DEFAULT_ENDPOINT_SUB. From the circus source code I see these constants:

DEFAULT_ENDPOINT_DEALER = "tcp://127.0.0.1:5555" DEFAULT_ENDPOINT_SUB = "tcp://127.0.0.1:5556" DEFAULT_ENDPOINT_STATS = "tcp://127.0.0.1:5557"

Check your system (netstat) and see if any process is listening on ports 5555, 5556, 5557. Or perhaps you are running this program twice and you forgot about the first one.