is it reasonable to run docker processes under runit/daemontools supervision

Question

I have been running docker processes (apps) via

docker run …

But under runit supervision (runit is like daemontools) - so runit ensures that the process stays up, passes signals etc.

Is this reasonable? Docker seems to want to run its own demonization - but it isn't as thorough as runit. Furthermore, when runit restarts the app - a new container is created each time (fine) but it leaves a trace of the old one around - this seems to imply I am doing it in the wrong way.

Should docker not be run this way?

Should I instead set up a container from the image, just once, and then have runit run/supervise that container for all time?

Answer 1

Docker does do some management of daemonized containers: if the system shuts down, then when the Docker daemon starts it will also restart any containers that were running at the time the system shut down. But if the container exits on its own or the kernel (or a user) kills the container while it is running, the Docker daemon won't restart it. In cases where you do want a restart, a process manager makes sense.

I don't know runit so I can't give specific configuration guidance. But you should probably make the process manager communicate with the docker daemon and check to see if a given container id is running (docker ps | grep container_id or equivalent, or use the Docker Remote API directly). If the container has stopped, use Docker to restart it (docker run container_id) instead of running a new container. Or, if you do want a new container each time, then begin with docker run -rm to automatically clean it up when it exits or stops.

If you don't want your process manager to poll docker, you could instead run something that watches docker events.

You can get the container_id when you start the container as the return value of starting a daemon, or you can ask Docker to write this out to a file (docker run -cidfile myfilename, like a PID file)

I hope that helps or helps another runit guru offer more detailed advice.

Answer 2

Yes, I think running docker under runit makes sense. Typically when you start a process there is a way to tell it not to daemonize if it does by default since the normal way to hand-off from the runit run script to a process is via exec on the last line of your run script. For docker this means making sure not to set the -d flag.

For example, with docker you probably want your run script to look something like this:

#!/bin/bash -e
exec 2>&1
exec chpst -u dockeruser docker run -a stdin -a stdout -i ...

Using exec and chpst should resolve most issues with processes not terminating correctly when you bring down a runit service.