How to synchronise writes from multiple containers to the same volume on a swarm?

Question

Is it safe / possible to use flock in a docker swarm?

I'm looking for a safe way for multiple containers to write to the same file (on the same volume). I know that on a single host docker will bind mount. What I'm not certain about is how this works when spinning up containers via docker-compose on a swarm.

If I have multiple instances of the same image in a service and these all share a volume then when I start this on a swarm will the containers be started on separate hosts? If so how will the the volume be shared at an OS level? What synchronisation options do I have for controlling concurrent writes?

Answer 1

Create a NFS volume

Example:

  # create a reusable volume
  $ docker volume create --driver local \
      --opt type=nfs \
      --opt o=nfsvers=4,addr=192.168.1.1,rw \
      --opt device=:/path/to/dir \
      foo

  # inside a docker-compose file
  ...
  volumes:
    nfs-data:
      driver: local
      driver_opts:
        type: nfs
        o: nfsvers=4,addr=192.168.1.1,rw
        device: ":/path/to/dir"

Answer 2

docker volumes are local to the node on which they are created, they are not shared between docker swarm nodes.

When running in a multi-node swarm cluster (or in Kubernetes) the container can end up on any of the nodes inside the cluster. For shared access to a docker volume (or PVC in the case of Kubernetes) the volume must be backed by something like a Network File System that can be accessed from each of the nodes inside the cluster.

NFS version 4 implements close-to-open consistency. Full details of what NFS v4 implements are available in "9.3. Data Caching" of RFC 3530:

Share reservations and record locks are the facilities the NFS version 4 protocol provides to allow applications to coordinate access by providing mutual exclusion facilities.

Tip: named docker volumes are bind-mounts, the volumes are created on disk in (by default) /var/lib/docker/volumes and bind-mounted during the docker run -v <named_docker_volume>:<container_mount_path> ....