NGINX won't run alongside Google Cloud Storage FUSE in Docker container

Question

I have two services defined in a docker-compose.yml file.

version: "3.9"
services:
  rtmp:
    build: ./live-stream
    ports:
      - "1935:1935"
      - "8080:8080"
    container_name: "live_server"
    privileged: true
    environment:
      GOOGLE_APPLICATION_CREDENTIALS: "/service-account-key.json"
    volumes:
      - ./data:/tmp/hls
  auth:
    build: ./auth
    container_name: auth_server

Inside live-stream I have the following Dockerfile:

FROM tiangolo/nginx-rtmp

RUN apt-get update \
&& apt-get install -y sudo

COPY index.html /www/
COPY service-account-key.json .

ENV GCSFUSE_REPO gcsfuse-buster
RUN echo $GCSFUSE_REPO
RUN echo "deb https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | sudo tee /etc/apt/sources.list.d/gcsfuse.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
RUN sudo apt-get update -y
RUN sudo apt-get install fuse gcsfuse -y

RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] 
https://packages.cloud.google.com/apt cloud-sdk main" | sudo tee -a 
/etc/apt/sources.list.d/google-cloud-sdk.list && curl 
https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo gpg --dearmor -o 
/usr/share/keyrings/cloud.google.gpg && sudo apt-get update -y && sudo apt-get 
install google-cloud-sdk -y

RUN sudo gcloud auth activate-service-account --key-file=/service-account-key.json
ENV GOOGLE_APPLICATION_CREDENTIALS "/service-account-key.json"
RUN mkdir /ls_media
CMD ["sudo", "gcsfuse", "--foreground", "--key-file", "/service-account-key.json", "****", "/ls_media"]

I'm using the tiangolo/nginx-rtmp image to create a nginx rtmp server. The RTMP server works perfectly when gcsfuse is not running, however as soon as it starts up, the NGINX server is no longer accessible.

There are no error logs from the container. I've tested each line and the NGINX server works fine up to CMD ["sudo", "gcsfuse", "--foreground", "--key-file", "/service-account-key.json", "****", "/ls_media"]

Here is my NGINX config.

events {}

rtmp {
    server {
        listen 1935;
        application live {
            live on;
            hls on;
            hls_path /tmp/hls;
            hls_fragment 2s;
            hls_playlist_length 40s;

            on_publish http://auth_server:8080/auth;
        }
    }
}

http {
    server {
        listen 8080;
        location / {
            root /www;
        }

        location /hls {
            # Types this root accepts
            types { 
                application/vnd.apple.mpegurl m3u8;
                application/octet-stream ts;
            }

            root /tmp;
            add_header Cache-Control no-cache;
            add_header Access-Control-Allow-Origin *;
        }
    }
}

Answer 1

The issue is that the CMD in your Dockerfile is overriding the CMD from the base image. When you don't have this CMD in your Dockerfile then NGINX will run. When you do have it then it will run instead of NGINX. You want them both to run.

Suppose that your project looks something (vaguely!) like this:

├── docker-compose.yml
└── live-stream
    ├── Dockerfile
    ├── entrypoint.sh
    ├── index.html
    └── service-account-key.json

docker-compose.yml (I stripped out some things that weren't germane to the actual problem.)

version: "3.9"
services:
  rtmp:
    build: ./live-stream
    ports:
      - "1935:1935"
      - "8080:8080"
    container_name: live_server
    privileged: true
    environment:
      GOOGLE_APPLICATION_CREDENTIALS: "/service-account-key.json"

Dockerfile

FROM tiangolo/nginx-rtmp

COPY index.html /www/
COPY service-account-key.json .

ENV GCSFUSE_REPO gcsfuse-buster
RUN echo $GCSFUSE_REPO
RUN echo "deb https://packages.cloud.google.com/apt $GCSFUSE_REPO main" | tee /etc/apt/sources.list.d/gcsfuse.list
RUN curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
RUN apt-get update -y && \
    apt-get install fuse gcsfuse -y

RUN echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | tee -a /etc/apt/sources.list.d/google-cloud-sdk.list && \
    curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | gpg --dearmor -o /usr/share/keyrings/cloud.google.gpg && \
    apt-get update -y && \
    apt-get install -y google-cloud-sdk

RUN gcloud auth activate-service-account --key-file=/service-account-key.json
ENV GOOGLE_APPLICATION_CREDENTIALS "/service-account-key.json"
RUN mkdir /ls_media

COPY entrypoint.sh .
CMD /bin/bash entrypoint.sh

entrypoint.sh

#! /bin/bash

gcsfuse --foreground --key-file /service-account-key.json **** /ls_media &
nginx -g "daemon off;"

I don't have a valid service-account-key.json, so could not test all of the above. However, the key is that you are not running the entrypoint.sh script when the container launches. That script will run gcsfuse in the background and then launch NGINX. You will then effectively have two processes running in the container. This will probably work fine. However, in general you want to have just one process per container, so a better design might be to have a Docker Compose stack with two services, one for NGINX and the other for gcsfuse.

Also you don't need to use sudo inside your Dockerfile.