I'm encountering an issue with my Docker container, which is set up for face recognition and recommendation using FastAPI, Uvicorn, and Gunicorn with OpenBLAS. The container intermittently hangs, and the logs are flooded with the warning: "OpenBLAS Warning: Detect OpenMP Loop, and this application may hang. Please rebuild the library with USE_OPENMP=1 option."
While the logs indicate a potential issue with OpenMP, it doesn't seem to be the root cause of the hang. The container hosts a retraining API that updates the model in batches, and it works fine initially. However, after a few requests, the server responds with "Empty reply from server," leading to a hang. To resolve this, I need to restart the server.
Here's an excerpt from my Dockerfile:
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9
COPY ./requirements.txt /app/requirements.txt
RUN git clone https://github.com/xianyi/OpenBLAS.git && \
cd OpenBLAS && \
make USE_OPENMP=1 && \
make PREFIX=/usr/local install && \
cd .. && \
rm -rf OpenBLAS
RUN pip install -r /app/requirements.txt
RUN apt-get update && apt-get install ffmpeg libsm6 libxext6 -y
COPY ./app app
EXPOSE 8000
CMD [ "python", "app/server.py" ]
Despite the OpenBLAS warning, the hang appears unrelated. I suspect it might be a resource issue or a problem with my retraining API. Has anyone encountered a similar problem or can provide insights into debugging and resolving this issue? Any help is appreciated.