Trouble with PrivateGPT when using 'PGPT_PROFILES=local make run' command

Question

I have been exploring PrivateGPT, and now I'm encountering an issue with my PrivateGPT local server, and I'm seeking assistance in resolving it. When I execute the command PGPT_PROFILES=local make run, I receive an unhandled error, but I'm uncertain about the root cause. It appears that this error is somehow connected to my GPU.

I've been following the instructions in the official PrivateGPT setup guide, which you can find here: PrivateGPT Installation and Settings.

While running the command PGPT_PROFILES=local make run I got the following errors

OSError: libcufft.so.11: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

    raise ValueError(f"{lib_name} not found in the system path {sys.path}")
ValueError: libcublas.so.*[0-9] not found in the system path ['/home/bennison/Documents/yavar-poc/privateGPT', '/home/bennison/.pyenv/versions/3.11.6/lib/python311.zip', '/home/bennison/.pyenv/versions/3.11.6/lib/python3.11', '/home/bennison/.pyenv/versions/3.11.6/lib/python3.11/lib-dynload', '/home/bennison/.cache/pypoetry/virtualenvs/private-gpt-edPgKWgS-py3.11/lib/python3.11/site-packages', '/home/bennison/Documents/yavar-poc/privateGPT']

System Configurations
Operating System (OS): Ubuntu 20.04
CPU: 11th Gen Intel Core i5-1135G7 @ 2.40GHz (It has 4 cores)
GPU: NV137 / Mesa Intel Xe Graphics (TGL GT2)
RAM: 16GB
Python Version: 3.11.6

I'm asking for help to figure out why I'm getting this error and how to fix it. Any advice or suggestions would be very helpful.

Answer 1

I've sorted out the issues, and they all revolve around Nvidia CUDA. You might encounter three consecutive problems:

1. libcublas.so.*[0–9] not found,
2. libcudnn.so.8: cannot open shared object file, and
3. libcudnn.so.2: cannot open shared object file.

I've detailed the solutions in a Medium article. If you're in need of help, please refer to my article for the solutions. Medium Article

Answer 2

Python Version:

3.11 -- https://python.org/downloads.

Download Repository:

git clone https://github.com/imartinez/privateGPT.git

cd privateGPT

Install Dependencies:

pip install poetry

poetry install --extras "ui llms-llama-cpp embeddings-huggingface vector-stores-qdrant"

poetry run python scripts\setup

ollama serve

(in other terminal): make run

• open your web browser and navigate to 127.0.0.1:8001