I'm following a tutorial to install PrivateGPT and be able to query with a LLM about my local documents.
I'm using a wizard-vicuna-13B.ggmlv3.q4_1.bin
model, and as per the README.md adjusted the example.env file settings to a new .env file:
from
PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4
to
PERSIST_DIRECTORY=db
MODEL_TYPE=LlamaCpp
MODEL_PATH=models/wizard-vicuna-13B.ggmlv3.q4_1.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4
I haven't changed any other code and successfully ran the ingest.py
command that processes the given source documents, yet I receive the following error when trying to run the program with python3 privateGPT.py
:
Using embedded DuckDB with persistence: data will be stored in: db
llama.cpp: loading model from models/wizard-vicuna-13B.ggmlv3.q4_1.bin
error loading model: unknown (magic, version) combination: 73726576, 206e6f69; is this really a GGML file?
llama_init_from_file: failed to load model
Traceback (most recent call last):
File "/home/ubuntu/aiExplore/privateGPT23.7.11/privateGPT/privateGPT.py", line 83, in <module>
main()
File "/home/ubuntu/aiExplore/privateGPT23.7.11/privateGPT/privateGPT.py", line 36, in main
llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=False)
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for LlamaCpp
__root__
Could not load Llama model from path: models/wizard-vicuna-13B.ggmlv3.q4_1.bin. Received error (type=value_error)
In a HF thread with the same error, TheBloke advised to
update llama.cpp, or whatever UI/code you're using
Is this error related to updating llama.cpp? If so, how can I do that?