How fix 'type=value_error' when loading a wizard-vicuna model to PrivateGPT?

550 Views Asked by At

I'm following a tutorial to install PrivateGPT and be able to query with a LLM about my local documents. I'm using a wizard-vicuna-13B.ggmlv3.q4_1.bin model, and as per the README.md adjusted the example.env file settings to a new .env file: from

PERSIST_DIRECTORY=db
MODEL_TYPE=GPT4All
MODEL_PATH=models/ggml-gpt4all-j-v1.3-groovy.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4

to

PERSIST_DIRECTORY=db
MODEL_TYPE=LlamaCpp
MODEL_PATH=models/wizard-vicuna-13B.ggmlv3.q4_1.bin
EMBEDDINGS_MODEL_NAME=all-MiniLM-L6-v2
MODEL_N_CTX=1000
MODEL_N_BATCH=8
TARGET_SOURCE_CHUNKS=4

I haven't changed any other code and successfully ran the ingest.py command that processes the given source documents, yet I receive the following error when trying to run the program with python3 privateGPT.py:

Using embedded DuckDB with persistence: data will be stored in: db
llama.cpp: loading model from models/wizard-vicuna-13B.ggmlv3.q4_1.bin
error loading model: unknown (magic, version) combination: 73726576, 206e6f69; is this really a GGML file?
llama_init_from_file: failed to load model
Traceback (most recent call last):
  File "/home/ubuntu/aiExplore/privateGPT23.7.11/privateGPT/privateGPT.py", line 83, in <module>
    main()
  File "/home/ubuntu/aiExplore/privateGPT23.7.11/privateGPT/privateGPT.py", line 36, in main
    llm = LlamaCpp(model_path=model_path, n_ctx=model_n_ctx, n_batch=model_n_batch, callbacks=callbacks, verbose=False)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 1 validation error for LlamaCpp
__root__
  Could not load Llama model from path: models/wizard-vicuna-13B.ggmlv3.q4_1.bin. Received error  (type=value_error)

In a HF thread with the same error, TheBloke advised to

update llama.cpp, or whatever UI/code you're using

Is this error related to updating llama.cpp? If so, how can I do that?

0

There are 0 best solutions below