GPT4All (2.0.0) Unable to instantiate model: code=129, Model format not supported

Question

enter image description here

when i use gpt4all==0.2.3 got error below (.venv) PS D:\Projects\owngpt> python gpt.py load INSTRUCTOR_Transformer max_seq_length 512 loading indexes loading indexes Traceback (most recent call last): File "D:\Projects\owngpt\gpt.py", line 29, in llm = GPT4All( File "D:\Projects\owngpt.venv\lib\site-packages\langchain\load\serializable.py", line 97, in init super().init(**kwargs) File "D:\Projects\owngpt.venv\lib\site-packages\pydantic\v1\main.py", line 341, in init raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All root GPT4All.init() got an unexpected keyword argument 'device' (type=type_error)

when i use gpt4all==2.0.0 got error below (.venv) PS D:\Projects\owngpt> python gpt.py load INSTRUCTOR_Transformer max_seq_length 512 loading indexes gguf_init_from_file: invalid magic number 67676a74 gguf_init_from_file: invalid magic number 67676a74 gguf_init_from_file: invalid magic number 67676a74 Traceback (most recent call last): File "D:\Projects\owngpt\gpt.py", line 23, in llm = GPT4All( File "D:\Projects\owngpt.venv\lib\site-packages\langchain\load\serializable.py", line 97, in init super().init(**kwargs) File "D:\Projects\owngpt.venv\lib\site-packages\pydantic\v1\main.py", line 341, in init raise validation_error pydantic.v1.error_wrappers.ValidationError: 1 validation error for GPT4All root Unable to instantiate model: code=129, Model format not supported (no matching implementation found) (type=value_error)

Answer 1

There was a problem with the model format in your code. gpt4all wanted the GGUF model format.

Below is the fixed code.

Dependencies:

pip install langchain faiss-cpu InstructorEmbedding torch sentence_transformers gpt4all

Fixed code:

from langchain.document_loaders import WebBaseLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.embeddings.huggingface import HuggingFaceInstructEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains import RetrievalQA
from langchain.llms import GPT4All

from langchain.globals import set_debug

set_debug(True)

# documents = TextLoader("data/text.txt").load()
documents = WebBaseLoader("https://leetcode.com/problem-list/top-interview-questions").load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1024, chunk_overlap=64)

texts = text_splitter.split_documents(documents)

instructor_embeddings = HuggingFaceInstructEmbeddings(
  model_name="hkunlp/instructor-large",
  model_kwargs={"device": "cpu"}
)

vectorstore = FAISS.from_documents(texts, instructor_embeddings)

question = "Difficulty Problem names"
retriever = vectorstore.as_retriever(search_kwargs={"k": 1})
docs = retriever.get_relevant_documents(question)

print("Loading indices")

llm = GPT4All(
    model="nous-hermes-llama2-13b.Q4_0.gguf",
    max_tokens=2048,
    allow_download=True,
    backend="gptj",
    verbose=True,
)

qa_chain = RetrievalQA.from_chain_type(
  llm, retriever=vectorstore.as_retriever()
)

qa_chain(question)

Output is incorrect unfortunately.

load INSTRUCTOR_Transformer
max_seq_length  512
Loading indices
{'query': 'Difficulty Problem names',
 'result': ' The difficulty of the problem is indicated by a number between 1 and 5, with 1 being the easiest and 5 being the hardest. The problem name corresponds to its LeetCode URL slug, which can be found in the link that appears when you click on the problem title. For example, if you click on "Pick One" from the list of problems above, it will take you to this page: https://leetcode.com/problems/pick-one/. The difficulty level for "Pick One" is 2, which means it\'s a medium problem.'}