I'm trying to understand what the correct strategy is for storing and using embeddings in a vector database, to be used with an LLM. If my goal is to reduce the amount of work the LLM has to do when generating a response, (So you can think of a RAG implementation where I've stored text, embeddings I've created using an LLM, and metadata about the text.) I'm then trying to generate responses using say openai model from queries about the data, and I don't want to have to spend a bunch of money and time chunking up the text and creating embeddings for it every time I want to answer a query about it.
If I create a vector database, for example a chroma database and I use an LLM to create embeddings for a corpus I have. I save those embeddings into the vector database, along with the text and metadata. Would the database use those embeddings I created to find the relevant text chunks, or would it make more sense for the vector database to use it's own query process to find the relevant chunks (not using the embeddings the LLM created)?
Also do I want to pass the embeddings from the vector database to the LLM to generate the response, or do I pass the text that the vectore database found was most relevant to the LLM along with original text query so the LLM can then generate a response?
You'll initially chunk up your own content, create vector embedding using any embedding model (of your choice) and persist them to a vectore store (of your choice). Later, using techniques like prompt engineering, you will ask your questions to interact with your own proprietary data that is stored within the vector store to help find insights. During this process, your question/ask will need to be converted to embeddings (this is where embedding model comes in) to be able to sent to the vector store, which will in-turn use those embeddings to figure out the nearest / closest responses (see benefits of vector search) based on the corpus of data and will return back responses. One will then use these responses and pass it on the OpenAI (can be substituted w/ others of your choice) to prepare the final response.
Yes, the vector store will use the vector embeddings to find the relevant match based on your input embedding
With techniques like RAG, you would pass the text (using own proprietary data corpus) that the vector database found as relevant to the LLM to generate the final resposne.
You could check out some techniques as highlighted here to see how to build GenAI apps to take some inspiration.
I hope that helps.