LangChain Ignore prompt

Question

I need help with ignoring a specific outcome from langchain. I need the QAstuffchain to reply with specifically: I don't know. Every time it doesn't have context to the given answer or it doesn't know the answer.

Here is what I have done so far and from what the docs tell me, this is how to do it. But it doesn't work.

I am also a novice coder please go easy on me.

const ignorePrompt = PromptTemplate.fromTemplate(
        "Given the text: {text}, answer the question: {question}. If the answer is not in the text or you don't know it, type: \"I don't know\""
      );
      const chain = loadQAStuffChain(llm, ignorePrompt);
      console.log("chain loaded");

Answer 1

It might just be that your prompt is not explicit enough, try out some different phrasings, such as:

"If the answer is not in the text, return only the following text: 'I don't know'"

"If the answer is not in the text, send only the text 'I don't know' "

or could go even more explicit:

"""
You are a question answering machine that receives text that may or may not contain the answer to a question. You can do two things:
 1. Return the answer to a question ONLY if the text contains the answer to the question.
 2. Return ONLY the text 'I dont know' if the text does not contain the answer to a question.

The text is {text}, and the question is {question}
"""

Try playing with the prompt phrasing and you may get what you're looking for - LLM's are not so good at knowing if they are correct or not so this might be a bit tricky.