LLaMA: reached the end of the context window so resizing

Question

I'm currently working on a project where I'm using the LLaMA library for natural language processing tasks. However, I've encountered an error message that I'm struggling to resolve. The error states: "LLaMA: reached the end of the context window so resizing."

Could someone please shed some light on what this error message means and how I can address it? I have searched the documentation and various online resources, but I couldn't find any information specifically related to this error. I want to understand the root cause and find a suitable solution.

I'm running a local model LLM (gpt4all) on my computer and I'm summarising a pdf. Sometimes this error occurs. I want to understand the background of it

Answer 1

do you wait long enough? It should continue from the last token, just after the resizing completes. My solution right now was to close the session before reaching the context limit, so one has to either do it often enough or do it on demand/count the tokens etc.

It starts:

with mgpt.chat_session() as session:
  ...
  #a deep loop which has speech recognition as well etc.

If I want to interrupt the session, I use a flag:

if bNewChatSession: session.close(); break

Then it returns up in nested loops/blocks and starts a new fresh session. However it will forget the current context.

If it has to be more continuous to the previous one, the first prompt for the new session may be prepended with part of the ending of the previous one, or it may be partially edited or some words selected and then appended with the ending part exactly etc. All this prepended in the first prompt. Some trade-off may be made, too long prepended prompt will be slow so may be similar to the built-in context-resizing.