How do you solve the problem of continuous output of the Openai API, such as letting the gpt api write an article. If the content is interrupted, you can continue to ask questions, so as to continue the output of the above content. This is very easy to do in ChatGPT, but after the Openai API adds the above to prompt, it will always report an error because the tokens exceed. If you don't add the above content, you can't continue the above content?
adds the above to prompt, it will always report an error because the tokens exceed.
What you likely want to do here build a system that can take the input, and make a summary of them. You can then keep track of the length of the inputs and ensure that the state of your "conversation memory" does not exceed the max number of tokens you want. Then, each time you make a request, you pass in the summarized/truncated version of your conversation history along with the prompt itself.