I have a durable function orchestrator running a whole bunch of activities. If any of the activities throw and exception the failure message will appear in the output of the orchestrator.
I am running into scenarios whereby if I encounter a failure, if I run the orchestration again and it passes, the failure message from the prior orchestration remains in the output of the new orchestration, even though it is an entirely new execution!
I cannot reproduce this locally, but its happening in deployed environments and I can't understand why the new output should be polluted by results from the previous orchestration that was run and failed.
It is worth mentioning that when triggering the orchestration we are intentionally giving them the same instance Id per client that we run it for. This allows us to easily query if an instance is already running for a given client and I would still assume that history should remain separate. But likely this has something to do with the issue we're seeing.
Would appreciate any insight on this issue from anyone. Thanks