Durable Function doesn't seem to be running Activity Functions in parallel

Question

I have a Durable Function App (.NET 7 Isolated Worker) where I'm running a foreach loop in the Orchestrator to have activity functions run in parallel in the following way:

 foreach (var item in itemList)
 {
     itemIndex++; //sent to activity function to determine if a new file needs to be created as we're storing results in blob storage
     activities.Add(context.CallActivityAsync(Constants.FUNCTION_NAME, new activityInput
     {
         activityInput data
     }));
 }

await Task.WhenAll(activities);

The completion of each activity function takes about 3 minutes (lots of business logic/DB access). As I continue to stress test the function and add more instances, the time to run increases linearly with however many Activity Functions are running. So, for a fan out of 10 functions, it takes roughly 30 minutes. This doesn't seem to be running in parallel based on the time it's taking to run (or maybe it is running in parallel but is so load-intensive it slows down)?

Is there something I'm missing or need to look for in the business logic of the activity function that could be halting the individual instances (like DB calls, other things)?

Answer 1

Check if you have set any concurrency limits for your Durable Function. In some cases it can cause activities to run sequentially instead of in parallel. Ensure that your host.json or function.json settings allow for sufficient concurrency.

High resource utilization on your worker instances can slow down parallel execution.

Can you check using Azure Monitor and Application Insights to gather more insights into the performance of your Durable Function and especially its dependencies. This can provide valuable metrics and traces to diagnose the issue.

https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-perf-and-scale#configuration-of-throttles