azure servicebus maxConcurrentCalls totally ignored

Question

I have thsese in my host.json but every time i run the function it runs in parallel runing much more threads then 1 ( so much as there are messages in queue)

{
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "prefetchCount": 1,
      "messageHandlerOptions": {
        "maxConcurrentCalls": 1
      }
    }
  }
}

my function

 [FunctionName(nameof(ResourceEventProcessorFunction))]
    public async Task Run([ServiceBusTrigger("%TopicName%", "%SubscriptionName%", Connection = "ServiceBusConnection", IsSessionsEnabled = true)]Message message, IMessageSession messageSession, ILogger log)

Answer 1

so the problem was that every message had a differnet sessionId. Disabling sessionId on subscription in azure solved this problem.

In details below for bounty :D azure docs doesnt exactly specify how to limit thread number, but I looked a bit dipper.

there is MessageRecievePump and SessionRecievePump one uses MaxConcurrentCalls the other one MaxConcurrentSessions and MaxConcurrentAcceptSessionCalls

be aware of this if you include session in your subscription (MaxConcurrentCalls doesnt work) it works only when session id is the same. when session is differnt try to use MaxConcurrentSessions or MaxConcurrentAcceptSessionCalls but be aware there are no docs about this....

Answer 2

Maybe you can set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1 to make the function run only one instance at a time.

If you develop locally, you can set it in local.settings.json, if you develop in Azure portal, you can set it in Configuration -> Application settings.

Note：

1. If you set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1, your function will not scale out and can only run in one instance.

2. In addition to setting WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT, you still need to set maxConcurrentCalls to 1

3. This setting is in preview. An app property for function max scale out has been added and is the recommended way to limit scale out.

For more details, you can refer to this official document.

Answer 3

Leveraging sessions

Since you are using sessions, you can use the same sessionId for all messages, and they will be processed in order by a single instance, regardless of the settings in your host.json.

https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-sessions

Using Singleton attribute

If you can't use the sessionId for your purpose, you should try the [Singleton] attribute on your function. This will ensure that only one instance across all of your function instances will process the request.

We have this working successfully for WebJobs in production, and it should work just the same for Azure Functions. If you have dedicated app service plans, using this attribute should be enough. This is not recommended for a consumption plan.

[Singleton] does work on functions. The Azure Function host will create or wait for a lock in the Azure Storage account. The lock is the host ID which should be the same for all hosts of an app across all instances - so all instances share this lock and will only allow one execution to occur at a time.

To test this I put 1000 queue messages at once on a function with [Singleton]. The function would wake up, emit the invocation ID, sleep, and then emit the invocation ID. After processing all 1000 I looked at logs and never saw invocation IDs overlap. Only one invocation would happen globally at a time.

https://github.com/Azure/azure-functions-host/issues/912#issuecomment-419608830

 [Singleton]
 [FunctionName(nameof(ResourceEventProcessorFunction))]
    public async Task Run([ServiceBusTrigger("%TopicName%", "%SubscriptionName%", Connection = "ServiceBusConnection", IsSessionsEnabled = true)]Message message, IMessageSession messageSession, ILogger log)

In a consumption plan

Continuing the quote above:

With that said I think the recommendation is: [Singleton] isn't recommended for consumption hosted function plans. If you have a dedicated app service plan it's fine (as you are paying for the instance anyway). If you want to enforce [Singleton] like behavior in a consumption plan you are likely best to:

1. Set WEBSITE_MAX_DYNAMIC_APPLICATION_SCALE_OUT to 1 so you never scale to more than one instance
2. Set the host.json file to only allow 1 concurrent execution at a time for that trigger (for instance a batch size of 1 for Azure Queues).

https://github.com/Azure/azure-functions-host/issues/912#issuecomment-419608830

{
  "version": "2.0",
  "extensions": {
    "serviceBus": {
      "prefetchCount": 1,
      "messageHandlerOptions": {
        "maxConcurrentCalls": 1
      }
    }
  }
}