QuotaExceededException: The remote server returned an error: (403) Forbidden. The request was terminated because the namespace XXX is being throttled

Question

When trying to retrieve a list of registrations for my notification hub I receive the following error:

[QuotaExceededException: The remote server returned an error: (403) Forbidden. The request was terminated because the namespace XXX is being throttled. Please wait 60 seconds and try again. TrackingId:c7e05299-24ba-4f9d-9017-885db746a032_G20,TimeStamp:11/19/2014 9:00:51 PM]
   Microsoft.ServiceBus.Common.AsyncResult.End(IAsyncResult result) +624
   Microsoft.ServiceBus.Messaging.ServiceBusResourceOperations.EndGetAll(IAsyncResult asyncResult, String& continuationToken) +12
   Microsoft.ServiceBus.NamespaceManager.EndGetAllRegistrations(IAsyncResult result) +33
   System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization) +52

This is only a test application so the number of devices is relatively low (~20) and so is the number of users of the code which is throwing the exception. The error message obviously indicates that we are going over our quota somehow, but I cannot tell in what aspect - none of the metrics in the Notification Hubs appear to be over 100 operations in the previous 24 hours. The number of available operations per day should be far more than we are using.

There is nowhere in the Azure Portal which seems to show the number of total operations, so I am at a loss for how to find the cause of this issue.

Strangely, this similar question - Azure QuotaExceededException - indicates that they received an indication of Max and Allowed numbers of operations, but my error shows no such thing.

Is there any way (besides Azure paid support) to find why I am being throttled?

Answer 1

Get all registrations is treated (and throttled!) as analytic operation. It means that it is not supposed to be used in your main runtime flow. Or in the other words - if you have to call it often, then something is wrong...

Describe your application in more detailed manner and I'll be glad to help to figure out a good NH usage pattern.

Answer 2

It turns out there is a bug in the NuGet package for Microsoft.ServiceBus 2.1.2.0 where calling NotificationHubClient.GetAllRegistrations(10) does not correctly retrieve only the top 10 registrations, instead it recursively retrieves ALL registrations in blocks of 10. In my case it turns out there were 250 registrations (most were old), so the API call was being made 25 times in quick succession (~5 seconds) which explains the QuotaExceededException.

The fix was to upgrade to the latest NuGet package for Microsoft.ServiceBus - currently 2.5.2.0.