Azure App Service scaling out specific webjobs possible?

Question

Is it possible to have multiple scale out conditions that scale out only specific webjobs.

For example:
1st scale out condition has triggered scaling out for webjob #1(scaling to 2 instances), but other webjobs will still be on one instance.

But now lets say another scale out condition was met and now I need to scale out webjob #2(scaling to 4 instances).

So I would like separate scaling on webjobs and I would like one webjob scaling to not affect other webjobs.

Is it even possible? ;)

Answer 1

Simply put, this is not possible. You should look at Azure Functions or if you want to containerize your app, Azure Container Apps.

Answer 2

Web Job scaling is tied to the underlying App Service Plan. To achieve independent scaling without writing your own management code, you will need to deploy each Web Job to it's own Web App and each of those Web Apps to their own App Service Plan. Then you can scale out the App Service Plans as you need to.

With Web Jobs, when you have different scaling needs a common deployment model is to group jobs together based on their resource usage and frequency of execution and deploy to multiple hosts, so one App Service Plan might be reserved for the resource intensive processing that requires both scale up and out, compared to the rest of your web jobs that might be more lightweight. It would be very expensive and a massive overhead to publish each web job independently, but you could do it.

Singleton Web Jobs

If you have written the code in some of your webjobs so that it does not support concurrent processing and therefore needs to be constrained to a single instance, then instead of deploying to a separate App Service Plan you can enable Singleton mode.

Set a continuous job as singleton
If a continuous job is set as singleton it'll run only on a single instance opposed to running on all instances. By default, it runs on all instances.

Triggered Functions
The Webjobs SDK provides a SingletonAttribute that ensures only one instance of a function runs, even when there are multiple instances of the host web app.

If Singleton doesn't suite your needs, then re-evaluate why you don't want to scale all the web jobs together. Splitting for scale reasons can be an anti-pattern if you cannot justify it. If you are paying for 1 web app that has higher instances and resources than another you can generally run the lightweight processes on the same App Service Plan and save the cost of running a new host, even if you deploy them to a separate Web App for management reasons.

If you are running the premium App Service Plan SKUs, then you can constrain individual Web Apps to run on specific instances or to only scale out to a specific number of instances. This is explained here: Limit number of instances of an azure webjob when scaling out, the link to the MS Docs is broken but here is a version of it: app service high density hosting