LongRunning Task behaviour

Question

I have a class that contains a BlockingCollection<Task> that is used to queue up a number of tasks to run in order (FIFO).

The queue itself is started on a separate long running task.

public class TaskQueue
{
    private readonly BlockingCollection<Task> _queue = new BlockingCollection<Task>();

    public TaskQueue()
    {
        Task.Factory.StartNew(() =>
        {
            ProcessQueue();
        }, TaskCreationOptions.LongRunning);
    }

    private async Task ProcessQueue()
    {
        foreach (var task in _queue.GetConsumingEnumerable())
            await task;
    }
    ...
}

When the application starts, I create many instances of TaskQueue as I need different queues for different components of the application.

The issue I am seeing is that even though ProcessQueue() is started on a LongRunning task .NET Long Running Task, after the first await of the task, it returns back on a .NET ThreadPool Worker i.e not a long running task anymore.

This results in all the instances of TaskQueue.ProcessQueue ending up on .NET ThreadPool Workers (after the first await), which then results in the application being very slow to start new tasks as it has reached the max number of concurrent running .NET ThreadPool Workers (I believe this is set to the number of cores the machine has, in my case 8), after which it will only create new .NET ThreadPool Workers every second or so.

So my questions are :

1. Is there a way to return back to a long running task?
2. Or should I just replace Task.Factory.StartNew with a new Thread? which I believe will solve the problem, but not sure if there is a better way.

Answer 1

Is there a way to return back to a long running task?

Yes, by not leaving the long running task in the first place. You can change the signature of the ProcessQueue to return void instead of Task like this:

private void ProcessQueue()
{
    foreach (var task in _queue.GetConsumingEnumerable())
        task.GetAwaiter().GetResult();
}

This way the LongRunning task will alternate between waiting to consume a task, and waiting for the consumed task to complete. The same thread will do all the waiting.

Btw both the original async version of the ProcessQueue, as well as the suggested here void version, are doing nothing constructive. They are not processing anything, they are just waiting. I assume that you have omitted the "process" part of the method, to keep the example simple. Otherwise the ProcessQueue would be just an exercise to futility.