Graceful shutdown of IHostedService / BackgroundService

Question

What is the best way to block shutdown of a service while jobs finish? I have a few background services that listen for messages on a message bus and handle processing.Some implement IHostService directly and some utilize BackgroundService

I know loops with a delay is absolutely not the best way, but this is the simplest illustration I can demonstrate. Another thought I had was a counter with number of total number of active jobs, but that would need a lock around it and I don't think that's a great idea either.

public interface IMessageBus
{
   Task<int> Subscribe(CancellationToken cancellationToken);
   Task Unsubscribe(int subscriptionId, CancellationToken cancellationToken);
}

public class Worker : IHostedService, IDisposable
{
    private readonly ILogger logger;
    private readonly IMessageBus messageBus;
    private readonly List<Task> activeTasks;
    private CancellationTokenSource stoppingCts;
    int? subscriptionId;

    public Worker(ILogger logger, IMessageBus messageBus)
    {
        this.logger = logger;
        this.messageBus = messageBus;
        this.activeTasks = new List<Task>();
    }

    public async Task StartAsync(CancellationToken cancellationToken)
    {
        this.stoppingCts = new CancellationTokenSource();

        this.logger.LogDebug("Subscribing to message bus.");
        this.subscriptionId = await this.messageBus.Subscribe(cancellationToken, this.Process);
    }

    public async Task StopAsync(CancellationToken cancellationToken)
    {
        this.logger.LogDebug("Stopping Worker");

        // this will stop new messages coming in....
        if (this.subscriptionId.HasValue)
        {
            this.logger.LogDebug("Unsubscribed from message bus");
            await this.messageBus.Unsubscribe(this.subscriptionId.Value, cancellationToken);
        }

        // this should block until existing jobs finished....i don't like it though
        this.logger.LogDebug("Waiting for all jobs to finish");
        while (!cancellationToken.IsCancellationRequested && this.activeTasks.Count > 0)
        {
            await Task.Delay(10);
            this.logger.LogDebug("Still waiting for all jobs to finish");
        }

        this.logger.LogDebug("Worker stopped.");
    }

    // this does real work, would be multiple running concurrently
    public async Task<bool> Process()
    {
        this.logger.LogInformation("Starting task..");

        var task = Task.Delay(100, this.stoppingCts.Token);
        this.activeTasks.Add(task);
        await task;
        this.activeTasks.Remove(task);

        this.logger.LogInformation("Task completed");

        return true;
    }

    public void Dispose()
    {
        this.stoppingCts?.Cancel();
    }
}

Answer 1

Use CountdownEvent initialized with 1 and wait until it reaches zero like here. Increment the counter in the event when task is started, and decrement when finished using AddCount and Signal respectively. Just remember about last call to Signal in StopAsync to decrement the initial value.