What is the best way to create a background Task for an ILogger

Question

I created a File ILogger provider. It has a background task that does the actual writing to disk so that a Log() call is not waiting for a disk write to complete.

What is the best way to create/manage this background task? I can put it in a Task.Run() that I then forget about. But that strikes me as problematic.

In addition, the ILogger.Provider.Dispose() is never called. I want the task to be disposed so it can close the file it has open.

And, when Dispose() is called, I need to end the loop inside the Task.Run(). Is it better in this case to just have a bool I set? Or should I use a CancellationToken?

Answer 1

Discussing this with a number of developers I respect, the consensus was two-fold. First, use a thread:

public void Start()
{
    _workerThread = new Thread(ProcessQueue)
    {
        IsBackground = true,
        Priority = ThreadPriority.BelowNormal
    };
    _workerThread.Start();
}

The second part was something I hadn't thought of. Use synchronous calls. No async/await, no Tasks. We're all so set on async everything we don't stop to think. But if the code in the thread is going to write to the log file, and that's the only thing it's doing, you want all that happening in the thread.

There's no advantage to creating tasks and there are disadvantages. So...

private void ProcessQueue()
{
    while (!_cancellationTokenSource.IsCancellationRequested)
    {
        _newItemEventSlim.Wait(_cancellationTokenSource.Token);

        var lines = new List<string>();
        while (_queue.TryDequeue(out var message))
            lines.Add(message);
        WriteLine(lines);

        Flush();

        _newItemEventSlim.Reset();
    }
}

WriteLine() and Flush() are both synchronous. Because what would you gain by having them be async?

Answer 2

You want to use a thread safe queue. The logger puts logging messages on the queue, and one or more threads/tasks pick messages from the queue and writes them to disk. Dispose should make the queue stop accepting new messages, and wait for any already queued messages to be written.

There are many ways to write such a queue, a very simple alternative is to use a blockingCollection:

public class QueueExample<T> : IAsyncDisposable
{
    private readonly Task task;
    private readonly BlockingCollection<T> queue = new();

    public QueueExample() => task = Task.Run(DoProcessing);

    public void Add(T item) => queue.Add(item);

    private void DoProcessing()
    {
        foreach (var item in queue.GetConsumingEnumerable())
        {
            // do processing
        }
    }

    public async ValueTask DisposeAsync()
    {
        queue.CompleteAdding();
        await task;
    }
}

The the loop will stop once CompleteAdding has been called and all items have been processed.

But my recommendation would be to use an existing logging framework. This will do buffering like this for you, as well as many other things that can be useful when logging.

I have mostly experience with nLog, and that is fairly easy to setup. Add the nuget dependency and put a configuration in your app.config file, something like this should work for .net framework at least:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <configSections>
    <section name="nlog" type="NLog.Config.ConfigSectionHandler, NLog" />
  </configSections>
  <nlog>
    <targets async="true">
      <target name="fileTarget"
              xsi:type="File"
              keepFileOpen ="true"
              openFileCacheTimeout ="30"
              fileName=".\Logs\Log-${date}.txt"/>
    </targets>

    <rules>
      <logger name="*" minlevel="Info" writeTo="fileTarget" />
    </rules>
  </nlog>
...

And create your logger like

private static readonly Logger log = LogManager.GetCurrentClassLogger();