Is it a good practice to use TaskFactory.StartNew for every layer in push model?

Question

Lets assume that I have a several layers:

1. Manager that reads data from a socket
2. Manager that subscribes to #1 and takes care about persisting the data
3. Manager that subscribes to #2 and takes care about deserialization of the data and propagating it to typed managers that are insterested in certain event types
4. WPF Controllers that display the data (are subscribed to #3)

As of right now I use

TaskFactory.StartNew(()=>subscriber.Publish(data)); 

on each layer. The reason for this is that I don't want to rely on the fact that every manager will do his work quickly and that ex. Socket manager is not stuck.

Is this a good approach?

Edit Let's say that Socket manager receives a price update There are 10 managers subscribed to Socket manager so when Socket manager propagates the message .StartNew is called 10 times.

Managers #2,#3 do nothing else but to propagate the message by .StartNew to a single subscriber

So ultimately per 1 message from socket 30x .StartNew() is called.

Answer 1

It seems a reasonable approach.

However, if one could meaningfully do:

subscriber.PublishAsync(data).LogExceptions(Log);

Where LogExceptions is something like:

// I'm thinking of Log4Net here, but of course something else could be used.
public static Task LogExceptions(this Task task, ILog log)
{
  return task.ContinueWith(ta => LogFailedTask(ta, log), TaskContinuationOptions.OnlyOnFaulted);
}
private static void LogFailedTask(Task ta, ILog log)
{
  var aggEx = ta.Exception;
  if(aggEx != null)
  {
    log.Error("Error in asynchronous event");
    int errCount = 0;
    foreach(var ex in aggEx.InnerExceptions)
      log.Error("Asynchronous error " + ++errCount, ex);
  }
}

So that fire-and-forget use of tasks still have errors logged, and PublishAsync in turn makes use of tasks where appropriate, then I'd be happier still. In particular, if the "publishing" has anything that would block a thread that can be handled with async like writing to or reading from a database or file system then the thread use could scale better.

Answer 2

Regarding Task.Run vs. TaskFactory.StartNew, they are essentially identical under the hood. Please read the following link: http://blogs.msdn.com/b/pfxteam/archive/2014/12/12/10229468.aspx

Even though these methods use the ThreadPool for decent performance, there is overhead associated with constantly creating new Tasks on-the-fly. Task is generally used more for infrequent, fire-and-forget type workload. Your statement of "30x .StartNew() per 1 message from the socket" is a bit concerning. How often do socket messages arrive? If you are really concerned with latency, I think the better way of doing this is that each manager should have its own dedicated thread. You can use a BlockingQueue implementation so that the threads are waiting to consume a parent input item in the parent's queue. This would be preferable to a simple spinlock, for example.

This is the sort of architecture used regularly in financial market messaging subscription and decoding that needs the fastest possible performance. Also keep in mind that more threads do not always equate to faster performance. If the threads have any shared data dependencies, they will all be contending for the same locks, causing context switching on one another, etc. This is why a preset number of dedicated threads can usually win out vs. a greater number of threads created on-the-fly. The only exception I can think of would be "embarrassingly parallel" tasks where there are no shared data dependencies at all. Note that dependencies can exist on both the input side and the output side (anywhere there is a lock the threads could run into).