How to await Parallel Linq actions to complete

5.7k Views Asked by At

I'm not sure how I'm supposed to mix plinq and async-await. Suppose that I have the following interface

public interface IDoSomething (
    Task Do();
}

I have a list of these which I would like to execute in parallel and be able to await the completion of all.

public async Task DoAll(IDoSomething[] doers) {
    //Execute all doers in parallel ideally using plinq and 
    //continue when all are complete
}

How to implement this? I'm not sure how to go from parallel linq to Tasks and vice versa.

I'm not terribly worried about exception handling. Ideally the first one would fire and break the whole process as I plan to discard the entire thing on error.

Edit: A lot of people are saying Task.WaitAll. I'm aware of this but my understanding (unless someone can demonstrate otherwise) is that it won't actively parallelize things for you to multiple available processor cores. What I'm specifically asking is twofold -

  1. if I await a Task within a Plinq Action does that get rid of a lot of the advantage since it schedules a new thread?

  2. If I doers.AsParallel().ForAll(async d => await d.Do()) which takes about 5 second on average, how do I not spin the invoking thread in the meantime?

2

There are 2 best solutions below

11
i3arnon On BEST ANSWER

What you're looking for is this:

public Task DoAllAsync(IEnumerable<IDoSomething> doers)
{
    return Task.WhenAll(doers.Select(doer => Task.Run(() => doer.Do())));
}

Using Task.Run will use a ThreadPool thread to execute each synchronous part of the async method Do in parallel while Task.WhenAll asynchronously waits for the asynchronous parts together that are executing concurrently.


This is a good idea only if you have substantial synchronous parts in these async methods (i.e. the parts before an await) for example:

async Task Do()
{
    for (int i = 0; i < 10000; i++)
    {
        Math.Pow(i,i);
    }

    await Task.Delay(10000);
}

Otherwise, there's no need for parallelism and you can just fire the asynchronous operations concurrently and wait for all the returned tasks using Task.WhenAll:

public Task DoAllAsync(IEnumerable<IDoSomething> doers)
{
    return Task.WhenAll(doers.Select(doer => doer.Do()));
}
3
spender On
public async Task DoAll(IDoSomething[] doers) {
    //using ToArray to materialize the query right here
    //so we don't accidentally run it twice later.
    var tasks = doers.Select(d => Task.Run(()=>d.Do())).ToArray();
    await Task.WhenAll(tasks);
}