Running scala futures somewhat in parallel

Question

I have some scala futures. I can easily run them in parallel with Future.sequence. I can also run them one-after-another with something like this:

def serFut[A, B](l: Iterable[A])(fn: A ⇒ Future[B]) : Future[List[B]] =
  l.foldLeft(Future(List.empty[B])) {
    (previousFuture, next) ⇒
      for {
        previousResults ← previousFuture
        next ← fn(next)
      } yield previousResults :+ next
  }

(Described here). Now suppose that I want to run them slightly in parallel - ie with the constraint that at most m of them are running at once. The above code does this for the special case of m=1. Is there a nice scala-idiomatic way of doing it for general m? Then for extra utility, what's the most elegant way to implement a kill-switch into the routine? And could I change m on the fly?

My own solutions to this keep leading me back to procedural code, which feels rather wimpy next to scala elegance.

Answer 1

You can achieve it by using an ExecutionContext which uses a pool of max m threads: How to configure a fine tuned thread pool for futures?

Put the implicit val ec = new ExecutionContext { ... somewhere within the scope of the serFut function so that it is used when creating futures.

Answer 2

The easiest way is to define your ExecutionContext like

implicit val ec = ExecutionContext.fromExecutor(Executors.newFixedThreadPool(m))