Space leaks in Haskell

Question

I have read it many times that lazy evaluation in Haskell may sometimes lead to space leaks. What kind of code can lead to space leaks? How to detect them? And what precautions can be taken on part of a programmer to avoid them?

Answer 1

You will get probably many answeres, this is the one, I have encountered when trying to do some 'real-world' application. I was using multithreading and some MVars to pass data around (MVar is something like locked shared memory). My typical pattern was:

a <- takeMVar mvar
putMVar mvar (a + 1)

And then, just sometimes, when a proper condition happened I did something like:

a <- takeMVar mvar
when (a > 10) ....

The problem is that the content of mvar was essentially (0 + 1 + 1 + 1 + ....)...which was quite intensive for numbers like 100k... This type of problem was quite pervasive in my code; unfortunately for multithreading applications it's very easy to get into such problems.

Detecting...what I did was starting haskell in the mode that produces data regarding memory consumption, starting and stopping different threads and looking if memory footprint is stable or not...

Anotomy of a thunk leak (with instructions how to debug it)

An example: Thunk memory leak as a result of map function

Answer 2

I've run into this problem when doing recursion over large data structures. The built up thunks get to be too much and then you get a space leak.

In Haskell, you need to be constantly aware of the possibility of running into a space leak. Since iteration doesn't exist, basically any recursive function has the potential to generate a space leak.

To avoid this problem, memoize recursive functions, or rewrite them tail-recursively.