Stateful generators with Haskell pipes

Question

1. Suppose I want to model, using Haskell pipes, a Python Generator[int, None, None] which keeps some internal state. Should I be using Producer int (State s) () or StateT s (Producer int m) (), where m is whatever type of effect I eventually want from the consumer?

2. How should I think about the notion of transducers in pipes? So in Oleg's simple generators, there is

   type Transducer m1 m2 e1 e2  = Producer m1 e1 -> Producer m2 e2
   

   but I don't know what the analog is in pipes, because any Proxy objects that interact seem to rely on the same underlying monad m, not switching from m1 to m2. See the Prelude functions, for instance.

I think I'm just misunderstanding something fundamental about the way pipes works. Thanks for your help.

Answer 1

In pipes, you typically wouldn't use effects in the base monad m of your overall Effect to model the internal state of a Producer. If you really wanted to use State for this purpose, it would be an internal implementation detail of the Producer in question (discharged by a runStateP or evalStateP inside the Producer, as explained below), and the State would not appear in the Producer's type.

It's also important to emphasize that a Producer, even when it's operating in the Identity base monad without any "effects" at its disposal, isn't some sort of pure function that would keep producing the same value over and over without monadic help. A Producer is basically a stream, and it can maintain state using the usual functional mechanisms (e.g., recursion, for one). So, you definitely don't need a State for a Producer to be stateful.

The upshot is that the usual model of a Python Generator[int, None, None] in Pipes is just a Monad m => Producer Int m () polymorphic in an unspecified base monad m. Only if the Producer needs some external effects (e.g., IO to access the filesystem) would you require more of m (e.g., a MonadIO m constraint or something).

To give you a concrete example, a Producer that generates pseudorandom numbers obviously has "state", but a typical implementation would be a "pure" Producer:

randoms :: (Monad m) => Word32 -> Producer Int m ()
randoms seed = do
  let seed' = 1664525 * seed + 1013904223
  yield $ fromIntegral seed'
  randoms seed'

with the state maintained via recursion.

If you really decided to maintain this state via the State monad, the type of the Producer wouldn't change. You'd just use a State internally. The Pipes.Lift module provides some helpers (like evalStateP used here) to locally add a monad layer to facilitate this:

randoms' :: (Monad m) => Word32 -> Producer Int m ()
randoms' seed = evalStateP seed $ forever $ do
  x <- get
  let x' = 1664525 * x + 1013904223
  yield $ fromIntegral x'
  put x'

Oleg's simple generators are entirely different. His producers and consumers produce and consume values only through monadic effects, and "monad changing" is central to the implementation. In particular, I believe his consumers and transducers can only maintain state via a monadic effect, like a State monad, though I'd have to look a little more carefully to be sure.

In contrast, pipes proxies can produce and consume values and maintain internal state independent of the underlying base monad.

Ultimately, the analog of Oleg's transducers in pipes are simply the Pipes. Both consume values from a producer and yield values to a consumer. The monad changing in Oleg's transducers is just an implementation detail.