Cloud Haskell - How to write "pure" for Closures?

Question

I've been playing with Cloud Haskell. I've noticed in the hackage documentation there's a kind of applicative interface. But in particular I'm trying to find or write a function closurePure with the following signature:

closurePure :: (Typeable a, Binary a) => a -> Closure a

This is basically a restricted version of pure.

Whilst the Closure datatype itself is abstract, the following closure provided:

closure :: Static (ByteString -> a) -> ByteString -> Closure a

So I can get this far:

closurePure :: (Typeable a, Binary a) => a -> Closure a
closurePure x = closure ??? (encode x)

The problem is what to put where the ???s are.

My first attempt was the following:

myDecode :: (Typeable a, Binary a) => Static (ByteString -> a)
myDecode = staticPtr (static decode)

But upon reading the GHC docs on static pointers, the show example suggested to me that you can't have a constraint because a constrained function doesn't have a Typeable instance. So I tried the work around suggested using Dict:

myDecode :: Typeable a => Static (Dict (Binary a) -> ByteString -> a)
myDecode = staticPtr (static (\Dict -> decode))

But now I've got the wrong type that doesn't fit into the closure function above.

Is there anyway to write closurePure or something similar (or have I missed it in the Cloud Haskell docs)? Raising binary plain types to Closures seems essential to using the applicative interface given, but I can't work out how to do it.

Note that I can do this:

class StaticDecode a where
  staticPtrDecode :: StaticPtr (ByteString -> a)

instance StaticDecode Int where
  staticPtrDecode = static Data.Binary.decode

instance StaticDecode Float where
  staticPtrDecode = static Data.Binary.decode

instance StaticDecode Integer where
  staticPtrDecode = static Data.Binary.decode

-- More instances etc...

myPure :: forall a. (Typeable a, StaticDecode a, Binary a) => a -> Closure a
myPure x = closure (staticPtr staticPtrDecode) (encode x)

Which works well but basically requires me to repeat an instance for each Binary instance. It seems messy and I'd prefer another way.

Answer 1

You're right, Closure has an applicative-like structure, a fact made even more explicit in both the interface and the implementation of distributed-closure. It's not quite applicative, because in the pure case we do have the additional constraint that the argument must somehow be serializable.

Actually, we have a stronger constraint. Not only must the argument be serializable, but the constraint must itself be serializable. Just like it's hard to serialize functions directly, you can imagine that it's hard to serialize constraints. But just like for functions, the trick is to serialize a static pointer to the constraint itself, if such a static pointer exists. How do we know that such a pointer exists? We could introduce a type class with a single method that gives us the name of the pointer, given a constraint:

class GimmeStaticPtr c where
  gimmeStaticPtr :: StaticPtr (Dict c)

There's a slight technical trick going on here. The kind of the type index for StaticPtr is the kind *, whereas a constraint has kind Constraint. So we reuse a trick from the constraints library that consists in wrapping a constraint into a data type (Dict above), which like all data types is of kind *. Constraints that have an associated GimmeStaticPtr instance are called static constraints.

In general, it's sometimes useful to compose static constraints to get more static constraints. StaticPtr is not composable, but Closure is. so what distributed-closure actually does is define a similar class, that we'll call,

class GimmeClosure c where
  gimmeClosure :: Closure (Dict c)   

Now we can define closurePure in a similar way that you did:

closurePure :: (Typeable a, GimmeClosure (Binary a)) => a -> Closure a

It would be great if in the future, the compiler could resolve GimmeClosure constraints on-the-fly by generating static pointers as needed. But for now, the thing that comes closest is Template Haskell. distributed-closure provides a module to autogenerate GimmeClosure (Cls a) constraints at the definition site for class Cls. See withStatic here.

Incidentally, Edsko de Vries gave a great talk about distributed-closure and the ideas embodied therein.

Answer 2

Let's take a moment to consider what you are asking for. Recall that typeclasses are basically shorthand for dictionary passing. So let's rewrite:

data BinaryDict a = BinaryDict 
  { bdEncode :: a -> ByteString
  , bdDecode :: ByteString -> a
  }

Now you wish to write a function:

closurePure :: (Typeable a) => BinaryDict a -> a -> Closure a

Your attempt is:

closurePure bdict = closure (staticPtr (static (bdDecode bdict))) . bdEncode bdict

Now that we can see what's going on explicitly, we can see that static's argument cannot be closed. If BinaryDicts were allowed to be created willy nilly, say from user data, this function would be impossible. We would instead need:

closurePure :: (Typeable a) => Static (BinaryDict a) -> a -> Closure a

That is, we need entries for the needed Binary instances in the static pointer table. Hence your enumeration solution, and why I suspect that such a solution is required. We also can't expect to enumerate it too automatically, because there are infinitely many instances.

It seems silly to me, however, since instances seem like just the sorts of things that you would want to be static automatically. They are static by nature (what's that, reflection? I can't hear you). This was probably at least ruminated about in the distributed Haskell papers (I haven't read them).

We could solve this problem in general by simply creating a class that concretely enumerates every instance of every class (déjà vu?).

class c => StaticConstraint c where
    staticConstraint :: StaticPtr (Dict c)
instance StaticConstraint (Show Int) where
    staticConstraint = static Dict
-- a handful more lines...

Somewhat more seriously, if you really don't want to enumerate (I don't blame you), you can at least ease the pain with a calling convention:

closurePure :: (Typeable a, Binary a) => StaticPtr (ByteString -> a) -> a -> Closure a
closurePure decodePtr = closure (staticPtr decodePtr) . encode

someClosure :: Closure Int
someClosure = closurePure (static decode) 42

This nonsense is necessary because static is a "syntactic form" rather than a function -- by mentioning it, we indicate that the Binary instance for Int must actually be generated and recorded in the static pointer table.

If you are feeling cheeky you could enable {-# LANGUAGE CPP #-} and

-- PURE :: (Binary a, Typeable a) => a -> Closure a, I promise
#define PURE (closurePure (static decode))

someClosure :: Closure Int
someClosure = PURE 42

Maybe someday Haskell will take the next step and graduate to the time-tested Segmentation fault (core dumped) of its predecessors instead of spouting off those arrogant type errors.