How to use Scala macros to create new partial functions or transform them?

Question

I am having trouble writing a macro that transforms a given partial function and creates a new partial function. For instance, I want to be able to decompose the given partial function into its elements - pattern binder, guard condition, and body; then I want to decompose the pattern binder and the guard condition into smaller pieces and reassemble new partial functions out of these pieces. However, I am getting strange errors at macro expansion that I can't debug.

The simplest problem that gives the same error is the code that decomposes the given partial function into the binder, the guard, and the body, and reassembles it back into the same partial function.

I can do this with a simple type PartialFunction[Int,Any] but not with types that involve case classes, PartialFunction[MyCaseClass,Any].

Here is the code that works and the code that doesn't.

Working code: take a partial function, destructure it using quasiquotes, assemble the same function again, and return it.

package sample

import scala.language.experimental.macros
import scala.reflect.macros.blackbox

object MacroTest {
  type Simple = PartialFunction[Int, Any]

  def no_problem(pf: Simple): Simple = macro no_problemImpl
  def no_problemImpl(c: blackbox.Context)(pf: c.Expr[Simple]) = {
    import c.universe._

    val q"{ case $binder  => $body }" = pf.tree
    q"{ case $binder  => $body }"
  }
}

This macro compiles and tests pass:

import MacroTest._

val testPf: Simple = { case x => x + 1 }
testPf(2) shouldEqual 3 // passes

  // now do the same with macro:
val result = no_problem({ case x => x + 1 })
result(2) shouldEqual 3 // passes

Non-working code: Exactly the same macro except for using a case class instead of Int as the argument of a partial function.

case class Blob(left: Int, right: Int)

type Complicated = PartialFunction[Blob, Any]

def problem(pf: Complicated): Complicated = macro problemImpl
def problemImpl(c: blackbox.Context)(pf: c.Expr[Complicated]) = {
    import c.universe._

    val q"{ case $binder  => $body }" = pf.tree
    q"{ case $binder  => $body }"
}

The code is exactly the same, only the type is different (Complicated instead of Simple).

The macro code compiles, but the test fails to compile (fails at macro expansion):

val blob = Blob(1,2)
val testPf: Complicated = { case Blob(x, y) => x + y }
testPf(blob) shouldEqual 3 // passes

  // now try the same with macro:
val result = problem({ case Blob(x, y) => x + y })
  // compile error when compiling the test code: 
Could not find proxy for case val x1: sample.Blob in List(value x1, method applyOrElse, <$anon: Function1>, value result, method apply, <$anon: Function0>, value <local MacroTestSpec>, class MacroTestSpec, package sample, package <root>) (currentOwner= value y )

I have simplified the problem to the barest minimum possible that still fails. In my actual code, the types are more complex, the partial functions may have guards, and I do transform the code of the partial function by rearranging its arguments and guards. I can sometimes make the transformation work when the guards are absent, but not when the argument of the partial function is a case class. Perhaps the presence of guards is not the root of the problem: the problem happens when there is a compound type with unapply somewhere. I get essentially the same error message as I get with this drastically simplified example shown above.

I cannot seem to solve this problem despite having tried many alternative ways of transforming the partial function:

• Use whitebox macro context
• Use plain quasiquotes, as in the examples shown above
• Use special quasiquotes for cases and patterns cq"...", pq"..." and q"{case ..$cases}" as shown in the documentation for quasiquotes
• Matching with guard: q"{case $binder if $guard => $body }", also with cq and pq quasiquotes
• Adding c.typecheck or c.untypecheck at various places (this used to be called resetAllAttrs, which is now deprecated)
• Use no quasiquotes but do everything in raw syntax trees: use Traverser with raw tree matching, such as case UnApply(Apply(Select(t@Ident(TermName(_)), TermName("unapply")), List(Ident(TermName("<unapply-selector>")))), List(binder)) if t.tpe <:< typeOf[Blob] and so on
• Try to replace Ident's in the pattern matcher by Ident's taken from the guard condition, and vice versa (this gives weird errors, "assertion failed", due to failed typechecking)
• Use Any instead of the specific types, returning PartialFunction[Any,Any], or a total function Function1[Blob,Any] and so on
• Use a type parameter T instead of Blob, parameterizing the macro and accepting PartialFunction[T,Any]

I would appreciate any help! I am using Scala 2.11.8 with straight macros (no "macro paradise").

Answer 1

I believe you're hitting a long standing issue in the Scala compiler. Typechecking is not idempotent in several cases, specifically extractors using unapply: SI-5465. There is no easy solution for this, but I can suggest two workarounds. Let me first explain the problem briefly.

The problem with def macros and typechecking

Def macros are expanded during the typechecking phase. As a consequence arguments to def macros are typed trees. Returning a well-typed or untyped tree is acceptable. However, returning a partially typed (your case) or ill-typed tree is very likely to trip the compiler, causing a typechecking error at best or an error in a subsequent phase. Note that quasiquotes generate untyped trees. How can these bad trees occur?

• Partially typed trees - usually by wrapping untyped code around the typed arguments or replacing parts of them with untyped code. In many cases you can get away with those, but not always.
• Ill-typed trees - by rearranging the typed arguments in a way that invalidates the original type information, e.g. splicing one argument into another. These will surely cause a problem.

Workarounds

Hopefully you can see that the problem is conceptual and deeply rooted. But you can take one of two approaches to solve the problem:

1. The hacky solution - do a roundtrip through a String representation of the final result:

   c.parse(showCode(q"{ case $binder  => $body }"))
   

   showCode will usually print parseable code, even when untypecheck is not idempotent. Of course this will incur some compile time performance overhead, which may or may not be acceptable for your use case.

2. The hard solution - manually typecheck the glue code by using internal compiler APIs. I can't explain how to do this in one post, but you would have to learn all about types, symbols and their owners. The worst part is that trees are mutable wrt type information. If you go that route, I recommend going through the source code of scala/async.

The best thing to do is probably avoid writing macros or wait until the semantic API of scala.meta is released and you can use it for def macros.