Is there an Encoder for Map type in Java Spark?

1.1k Views Asked by At

I am trying to create a custom Aggregator function producing a Map as the result, however it requires an Encoders. As referenced in https://spark.apache.org/docs/2.1.0/api/java/org/apache/spark/sql/Encoders.html, there isn't one for now.

Does anyone know a workaround for this? Thank you in advance!

1

There are 1 best solutions below

3
leleogere On BEST ANSWER

This is not exactly an answer, but if someone came here with the same question but in Scala instead of Java, they can use the ExpressionEncoder:

import org.apache.spark.sql.catalyst.encoders.ExpressionEncoder

// ...

def outputEncoder: Encoder[Map[String, Int]] = ExpressionEncoder()

For the original question in Java, this link about how to write a custom encoder might help.