Is it possible to get schema from a type field in trait?

Question

I have many HIVE tables, and I want wrap query by trait.

trait Trait1 {
  type DT <: Product
  val SCHEMA = org.apache.spark.sql.catalyst.ScalaReflection.schemaFor[DT].dataType.asInstanceOf[org.apache.spark.sql.types.StructType]
}
object O1 extends Trait1 {
  case class o1_info(gameId:Int, logdate:String)
  override type DT = o1_info
}

// TETS: println(O1.SCHEMA)

got compile error:

Error:(11, 73) No TypeTag available for Trait1.this.DT ......

Error:(11, 73) not enough arguments for method schemaFor: (implicit evidence$5: org.apache.spark.sql.catalyst.ScalaReflection.universe.TypeTag[Trait1.this.DT])org.apache.spark.sql.catalyst.ScalaReflection.Schema. Unspecified value parameter evidence$5.

Can you help me? Thanks

Answer 1

Try to introduce an abstract member for TypeTag, resolve it as implicit parameter in schemaFor call manually, and make SCHEMA lazy val.

import org.apache.spark.sql.catalyst.ScalaReflection
import org.apache.spark.sql.catalyst.ScalaReflection.universe.{TypeTag, typeTag}
import org.apache.spark.sql.types.StructType

trait Trait1{
  type DT <: Product
  def ttag: TypeTag[DT]
  lazy val SCHEMA = ScalaReflection.schemaFor[DT](ttag).dataType.asInstanceOf[StructType]
}
object O1 extends Trait1 {
  case class o1_info(gameId:Int, logdate:String)
  override type DT = o1_info
  override val ttag: TypeTag[o1_info] = typeTag[o1_info]
}

println(O1.SCHEMA)
//StructType(StructField(gameId,IntegerType,false), StructField(logdate,StringType,true))