Scio Scala : SCOLLECTION TO LONG

Question

I am getting the max(a) of a pipeline in scala so the result is a SCollection[Long] and I want to transform the type to Long because I want after to substract the max(a) from min(a) and it's not possible to do it if the type is not Long.

I have tried to use asInstanceOf[Long] but it didn't work.

Any help would be appreciated.

val max = pipe.map(_.a).max              
val min = pipe.map(_.a).min           
val diff = max - min

Answer 1

You can use materialize to extract the max and min values from the SCollection as a closed Tap. The Tap will be available once the pipeline completes successfully. materialize() must be called before the ScioContext is run e.g.

 val max =  pipe.map(_.a).max.materialize
 val min =  pipe.map(_.a).min.materialize
 val result = sc.run().waitUntilDone()
 max.get(result).value.next() - min.get(result).value.next()