Unit testing for spark-bigquery-connector

508 Views Asked by At

I am trying to write unit test case for my spark bigquery implementation. Say, If I want to test this below piece of code:

val wordsDF = (spark.read.format("bigquery")
              .option("table","bigquery-public-data:samples.shakespeare")
              .load()
              .cache())

And do some assertion on wordsDF

I am trying to think on the lines of:

  • Embedded kafka for kafka
  • Embedded cassandra for cassandra

Do we have any such library or alternative that can be used to write spark bigquery unit test

0

There are 0 best solutions below