Is there a C# equivalent for the Java MongoDB Hadoop Connector?

Question

I'm playing with Mobius (the C# language binding for Spark) and the C# Driver for MongoDB. What I'm aiming to do is use MongoDB as the input/output for the Spark queries within my C# application. I know there's a Java MongoDB Hadoop Connector but I would like to continue using Mobius to write my Spark queries.

Answer 1

You could use MongoDB Spark Connector and DataFrame API in Mobius for querying MongoDB. The code to load data will look like

var mongoDbDataFrame = sqlContext.Read.Format("com.mongodb.spark.sql").Load()

Once the data is loaded, you could do Select(), Filter() operations on the DataFrame. You could also register the DataFrame as TempTable for using SQL queries using the code template below

mongoDbDataFrame.RegisterTempTable("MongDbDataFrameTempTable")
sqlContext.Sql("SELECT <columns> FROM MongDbDataFrameTempTable WHERE <condition>")

Note that you need to include the connector and its dependencies in the classpath and "--jars" parameter could be used for that.