how handle this error that i am facing when trying to write from SQL to Kudu via Pyspark

310 Views Asked by At

I want to write a huge table from SQL to Kudu Table, I am not able to write it to a Kudu table.

With the following code:

kuduDF.write.format('org.apache.kudu.spark.kudu')
              .option('kudu.master',kudu_master)
              .option('kudu.table',"impala::"+kudu_database_name+"."+kudu_table_name).mode('append')
              .option("kudu.operation", operation_type)
              .save()

but I am getting the following error:

An error occurred while calling o202.save. : java.lang.AbstractMethodError: org.apache.kudu.spark.kudu.DefaultSource.createRelation(Lorg/apache/spark/sql/SQLContext;Lorg/apache/spark/sql/SaveMode;Lscala/collection/immutable/Map;Lorg/apache/spark/sql/Dataset;)Lorg/apache/spark/sql/sources/BaseRelation;

1

There are 1 best solutions below

0
On

Can you try with this jar kudu-spark2-tools_2.11.jar . Which jar did you use to make it work