Error encountered writing to Kudu using Spark / Scala

307 Views Asked by At

I'm trying to write data into Kudu from Spark and I'm getting this error

   java.lang.UnsupportedOperationException at org.apache.spark.sql.execution.datasources.parquet.VectorizedColumnReader.decodeDictionaryIds(VectorizedColumnReader.java:296) at org.apache.spark.sql.execution.datasources.parquet.VectorizedColumnReader.readBatch(VectorizedColumnReader.java:174) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.nextBatch(VectorizedParquetRecordReader.java:230) at org.apache.spark.sql.execution.datasources.parquet.VectorizedParquetRecordReader.nextKeyValue(VectorizedParquetRecordReader.java:137) at org.apache.spark.sql.execution.datasources.RecordReaderIterator.hasNext(RecordReaderIterator.scala:39) at org.apache.spark.sql.execution.datasources.FileScanRDD$$anon$1.hasNext(FileScanRDD.scala:105)

Code example:

df.write.options(Map("kudu.master"-> "ip:7051",
                     "kudu.table"-> "test_kudu")).mode("append").kudu

Used libraries:

org.apache.kudu:kudu-spark2_2.11:1.8.0  
org.apache.kudu:kudu-client:1.8.0 

Thanks!

0

There are 0 best solutions below