I am trying to execute this example to create spatial data frame from csv using sedona
SparkSession sparkSession = SedonaContext.builder()
.master("local[*]") // Delete this if run in cluster mode
.appName("readTestScala") // Change this to a proper name
.getOrCreate();
String csvFilePath = "test.csv";
Dataset<Row> rawDf = sparkSession.read()
.format("csv")
.option("header", "true")
.option("inferSchema", "true")
.load(csvFilePath);
// Register the DataFrame as a temporary view
rawDf.createOrReplaceTempView("data");
// Perform a SQL query to convert the WKT representation into a geometry column
Dataset<Row> spatialDF = sparkSession.sql("SELECT *, ST_GeomFromWKT(geometry) AS geom FROM data");
but getting this exception
Caused by: org.apache.spark.sql.AnalysisException: Undefined function:
'ST_GeomFromWKT'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.
I followed their website and added these libraries
<dependency>
<groupId>org.apache.sedona</groupId>
<artifactId>sedona-spark-shaded-3.0_2.12</artifactId>
<version>1.4.0</version>
</dependency>
<dependency>
<groupId>org.apache.sedona</groupId>
<artifactId>sedona-viz-3.0_2.12</artifactId>
<version>1.4.0</version>
</dependency>
<!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper -->
<dependency>
<groupId>org.datasyslab</groupId>
<artifactId>geotools-wrapper</artifactId>
<version>1.4.0-28.2</version>
</dependency>
I have spark version 3.4 and scala version 2.12
What else is needed to execute these ST_ functions?