How to run function ST_GeomFromWKT within sedona context

305 Views Asked by At

I am trying to execute this example to create spatial data frame from csv using sedona

SparkSession sparkSession = SedonaContext.builder()
            .master("local[*]") // Delete this if run in cluster mode
            .appName("readTestScala") // Change this to a proper name
            .getOrCreate();

        String csvFilePath = "test.csv";

        Dataset<Row> rawDf = sparkSession.read()
            .format("csv")
            .option("header", "true")
            .option("inferSchema", "true")
            .load(csvFilePath);

        // Register the DataFrame as a temporary view
        rawDf.createOrReplaceTempView("data");

        // Perform a SQL query to convert the WKT representation into a geometry column
        Dataset<Row> spatialDF = sparkSession.sql("SELECT *, ST_GeomFromWKT(geometry) AS geom FROM data");

but getting this exception

Caused by: org.apache.spark.sql.AnalysisException: Undefined function: 
'ST_GeomFromWKT'. This function is neither a registered temporary function nor a permanent function registered in the database 'default'.

I followed their website and added these libraries

<dependency>
  <groupId>org.apache.sedona</groupId>
  <artifactId>sedona-spark-shaded-3.0_2.12</artifactId>
  <version>1.4.0</version>
</dependency>
<dependency>
  <groupId>org.apache.sedona</groupId>
  <artifactId>sedona-viz-3.0_2.12</artifactId>
  <version>1.4.0</version>
</dependency>
<!-- Optional: https://mvnrepository.com/artifact/org.datasyslab/geotools-wrapper -->
<dependency>
    <groupId>org.datasyslab</groupId>
    <artifactId>geotools-wrapper</artifactId>
    <version>1.4.0-28.2</version>
</dependency>

I have spark version 3.4 and scala version 2.12

What else is needed to execute these ST_ functions?

0

There are 0 best solutions below