Geomesa Pyspark AnalysisException: Undefined function: st_transform

Question

I am trying to get the area of a bunch of polygons. I am able to use st_area and st_geomFromText, but I get an undefined function error when trying to use st_transform. I need to transform from 4326 to 3857 (or whatever will give me acres).

geomesa version: I had 2.4.2, but now I have 3.4.

error:

AnalysisException: Undefined function: st_transform. This function is neither a built-in/temporary function, nor a persistent function that is qualified as spark_catalog.default.st_transform.; line 4 pos 2

code:

%scala
import spark.implicits._
import org.apache.spark.sql.functions._
import org.locationtech.geomesa.spark.jts._
spark.withJTS
import org.locationtech.geomesa.spark.geotools._


%sql
select 
a.user_id
, Sum(st_area(st_geomFromText(a.polygon))) --this works
, Sum(st_transform(st_geomFromText(a.polygon), 'EPSG:4326','EPSG:3857')) --this does not work
from core_table a
group by a.user_id

I've tried changing the fromCRS and toCRS strings. I've tried using lowercase, different numbers, with/without quotes and letters, etc. I've tried encapsulating the st_transform() in a SUM(), but nothing works.

I've also tried something similar to this.

Answer 1

The geometric functions are in a different package, because they require GeoTools and not just JTS. I believe you need to call org.apache.spark.sql.SQLTypes#init or equivalently org.locationtech.geomesa.spark.GeometricDistanceFunctions#registerFunctions to make them available.