I'm trying to run a query that filters based on json attributes against a Geomesa HBase cluster with remote filtering enabled (hbase.remote.filtering = true). However, the query is failing due to an IllegalArgumentException with error message Property '$.featurePayload.properties.user' does not exist in feature type. Is there a way to enable remote filtering based on json attributes with Geomsa and HBase?
Note: I was able to run this query successfully when hbase.remote.filtering was set to false and the filtering was done on the client side where the application connects to a GeoTools DataStore to get the features by query. My main confusion is what is different on the remote filtering that is throwing the exception and if there is anyway to get json attribute filtering remotely working.
This is an example query I am trying to run:
\"$.featurePayload.properties.user\" = 'someUser'
This is the table schema:
INFO Describing attributes of feature 'OSMNodes'
geometry | Point (Spatio-temporally indexed)
ingestionTimestamp | Timestamp (Spatio-temporally indexed)
nextTimestamp | Timestamp
serializerVersion | String
featurePayload | String
Example feature in the table:
id,*geometry:Point:srid=4326,ingestionTimestamp:Timestamp,nextTimestamp:Timestamp,serializerVersion:String,featurePayload:String
1789928211#3165330648,POINT (-0.1844331 27.2077326),2019-12-23T03:00:00.000Z,9999-12-30T23:59:59.000Z,geojson,"{""geometry"":{""type"":""Point"",""coordinates"":[-0.1844331,27.2077326]},""id"":""3165330648"",""properties"":{""tags"":{},""changesetId"":123456,""version"":1,""uid"":1111111,""user"":""someUser"",""featureSource"":""OSM"",""sourceTimestamp"":""2014-11-03 19:40:40"",""ingestionTimestamp"":""2019-12-23 03:00:00""}}"
Stack trace:
1. Caused by: java.lang.IllegalArgumentException: Property '$.featurePayload.properties.user' does not exist in feature type
2. at org.locationtech.geomesa.filter.factory.FastFilterFactory$$anonfun$1.apply(FastFilterFactory.scala:263)
3. at org.locationtech.geomesa.filter.factory.FastFilterFactory$$anonfun$1.apply(FastFilterFactory.scala:263)
4. at scala.Option.getOrElse(Option.scala:121)
5. at org.locationtech.geomesa.filter.factory.FastFilterFactory.property(FastFilterFactory.scala:262)
6. at org.locationtech.geomesa.filter.visitor.QueryPlanFilterVisitor.visit(QueryPlanFilterVisitor.scala:139)
7. at org.geotools.filter.AttributeExpressionImpl.accept(AttributeExpressionImpl.java:347)
8. at org.geotools.filter.visitor.DuplicatingFilterVisitor.visit(DuplicatingFilterVisitor.java:118)
9. at org.locationtech.geomesa.filter.visitor.QueryPlanFilterVisitor.visit(QueryPlanFilterVisitor.scala:318)
10. at org.locationtech.geomesa.filter.visitor.QueryPlanFilterVisitor.org$locationtech$geomesa$filter$visitor$QueryPlanFilterVisitor$$bind(QueryPlanFilterVisitor.scala:351)
11. at org.locationtech.geomesa.filter.visitor.QueryPlanFilterVisitor.visit(QueryPlanFilterVisitor.scala:150)
12. at org.geotools.filter.IsEqualsToImpl.accept(IsEqualsToImpl.java:149)
13. at org.locationtech.geomesa.filter.visitor.QueryPlanFilterVisitor.visit(QueryPlanFilterVisitor.scala:81)
14. at org.geotools.filter.AndImpl.accept(AndImpl.java:45)
15. at org.locationtech.geomesa.filter.factory.FastFilterFactory$.optimize(FastFilterFactory.scala:412)
16. at org.locationtech.geomesa.filter.factory.FastFilterFactory$.toFilter(FastFilterFactory.scala:401)
17. at org.locationtech.geomesa.index.iterators.IteratorCache$$anonfun$filter$2.apply(IteratorCache.scala:82)
18. at org.locationtech.geomesa.index.iterators.IteratorCache$$anonfun$filter$2.apply(IteratorCache.scala:82)
19. at org.locationtech.geomesa.utils.cache.ThreadLocalCache.getOrElseUpdate(ThreadLocalCache.scala:68)
20. at org.locationtech.geomesa.index.iterators.IteratorCache$.filter(IteratorCache.scala:82)
21. at org.locationtech.geomesa.hbase.filters.CqlTransformFilter$.deserialize(CqlTransformFilter.scala:286)
22. ... 19 more