List["a", "b", "c"]) My data on elastic has a field called "names". I want to query elastic search " /> List["a", "b", "c"]) My data on elastic has a field called "names". I want to query elastic search " /> List["a", "b", "c"]) My data on elastic has a field called "names". I want to query elastic search "/>

Spark-elasticsearch fetch filtered records from elasticsearch using spark

305 Views Asked by At

I have a map which is as follows :

 Map("index1" -> List["a", "b", "c"])

My data on elastic has a field called "names". I want to query elastic search from spark and return all records which has "a", "b", "c" as the value of the "name" field.

I dont want to hardcode the names in the elastic query.

 session.read.format("org.elasticsearch.spark.sql")
  .option("es.query", query)
  .load(indexName)

Can someone please help me with framing a dynamic query for the same which accepts a list of strings.

1

There are 1 best solutions below

0
mythic On
 val listOfIds = List("a", "b")
 val strings = listOfIds.mkString("[\"", "\",\"", "\"]")
 val query = s"""{"query":{"filtered":{"query":{"match_all":{}},"filter": 
 {"terms":{"genres":${strings}}}}}"""

Used to this and solved it.