How to intersec differents JavaPairRDD

Question

I have two different JavaPairRdd one with Key1,value and the second one with key2,value . What I try to achieve is merge them but get only the items with the same value.

I have tried the following:

 JavaPairRDD<String, String> finalRdd = filteredRdd.intersection(filteredsmallRdd);

Where filteredRdd contains key:Country , value and filteredsmallRdd contains: key:id , value . and I need which elements have the same value , with intersection i think that compare the only the key and I got an empty solution , any idea of how to do that?

Answer 1

You can use swap to change the value as the key for each of the rdd and then perform the intersection.

JavaPairRDD<String,String> finalRdd = filteredRdd.mapToPair(f -> f.swap()).intersection(filteredsmallRdd.mapToPair(f -> f.swap()));