How to combine two JavapairRDD to a custom JavapairRDD?

133 Views Asked by At

I have created the following JavaPairRdds from the data received from different API endpoints.

listHeaderRDD<Integer, TeachersList> -> {list_id, list_details} 
e.g {1,{list_id:1,name:"abc",quantity:"2"}},  
    {2,{list_id:2,name:"xyz",quantity:"5"}}...

    ItemsGroupListRDD<Integer, Iterable<Tuple2<Integer, TeachersListItem>>> -> 
{list_id, {{item_id1,item_details1},{item_id2,item_details2}..}}

    e.g {1, {{11,{item_id:11,item_name:"abc"}},{12,{item_id:12,item_name:"acv"}}}..}
        {2, {{14,{item_id:14,item_name:"bnh"}},{18,{item_id:18,item_name:"hjk"}}}..}

Desired output:

teachersListRDD<TeachersList, Iterable<TeachersListItem>> -> {list_details, all_item_details}

e.g {{{list_id:1,name:"abc",quantity:"2"},{{item_id:11,item_name:"abc"},{item_id:12,item_name:"acv"}}},

{{list_id:2,name:"xyz",quantity:"5"},{{item_id:14,item_name:"bnh"},{item_id:18,item_name:"hjk"}}}
}

Basically I want the value of first RDD to be the key in the desired RDD and the group of item_details from the second RDD corresponding to that list_id as the value for the desired RDD i.e teachersListRDD I have tried different ways to do it but unable to get the desired output.

0

There are 0 best solutions below