Scan a file using AWS EMR

59 Views Asked by At

I have a file which stores more than 1 million entries in the form of (a,b).My system has to scan through the file and find out 'b' for given 'a' where people suggested me to use AWS EMR since we deal with AWS resources and services. Can someone help me on this since i do not have much idea about EMR? I have gone through the basic stuffs about EMR. Does EMR solves my problem?

1

There are 1 best solutions below

0
On

I'm not really sure how is your algorithm is working and how big are your data. It hard to say, what's the minimum amount of data from where it makes sense to use MapReduce technologies. If your algorithm is already written in any EMR compatible languages and you have a lots of data, then EMR can be a good choice.

But I would also check if DynamoDB is a good solution.