Can I write mapper and reducer program in different language

572 Views Asked by At

I felt doing my Mapper operation in Perl script but then I realized it would be easier to write Reducer in Python. Can Mapper and Reducer can work in different programming language?

1

There are 1 best solutions below

1
On

From the tags it sounds like you're using Amazon's Elastic Map Reduce service. This question is answered in their excellent FAQ:

Q: What programming languages does Amazon Elastic MapReduce support?

You can use Java to implement Hadoop custom jars. Alternatively, you may use other languages including Perl, Python, Ruby, C++, PHP, and R via Hadoop Streaming. Please refer to the Developer’s Guide for instructions on using Hadoop Streaming.

In other words, anything that can write to a file can be used in an EMR map or reduce step.