mrunit - Using mrunit.mapreduce.MapDriver not callling customised Record Reader

29 Views Asked by At

I am modifying an MapReduce program in the Record Reader and wanted to write a test case for mapper to call customised InputFormat or Record Reader. I have modified test case for record reader but the record reader test cases are not an mrunit one.

How to call the customised Record Reader from Mapper, as withInputFormat function is not listed under MapDriver.newMapDriver?

Please find the snapshot of my code:

@Before
public void setUp(){
MapDriver<Object, Text, Text, Text> mapDriver = MapDriver.newMapDriver(new myMapper());
}

@Test
public void testFunction1() throws IOException {
mapDriver.withInput(...).withOutput(...).runTest();
}

Thanks

1

There are 1 best solutions below

1
Kenry Sanchez On

I was checking the MRUnit API, so. If you want to add a custom RecordReader i would guess you must have a custom InputFormat due to must assign the custom RecordReader into the createRecordReader method of the custom InputFormat class.

So, MRUnit API allows you to assign a custom InputFormat with also the custom OutputFormat.

public MapDriver<K1,V1,K2,V2> withOutputFormat(Class<? extends org.apache.hadoop.mapreduce.OutputFormat> outputFormatClass,
                                      Class<? extends org.apache.hadoop.mapreduce.InputFormat> inputFormatClass)

Configure Mapper to output with a real OutputFormat. Set InputFormat to read 
output back in for use with run* methods

Parameters:
outputFormatClass -
inputFormatClass -

Returns:
this for fluent style

Based on this, you can call mapDriver.withOutFormat(customOutputFormat.class, customInputFormat.class). With this way, you can use your RecordReader for testing.