Even I have a similar requirement. I want to read avro records having some schema and emit text datatype key values, I need to write the MR Unit test cases for this. I have wrote the following code but it is giving me the following exception:
org.apache.avro.AvroTypeException: Found string, expecting xyz.abc.Myschema
at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:231)
at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
.....
.....
Following is my codebase in setup function:
MyMapper myMapper = new MyMapper();
mapDriver = new MapDriver<AvroKey<Myschema>, NullWritable, Text, Text>();
mapDriver.setMapper(myMapper);
Configuration configuration = mapDriver.getConfiguration();
//Copy over the default io.serializations. If you don't do this then you will
//not be able to deserialize the inputs to the mapper
String[] strings = mapDriver.getConfiguration().getStrings("io.serializations");
String[] newStrings = new String[strings.length +1];
System.arraycopy( strings, 0, newStrings, 0, strings.length );
newStrings[newStrings.length-1] = AvroSerialization.class.getName();
//Now you have to configure AvroSerialization by sepecifying the key
//writer Schema and the value writer schema.
configuration.setStrings("io.serializations", newStrings);
Text x = new Text();
Configuration conf = mapDriver.getConfiguration();
AvroSerialization.addToConfiguration(conf);
AvroSerialization.setKeyWriterSchema(conf, Schema.create(Schema.Type.STRING));
AvroSerialization.setKeyReaderSchema(conf, new Myschema().getSchema());
Job job = new Job(conf);
job.setMapperClass(MyMapper.class);
job.setInputFormatClass(AvroKeyInputFormat.class);
AvroJob.setInputKeySchema(job, new Myschema().getSchema());
job.setOutputKeyClass((new Text()).getClass());
I need to read avro based records having Myschema schema and emit key values pairs having text datatypes. Following is my mapper class:
public class MyMapper extends Mapper<AvroKey<Myschema>, NullWritable, Text, Text>...
protected void map(AvroKey<Myschema> key, NullWritable value, Context context)...
Can someone check if there is any configuration parameter that i'm missing and help me out?