Hive Runtime Error: Unable to deserialize reduce input key

Question

I am trying to run a Insert in to partition table with group by involved

query

'set hive.exec.dynamic.partition=true;
 set hive.exec.dynamic.partition.mode=nonstrict;
 set hive.execution.engine=tez;
 INSERT OVERWRITE TABLE table1 PARTITION (date) select col1,CONCAT(COALESCE(substr(Cdate,1,4),'-'),'',COALESCE(substr(Cdate,6,2),'-'),'',COALESCE(substr(Cdate,9,2),'-')),col3,col4,'mobile-data',data,date 
from
(select col1,substr(CDate,1,10) as Cdate,u.col3 as col3,u.col4 as col4,date,sum(u.col5+u.col6) as data from json_table lateral view explode(json_table.mobiledata) oc as u 
where date >= from_unixtime(unix_timestamp()-4*60*60*24, 'yyyyMMdd')
group by col1,substr(CDate,1,10),u.col3,u.col4,date)t;'

error:

'Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error: Unable to deserialize reduce input key from x1x50x48x49x53x48x56x49x57x0x1x117x115x101x114x49x0x1x129x51x122x35x1x65x78x84x32x72x65x76x32x83x101x114x118x105x99x101x0x1x99x111x109x46x100x115x105x46x97x110x116x46x115x101x114x118x101x114x0x1x109x111x98x105x108x101x45x100x97x116x97x0x255 with properties{columns=reducesinkkey0,reducesinkkey1,reducesinkkey2,reducesinkkey3,reducesinkkey4,reducesinkkey5, serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe, serialization.sort.order=++++++, columns.types=string,string,string,string,string,string}'

I have never seen this error. Does anyone know how to take care of this?

This query is working fine if i just use the select part of the query excluding the insert into partitioned table.

Answer 1

I was able to get this resolved using help from this link

http://documentation.altiscale.com/hive-runtime-error-unable-to-deserialize-input-key

All i needed was a setting change.

'Set hive.optimize.sort.dynamic.partition = false;'

Once i add this setting, query(insert into partitioned table) is working fine.