I am trying to create a meterialized view for Druid storage handler from Hive.
Steps:
CREATE TABLE employee_trans2 (
`__time` TIMESTAMP,
id int,
name string,
age int,
gender string)
STORED AS ORC
TBLPROPERTIES ('transactional'='true',"orc.compress"="NONE");
Insert data
insert into employee_trans2 values(CURRENT_TIMESTAMP(),10,'basa',30,'M');
Select data
select `__time`,id,name,age,gender from default.employee_trans1;
+--------------------------+-----+--------+------+---------+
| __time | id | name | age | gender |
+--------------------------+-----+--------+------+---------+
| 2023-10-30 22:43:35.534 | 10 | basa | 30 | M |
| 2023-10-30 22:44:29.56 | 10 | basa | 30 | M |
| 2023-10-30 22:44:29.56 | 20 | basa2 | 20 | M |
| 2023-10-30 22:44:29.56 | 30 | basa3 | 33 | M |
+--------------------------+-----+--------+------+---------+
Create meterialize table
CREATE MATERIALIZED VIEW mat_emp_hive_druid STORED BY 'org.apache.hadoop.hive.druid.DruidStorageHandler' AS select `__time`, id,name,age,gender from default.employee_trans1;
Error
INFO : Completed executing command(queryId=hive_20231030232047_32c5e387-f582-49ac-b4e9-6581997dff07); Time taken: 29.552 seconds
Error: Error while compiling statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Reducer 2, vertexId=vertex_1697781224687_0009_3_01, diagnostics=[Task failed, taskId=task_1697781224687_0009_3_01_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1697781224687_0009_3_01_000001_0:java.lang.RuntimeException: java.lang.RuntimeException: Hive Runtime Error while closing operators: java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
But I am able to see the jar from hive lib
root@odp01:/usr/odp/3.2.2.0-1/hive# grep -irn "net/jpountz/lz4/LZ4Factory" /usr/odp/3.2.2.0-1/
Binary file /usr/odp/3.2.2.0-1/hive/lib/lz4-java-1.7.1.jar matches
Binary file /usr/odp/3.2.2.0-1/oozie/share/lib/hive/lz4-java-1.7.1.jar matches
Binary file /usr/odp/3.2.2.0-1/oozie/share/lib/spark/lz4-java-1.4.0.jar matches
Binary file /usr/odp/3.2.2.0-1/oozie/embedded-oozie-server/webapp/WEB-INF/lib/lz4-java-1.7.1.jar matches
Binary file /usr/odp/3.2.2.0-1/druid/lib/lz4-java-1.8.0.jar matches
Binary file /usr/odp/3.2.2.0-1/druid/extensions/druid-protobuf-extensions/druid-protobuf-extensions-27.0.0.3.2.2.0-1.jar matches
Binary file /usr/odp/3.2.2.0-1/druid/extensions/druid-kafka-extraction-namespace/lz4-java-1.8.0.jar matches
Binary file /usr/odp/3.2.2.0-1/druid/extensions/druid-kafka-indexing-service/lz4-java-1.8.0.jar matches
Binary file /usr/odp/3.2.2.0-1/druid/extensions/druid-avro-extensions/lz4-java-1.8.0.jar matches
Binary file /usr/odp/3.2.2.0-1/kafka/libs/lz4-java-1.7.1.jar matches
Binary file /usr/odp/3.2.2.0-1/hadoop-mapreduce/lz4-java-1.7.1.jar matches
tried to create using NONE compression format, SNAPPY compression format, but still getting the error as
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
Any help would be appreciated.
tried to create using NONE compression format, SNAPPY compression format, but still getting the error as
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: net/jpountz/lz4/LZ4Factory
Any help would be appreciated.