Unable to Import table from RDMS from Hive

155 Views Asked by At

I am Trying to Import table from RDMS to Hive but It is giving error. Sqoop eval command working fine, able fetch records.

sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=false" \
> --connect "jdbc:tibero:thin:@hostname:8629:DBI" \
> --driver com.tmax.tibero.jdbc.TbDriver \
> --username XXX --password XXXX \
> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 15:16:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 15:16:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 15:16:38 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
21/09/27 15:16:38 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
21/09/27 15:16:38 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
21/09/27 15:16:38 INFO manager.SqlManager: Using default fetchSize of 1000
21/09/27 15:16:38 INFO tool.CodeGenTool: Beginning code generation
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.0.1.0-187/hadoop-mapreduce
21/09/27 15:16:41 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/8fd46edea8696bb5156e637113626b10/DMSDBA.cmm_cadorg_Tb.jar
21/09/27 15:16:41 ERROR tool.ImportTool: Import failed: No primary key could be found for table DMSDBA.cmm_cadorg_Tb. Please specify one with --split-by or perform a sequential import with '-m 1'.

I also tried to pass --split-by

> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1\
> --split-by DORG_UPDT_EMP_NO
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 13:41:10 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 13:41:10 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 13:41:10 ERROR tool.BaseSqoopTool: Error parsing arguments for import:
21/09/27 13:41:10 ERROR tool.BaseSqoopTool: Unrecognized argument: DORG_UPDT_EMP_NO

Also tried to pass MapReduce argument but again getting error

> --table DMSDBA.cmm_cadorg_Tb \
> --hive-import \
> --create-hive-table \
> --hive-table DMSDBA.cmm_cadorg_Tb1\
> -m 4
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/hdp/3.0.1.0-187/hive/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
21/09/27 15:16:38 INFO sqoop.Sqoop: Running Sqoop version: 1.4.8.3.0.1.0-187
21/09/27 15:16:38 WARN tool.BaseSqoopTool: Setting your password on the command-line is insecure. Consider using -P instead.
21/09/27 15:16:38 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
21/09/27 15:16:38 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
21/09/27 15:16:38 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
21/09/27 15:16:38 INFO manager.SqlManager: Using default fetchSize of 1000
21/09/27 15:16:38 INFO tool.CodeGenTool: Beginning code generation
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM DMSDBA.cmm_cadorg_Tb AS t WHERE 1=0
21/09/27 15:16:39 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /usr/hdp/3.0.1.0-187/hadoop-mapreduce
21/09/27 15:16:41 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-root/compile/8fd46edea8696bb5156e637113626b10/DMSDBA.cmm_cadorg_Tb.jar
21/09/27 15:16:41 ERROR tool.ImportTool: Import failed: No primary key could be found for table DMSDBA.cmm_cadorg_Tb. Please specify one with --split-by or perform a sequential import with '-m 1'.

Please help me on this. Thank You So much in Advance. please Ignore formatting if there is any issue in format.

1

There are 1 best solutions below

0
PR Kumar On

Try passing target directory.

sqoop import "-Dorg.apache.sqoop.splitter.allow_text_splitter=true" \
 --connect "jdbc:tibero:thin:@hostname:8629:DBI" \
 --driver com.tmax.tibero.jdbc.TbDriver \
 --target-dir /project/whatever \
 --username XXX --password XXXX \
 --table DMSDBA.cmm_cadorg_Tb \
 --hive-import \
 --create-hive-table \
 --hive-table DMSDBA.cmm_cadorg_Tb1