How to load data from oracle and sql server to HAWQ using Spring XD

Question

Hi I have tables in Oracle and SQL Server. I need to load data from oracle and sql server into Pivotal HAWQ using Spring XD. Couldn't find in documentation.

Answer 1

You need to integrate sqoop jobs with Spring XD. See link below for sqoop jobs with springxd

https://github.com/tzolov/spring-xd-sqoop-job

Answer 2

You can use jdbchdfs job to load the data in HDFS as CSV or any PXF supported format. Then you can map the loaded data to HAWQ tables using PXF External tables support. If you need to load this data to Native HAWQ tables then you can do a SELECT INSERT from there, or have SELECT INSERT configured as another batch job that loads the data from PXF External table to HAWQ native.

Answer 3

Outsourcer is another open source solution that was originally designed to load data from Oracle and SQL Server into Greenplum but was enhanced some time ago to also support HAWQ.

All of the documentation and downloads are on http://www.pivotalguru.com/

And if you are interested in seeing the source code, here it is: https://github.com/pivotalguru/outsourcer