How to push data from HAWQ into GREENPLUM?

Question

I have this erratic client who wants to push data from HAWQ to GREENPLUM after some pre processing. Is there any way to do this? If not, Is it possible to create an external table in greenplum that reads it from the HDFS in which HAWQ is running?

Any help will be appreciated.

Answer 1

The simplest you can do - push the data from HAWQ to HDFS using external writable table and then read it from Greenplum using external readable table using gphdfs protocol. In my opinion this would be the fastest option.

Another option would be to store the data in gzipped CSV files on HDFS and work with them directly from HAWQ. This way when you need this data in Greenplum you can just query it in the same way, as an external table

Answer 2

HAWQ is same as Greenplum, only underlying storage is hdfs,

One way is You can create a externale(writable) table in HAWQ which will write your data into a file, now after this you can create a external(readable) table in Greenplum which will read data from that created file

Another way You can copy from one server to another using Standard Input/Output, I use it many times when required to puch data from development environment to Prodcution or vice-versa

Another way You can table a backup using pg_dump/gp_dump for particular table/tables then restore using pg_restore/gp_restore

Thanks