Dir's Move to Flume

Question

I would like to load a DIR that contains sub dir and a list of files using flume API or java API into HDFS under /flume/events. I am using cloudera cdh5.1

Please could you help me in this regard. I would like to go for coding of how to ingest/load dir that contains sub dir's and files. I have an idea about files loading into hdfs using flume api/java api

Thanks in Advance

Answer 1

Is there a reason you're using Flume or Java? HDFS includes a utility to copy from a local file system to HDFS:

$ hdfs dfs -put <localsrc> ... <dst>

The switches -copyFromLocal does the same thing; using -moveFromLocal will delete the local file after it is copied.

Use

$ hdfs dfs -help

for more information.