How to read subdirectory data into bigsql table?

Question

Bigsql is not able to read data from subdirectory like hive can by setting parameters.

set tblproperties (
    "hive.input.dir.recursive" = "TRUE",
    "hive.mapred.supports.subdirectories" = "TRUE",
    "hive.supports.subdirectories" = "TRUE",
    "mapred.input.dir.recursive" = "TRUE")

I tried adding above parameters into bigsql tableproperties but it's not able to read subdirectory data.

What parameters I need to set in bigsql to read subdirectory data?

Answer 1

As per my understanding, Bigsql only looks up for files in parent directory. It doesn't even bother checking sub-directories. It will display an empty table because it cannot read the data recursively.This feature is still under product improvement ideas with IBM product and engineering team.