how to recursively traverse through given HDFS directory and fetch the filename with path in pyspark

671 Views Asked by At

Structure of my directory in hdfs looks like below:

----Directory

 |----sub folder 1
          |--------file 1
          |---------file 2
 |                          
 |----sub folder 2
            |--------file 1
            |---------file 2
 |----sub folder 3
            |--------file 1
            |---------file 2
 |----file 1

i want to traverse recursively through the HDFS directory to retrieve all filename and its path.

0

There are 0 best solutions below