Iterate through a folder just once JAVA Files.walk

Question

I am stuck regarding reading only the directories. I want to iterate a folder and see all the subdirectories that the folder contains. If the subdirectory contains txt files, then send it to the "DistributionFolder" class which will read that subdirectory. Now is reading a subdirectory, for instance:

Can read the content of "check", which is a subdirectory of the "Alex" directory and both subdirectories of "check", but unfortunately, if I will add another directory, for instance:

The content of "1111" directory will be read it twice.

This is the code that splits the directory and subdirectories:

 private boolean readingDirectory(File files, WriteDatasetInfo writeDatasetInfo) throws IOException, InterruptedException {



        List<Future> futures = new ArrayList<>();

        // is getting the number of available processes
        int nrThreads = Runtime.getRuntime().availableProcessors();

        // Creating a ExecutorService pool with the number of available processes
        ExecutorService pool = Executors.newFixedThreadPool(nrThreads);

        //Traverse the dataset using File walker

        try (Stream<Path> paths = Files.walk(Paths.get(String.valueOf(files)))){
            //filter the file to be a regular file
            paths.filter(Files ::isDirectory)
                    .filter(mainReader::isNotTheDefault)
            // for each file submit the work to the ExecutorService pool to be executed and add the videoNumber
            .forEach(
                path -> {
                    DistributionFolder distributionFolder = new DistributionFolder(this.directory,this.videoNumber);
                    try {
                        System.out.println(path);
                        this.videoNumber=distributionFolder.readingDirectory(path.toFile(),writeDatasetInfo);
                    } catch (IOException e) {
                        throw new RuntimeException(e);
                    } catch (InterruptedException e) {
                        throw new RuntimeException(e);
                    }
                });

        }catch (Exception e){
            throw new RuntimeException(e);
        }
        finally {
            pool.shutdown();
            while (!pool.awaitTermination(24L, TimeUnit.HOURS)) {
                System.out.println("Not yet. Still waiting for termination");
            }
            Regression.FullRegression(this.directory);
            return true;
        }
    }

    private static boolean isNotTheDefault(Path path){
        if(!defaultFile.equals(path.toFile())){
            return true;
        }
        return false;
    }

This is the "DistributionFolder" class

protected final String directory;
protected int videoNumber;


public DistributionFolder(String directory,int videoNumber){
    this.directory=directory;
    this.videoNumber=videoNumber;
}


protected int readingDirectory(File files, WriteDatasetInfo writeDatasetInfo) throws IOException, InterruptedException {

    List<Future> futures = new ArrayList<>();

    // is getting the number of available processes
    int nrThreads = Runtime.getRuntime().availableProcessors();

    // Creating a ExecutorService pool with the number of available processes
    ExecutorService pool = Executors.newFixedThreadPool(nrThreads);

    //Traverse the dataset using File walker
    try (Stream<Path> paths = Files.walk(Paths.get(String.valueOf(files)))){
        //filter the file to be a regular file
        paths.filter(Files::isRegularFile)
                // apply the custom filter in order to read only txt files
                .filter(DistributionFolder::isTXT)
                // for each file submit the work to the ExecutorService pool to be executed and add the videoNumber
                .forEach(
                        path -> {

                            String[] strPath = String.valueOf(path).split("/");

                            pool.submit(new Reader(String.valueOf(path), videoNumber, this.directory, writeDatasetInfo,strPath[strPath.length-2]));
                            this.videoNumber++;

                        });

    }catch (Exception e){
        throw new RuntimeException(e);
    }
    finally {
        // wait fo the execution of the pool to be done
        pool.shutdown();
        while (!pool.awaitTermination(24L, TimeUnit.HOURS)) {
            System.out.println("Not yet. Still waiting for termination");
        }
        String[] splitFile = files.toString().split("/");

        MaintainedForRegression.SendMapForRegression(splitFile[splitFile.length-1],this.directory);
        return this.videoNumber;
    }
}

private static boolean isTXT(Path path) {
    String pathFile = String.valueOf(path);
    String[] slittedpath = pathFile.split("/");
    String[] splittedname = slittedpath[slittedpath.length - 1].split("\\.");
    return splittedname[splittedname.length - 1].equals("txt");
}

This is the output when a new directory is inside of the first subdirectory of the check directory:

This is an example that works perfectly fine, when the is no other subdirectory:

The output:

I edited the question to add more context.

My theory is: when I pass the path of the directory to the "DistributionFolder" will go to the subdirectories as well

If you have any suggestion please let me know. Thanks!

Answer 1

In order limit the maximum depth that the walk is going I used the "Files.walk" with all 3 parameters. I used in the "DistributionFolder" class for that try and catch block:

 try (Stream<Path> paths = Files.walk(files.toPath(),1, FileVisitOption.FOLLOW_LINKS)){