How do I speed up this code for building a list of all files in a network directory?

Question

I have a WPF program that grabs all of the directories within a certain network directory and lists them in a listview.

The problems is that there is so many directories that it can take up to 5 seconds for the list to load.

I am wondering if there is a way to speed this up with more efficient code. Or maybe even a way to store the list in an array and just look for changes every time after the first?

For Each i As String In Directory.GetDirectories(dir)
    If File.Exists(Path.GetFullPath(i) & "\cover.jpg") Then
        If (File.GetAttributes(Path.GetFullPath(i)) And FileAttributes.Hidden) <> FileAttributes.Hidden Then
            Dim foldername As String = Path.GetFileName(i)
            Dim moviename As String = foldername
            If moviename.Length > 4 Then
                If moviename.Substring(0, 4) = "The " Then
                    moviename = moviename.Remove(0, 4)
                End If
            End If
            Dim display As Boolean = True
            If IO.Directory.GetDirectories(Path.GetFullPath(i)).Length < 1 Then
                If IO.Directory.GetFiles(Path.GetFullPath(i), "*.avi").Length < 1 Then
                    If IO.Directory.GetFiles(Path.GetFullPath(i), "*.mp4").Length < 1 Then
                        If IO.Directory.GetFiles(Path.GetFullPath(i), "*.mkv").Length < 1 Then
                            display = False
                        End If
                    End If
                End If
            End If
            If display = True Then
                Showslist.Items.Add(New With {Key .img = Path.GetFullPath(i) & "\cover.jpg", .name = foldername, .path = Path.GetFullPath(i), .created = Directory.GetCreationTime(Path.GetFullPath(i)), .moviename = moviename})
            End If
        End If
    End If
Next

Answer 1

1. Do not read the contents of a directory more than once. Instead, read all required content once and cache it in a (in-memory) variable. This will help performance because accessing memory is a lot faster than doing I/O (here: accessing the file system). For example:

If IO.Directory.GetFiles(…, "*.avi").Length < 1 Then
    If IO.Directory.GetFiles(…, "*.mp4").Length < 1 Then
        If IO.Directory.GetFiles(…, "*.mkv").Length < 1 Then
            …

You just queried the same directory three times. You could change this to only read the contents once (thus potentially speeding up your code up to three times), and then filter it in-memory:

'Imports System.IO
'Imports System.Linq

' access the file system only once and store the results in-memory…
Dim filePaths As String() = Directory.GetFiles(…, "*")

' … and perform everything else on that in-memory cache:
If Not filePaths.Any(Function(fp) Path.GetExtension(fp) = ".avi") Then
    If Not filePaths.Any(…) Then
        If Not filePaths.Any(…) Then
            …

_{P.S.: Perhaps it's worth pointing out that unlike Directory.GetFiles, the System.IO.Path methods should not cause expensive file system hits: They simply operate on strings that are known to contain file system paths.}

---

2. Consider reading the root directory's complete contents recursively. The Directory.GetFiles method has an overload Directory.GetFiles(String, String, SearchOption) whose SearchOption parameter can be set to SearchOption.AllDirectories. In that case, you will not only get the contents from the specified directory itself, but also from all of its sub-directories.

That means, you would need a single one call to Directory.GetFiles (for the root directory) instead of many calls, which means you're again reducing the number of expensive I/O calls. Store the results in an array and proceed to build your WPF list from there.

' read the contents of the network root directory, including all of its sub-directories…
Dim filePaths As String() = Directory.GetFiles(…, "*", SearchOption.AllDirectories)

' …and do everything else using the in-memory cache built above:
For Each filePath As String in filePaths
    …
    Showslist.Items.Add(…)
Next