Error handling problem with use Get-Filehash on Powershell when a file is open

Question

I created a command to find duplicate file on folders (with file length information) in Powershell that works fine but I have problem when a file that I analyze is open.

This is the command:

Get-ChildItem -Recurse |
select-object length, fullname, @{n="Hash";e={get-filehash -algorithm MD5 -path $_.FullName -ErrorAction SilentlyContinue | Select-object -expandproperty Hash}} | 
Group -Property Hash |
where {$_.Count -gt 1} |
foreach { $_.Group | select fullname, hash, length} |
Export-Csv -Path c:\temp\filelist.csv -Delimiter "`t" -NoTypeInformation

When the file is open I receive this error:

Group : Object reference not set to an instance of an object.
At line:2 char:164
+ ... tinue | Select-object -expandproperty Hash}} | Group -Property Hash |
+                                                    ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [Group-Object], NullReferenceException
    + FullyQualifiedErrorId : System.NullReferenceException,Microsoft.PowerShell.Commands.GroupObjectCommand

The problem is that Get-FileHash fails and the execution stop when a file is open.

I want only that the files that is open ignore and the command continue. I tried many solutions (-ErrorAction SilentlyContinue) without success. Please can you help me? Thank you.

Answer 1

Change your code using a loop instead of Select-Object and error handling to skip files where Get-FileHash fails to obtain the hash:

# Add `-File` to ensure the cmdlet outputs files only
Get-ChildItem -Recurse -File -ErrorAction SilentlyContinue |
    ForEach-Object {
        try {
            [pscustomobject]@{
                Length   = $_.Length
                Fullname = $_.FullName
                Hash     = ($_ | Get-FileHash -Algorithm MD5 -ErrorAction Stop).Hash
            }
        }
        catch {
            Write-Warning $_
        }
    } |
    Group-Object -Property Hash |
    Where-Object Count -GT 1 |
    ForEach-Object Group |
    Export-Csv -Path c:\temp\filelist.csv -Delimiter "`t" -NoTypeInformation

Another alternative that should work would be using the -PipelineVariable common parameter:

Get-ChildItem -Recurse -File -ErrorAction SilentlyContinue -PipelineVariable pv |
    # Change `-ErrorAction` to `Continue` if you want errors to be displayed
    Get-FileHash -Algorithm MD5 -ErrorAction SilentlyContinue |
    Select-Object *, @{ N='Length'; E={ $pv.Length }} -ExcludeProperty Algorithm |
    Group-Object Hash |
    Where-Object Count -GT 1 |
    ForEach-Object Group |
    Export-Csv ....

Answer 2

Preface:

• Santiago's helpful answer offers better reformulations of your command that bypass the original problem.

• The answer below explains the problem you faced - which is due to a Windows PowerShell bug - and offers an immediate fix that is required to make your command work.

---

Replace:

get-filehash -algorithm MD5 -path $_.FullName -ErrorAction SilentlyContinue | Select-object -expandproperty Hash

with:

(get-filehash -algorithm MD5 -path $_.FullName -ErrorAction SilentlyContinue).Hash

By using property access (.Hash), the expression result becomes $null if the Get-FileHash command fails and produces no output, whereas with Select-Object -ExpandProperty it becomes the null-like "enumerable null" aka "Automation Null" value (the special [System.Management.Automation.Internal.AutomationNull]::Value] singleton that represents the absence of command output).

It is the latter that causes a problem, due to a bug in Windows PowerShell. Read on for details.

---

You're seeing a bug in the Windows PowerShell implementation of Group-Object that has since been fixed in PowerShell (Core) 7+:

If the value of a grouping property among the input object happens to be the aforementioned "enumerable null", the error you saw occurs.

Here's a simplified example, using & {} (i.e. execution of an empty script block) to create the "enumerable null":

# !! In *Windows PowerShell only*, the following error occurs:
#    Group-Object : Object reference not set to an instance of an object.
[pscustomobject] @{ Foo = 42 }, [pscustomobject] @{ Foo = & {} } |
  Group-Object Foo