Write to external array inside a running powershell job

Question

I am trying to write data to external array while running a powershell job-

This is my code sample that I' m trying-

$datafromJob = @()
$wmijobs = @()
foreach ($c in $computers) {
    $wmijobs += Start-Job -Name WMIInventory -ScriptBlock {
        $jobdata = get-wmiobject -ComputerName $args[0] -Class win32_computersystem -Credential $Cred -ErrorVariable Err -ErrorAction SilentlyContinue
        if ($Err.length) {
            Add-content -Path D:\InventoryError.log -Force -Value $Err
            $details = @{
                Domain       = "Error"
                Manufacturer = "Error"
                Computer     = $args[0]
                Name         = "Error"
            }
            $args[3] += New-Object PSObject -Property $details
        }
        if ($jobdata.length) {
            $details = @{
                Domain       = $jobdata.Domain
                Manufacturer = $jobdata.Manufacturer
                Computer     = $args[2]
                Name         = $jobdata.Name
            }
            $args[3] += New-Object PSObject -Property $details
        }
        -ArgumentList $c, $Cred, "Test", $datafromJob
    }
}

Expecting Output in $datafromJob Variable, but the end of job and loop variable is empty, M not getting how it will work, anyhelp,

Do let me know if any queries on this question

Answer 1

Background jobs run in a separate (child) process, so you fundamentally cannot directly update values in the caller's scope from them.^[1]

Instead, make your job script block produce output that the caller can capture with Receive-Job.

A simple example:

# Create a 10-element array in a background job and output it.
# Receive-Job collects the output.
$arrayFromJob = Start-Job { 1..10 } | Receive-Job -Wait -AutoRemoveJob

Note: If what you output from a background job are complex objects, they will typically not retain their original type and instead be custom-object emulations, due to the limitations of PowerShell's XML-based cross-process serialization infrastructure; only a limited set of well-known types deserialize with type fidelity, including primitive .NET types, hashtables and [pscustomobject] instances (with the type-fidelity limitations again applying to their properties and entries). - see this answer for background information.

---

A few asides:

• There is no need to call Start-Job / Get-WmiObject in a loop, because the latter's -ComputerName parameter can accept an array of target computers to connect to in a single call.

  • Since the target computers are then queried in parallel, you may not need a background job (Start-Job) at all.

• The CIM cmdlets (e.g.,Get-CimInstance) superseded the WMI cmdlets (e.g., Get-WmiObject) in PowerShell v3 (released in September 2012). Therefore, the WMI cmdlets should be avoided, not least because PowerShell [Core] (version 6 and above), where all future effort will go, doesn't even have them anymore.

  • Remote use of the CIM cmdlets by default requires that the target computers be set up for WS-Management connections, as they implicitly are if PowerShell remoting is enabled on them - see about_Remote_Requirements; alternatively, however, you can use the DCOM protocol (which is what the WMI cmdlets used) - see this answer for more information.

Applying the above to your case:

# Create a CIM session that targets all computers.
# By default, the WS-Management protocol is used, which target computers
# are implicitly set up for if PowerShell remoting is enabled on them.
# However, you can opt to use DCOM - as the WMI cmdlets did - as follows:
#   -SessionOption (New-CimSessionOption -Protocol DCOM)
$session = New-CimSession -ComputerName $computers -Credential $Cred

# Get the CIM data from all target computers in parallel.
[array] $cimData = Get-CimInstance -CimSession $session -Class win32_computersystem -ErrorVariable Err -ErrorAction SilentlyContinue |
  ForEach-Object {
    [pscustomobject] @{
      Domain       = $_.Domain
      Manufacturer = $_.Manufacturer
      Computer     = $_.ComputerName
      Name         = $_.Name
    }
  }

# Cleanup: Remove the session.
Remove-CimSession $session

# Add error information, if any.
if ($Err) {
  Set-Content D:\InventoryError.log -Force -Value $Err
  $cimData += $Err | ForEach-Object {
    [pscustomobject] @{
      Domain       = "Error"
      Manufacturer = "Error"
      Computer     = $_.ComputerName
      Name         = "Error"
    }
  }
}

Caveat re targeting a large number of computers at once:

• As of this writing, neither the Get-CimInstance help topic nor the conceptual about_CimSession topic discuss connection throttling (limiting the number of concurrent connections to remote computers to prevent overwhelming the system).

• PowerShell's general-purpose Invoke-Command remoting command, by contrast, has a -ThrottleLimit parameter that defaults to 32. Note that PowerShell remoting must first be enabled on the target computers in order to be able to use Invoke-Command on them remotely - see about_Remote_Requirements.

Therefore, to have more control over how the computers are targeted in parallel, consider combining Invoke-Command with local invocation of Get-CimInstance on each remote machine; for instance:

Invoke-Command -ComputerName $computers -ThrottleLimit 16 {
    Get-CimInstance win32_computersystem  
}  -Credential $Cred -ErrorVariable Err -ErrorAction

Also passing a sessions-options object to Invoke-Command's -SessionOption parameter, created with New-PSSessionOption, additionally gives you control over various timeouts.

---

^{[1] In a script block executed in a background job, the automatic $args variable contains deserialized copies of the values passed by the caller - see this answer for background information.
Note that the usually preferable, thread-based Start-ThreadJob cmdlet - see this answer - can receive live references to reference-type instances in the caller's scope, though modifying such objects then requires explicit synchronization, if multiple thread jobs access them in parallel; the same applies to the PowerShell 7+ ForEach-Object -Parallel feature.}

Answer 2

As per @mklement0 suggestion, have updated my script,

$wmijobs = @()
foreach ($c in $computers) {
    $wmijobs += Start-Job -Name WMIInventory -ScriptBlock {
        $jobdata = get-CimInstance -ComputerName $args[0] -Class win32_computersystem -Credential $Cred -ErrorVariable Err -ErrorAction SilentlyContinue
        if ($Err.length) {
            Add-content -Path D:\InventoryError.log -Force -Value $Err
            $details = @{
                Domain       = "Error"
                Manufacturer = "Error"
                Computer     = $args[0]
                Name         = "Error"
            }
          return New-Object PSObject -Property $details
        }
        if ($jobdata) {
            $details = @{
                Domain       = $jobdata.Domain
                Manufacturer = $jobdata.Manufacturer
                Computer     = $args[2]
                Name         = $jobdata.Name
            }
            return New-Object PSObject -Property $details
        }
        -ArgumentList $c, $Cred, "Test", $datafromJob
    }
}
$wmijobs = $wmijobs | get-job | receive-job -AutoRemoveJob -wait

Answer 3

It is technically possible with threadjobs, but it probably wouldn't work well in your case. Arrays aren't thread safe.

$a = 1,2,3
start-threadjob { $b = $using:a; $b[0] = 2 } | receive-job -wait -auto
$a

2
2
3

Hmm, thread safe updating of a dictionary collection from the bottom here: https://devblogs.microsoft.com/powershell/powershell-foreach-object-parallel-feature/

$threadSafeDictionary =
[System.Collections.Concurrent.ConcurrentDictionary[string,object]]::new()

Get-Process | ForEach-Object -Parallel {
    $dict = $using:threadSafeDictionary
    $dict.TryAdd($_.ProcessName, $_)
}

$threadSafeDictionary["pwsh"]

"Concurrent bag":

$threadSafeArray =
[System.Collections.Concurrent.ConcurrentBag[object]]::new()

1..10 | foreach-object -parallel { 
    $array = $using:threadSafeArray
    $array.Add($_)
} 

$threadSafeArray

10
9
8
7
6
5
4
3
2
1