Microsoft Windows: is there a way to limit the number of commands running in parallel?

Question

I have a list of about 500 commands defined on a command-list.txt file. My goal is to run them all as much as possible in parallel (and not simply sequentially).

I'm working on a PowerShell script to:

1. Take the list of commands as input
2. Run the first 8 commands in parallel
3. As soon as one of those 8 commands would complete the 9th would start till to the last one.

$files = Get-ChildItem -Path D:\users -Recurse | Select-Object -ExpandProperty FullName
ForEach-Object $files {
    Start-ThreadJob {
        C:\stefano\PostProcess\analyze.exe $_
    } -ThrottleLimit 8
}

But I get this error:

ForEach-Object : Cannot bind parameter 'RemainingScripts'. Cannot convert the "-Parallel" value of type "System.String" to type "System.Management.Automation.ScriptBlock".

The goal is to avoid the possibility to run 500 commands in parallel, to limit the system resources as long as to optimize the number of thread that can be handled in parallel.

Can anyone help me?

Answer 1

The syntax for processing the analyze task in parallel would look like this using ThreadJob:

Get-ChildItem -Path D:\users -Recurse | ForEach-Object {
    Start-ThreadJob {
        C:\stefano\PostProcess\analyze.exe ($using:_).FullName
    } -ThrottleLimit 8
} | Receive-Job -Wait -AutoRemoveJob

If you have access to PowerShell 7+ then it becomes even simpler with ForEach-Object -Parallel:

Get-ChildItem -Path D:\users -Recurse | ForEach-Object -Parallel {
    C:\stefano\PostProcess\analyze.exe $_.FullName
} -ThrottleLimit 8

Worth noting, Get-ChildItem has a -File switch if you need to target only files.