We are moving our DevOps
pipelines to a new cluster and while at it, we bumped into a weird behavior when calling kind
with PowerShell
. This applies to kubectl
also.
The below should be taken only as a repro, not a real world application. In other words, I'm not looking to fix the below code but I am searching for an explanation why the error happens:
curl.exe -Lo kind-windows-amd64.exe https://kind.sigs.k8s.io/dl/v0.10.0/kind-windows-amd64
Move-Item .\kind-windows-amd64.exe c:\temp\kind.exe -Force
$job = Start-Job -ScriptBlock { iex "$args" } -ArgumentList c:\temp\kind.exe, get, clusters
$job | Receive-Job -Wait -AutoRemoveJob
Now, if I directly execute the c:\temp\kind.exe get clusters
command in the PowerShell window, the error won't happen:
In other words, why does PowerShell (any version) consider the STDOUT
of kind/kubectl
as STDERR
? And how can I prevent this from happening?
There must be an environmental factor to it as the same exact code runs fine in one system while on another it throws an error...
tl;dr
kind
outputs its status messages to stderr, which in the context of PowerShell jobs surface via PowerShell's error output stream, which makes them print in red (and susceptible to$ErrorActionPreference = 'Stop'
and-ErrorAction Stop
).Either:
Silence stderr: Use
2>$null
as a general mechanism or, as David Kruk suggests, use a program-specific option to achieve the same effect, which in the case ofkind
is-q
(--quiet
)Re-route stderr output through PowerShell's success output stream, merged with stdout output, using
*>&1
.Also, if you want to know whether the external program reported failure or success, you need to include the value of the automatic
$LASTEXITCODE
variable, which contains the most recently executed external program's process exit code, in the job's output (the exit code is the only reliably success/failure indicator - not the presence or absence of stderr output).A simplified example with
*>&1
(for Windows; on Unix-like platforms, replacecmd
and/c
withsh
and-c
):As many utilities do,
kind
apparently reports status messages via stderr.Given that stdout is for data, it makes sense to use the only other available output stream, stderr, for anything that isn't data, so as to prevent pollution of the data output. The upshot is that stderr output doesn't necessarily indicate actual errors (success vs. failure should solely be inferred from an external program's process exit code).
PowerShell (for its own commands only) commendably has a more diverse system of output streams, documented in the conceptual about_Redirection help topic, allowing you to report status messages via
Write-Verbose
, for instance.PowerShell maps an external program's output streams to its own streams as follows:
Stdout output:
1
, analogous to how stdout can be referred to incmd.exe
and POSIX-compatible shells), allowing it to be captured in a variable ($output = ...
) or redirected to a file (> output.txt
) or sent through the pipeline to another command.Stderr output:
In local, foreground processing in a console (terminal), stderr is by default not mapped at all, and is passed through to the display (not colored in red) - unless a
2>
redirection is used, which allows you to suppress stderr output (2>$null
) or to send it to a file (2>errs.txt
)Unfortunately, as of PowerShell 7.2, in the context of PowerShell jobs (created with
Start-Job
orStart-ThreadJob
) and remoting (e.g., inInvoke-Command
-ComputerName ...
calls), stderr output is mapped to PowerShell's error stream (the stream with number2
, analogous to how stdout can be referred to incmd.exe
and POSIX-compatible shells).$ErrorActionPreference = 'Stop'
is in effect or-ErrorAction Stop
is passed toReceive-Job
orInvoke-Command
, for instance, any stderr output from external programs will trigger a script-terminating error - even with stderr output comprising status messages only. Due to a bug in PowerShell 7.1 and below this can also happen in local, foreground invocation if a2>
redirection is used.The upshot:
To silence stderr output, apply
2>$null
- either at the source (inside the job or remote command), or on the receiving end.To route stderr output (all streams) via the success output stream / stdout, i.e. to merge all streams, use
*>&1
To prevent the stderr lines from printing in red (when originating from jobs or remote commands), apply this redirection at the source - which also guards against side effects from
$ErrorActionPreference = 'Stop'
/-ErrorAction Stop
on the caller side.Note: If you merge all streams with
*>&1
, the order in which stdout and stderr lines are output is not guaranteed to reflect the original output order, as of PowerShell 7.2.If needed, PowerShell still allows you to later separate the output lines based on whether they originated from stdout or stderr - see this answer.