Calling a powershell script from another powershell script and guaranteeing it is UTF8

Question

I assembled a Powershell script that is designed to grab other scripts that are hosted on Azure blobs, and execute them.

The relevant code blocks:

Obtaining the script:

$resp = (Invoke-WebRequest -Uri $scriptUri -Method GET -ContentType "application/octet-stream;charset=utf-8")
$migrationScript = [system.Text.Encoding]::UTF8.GetString($resp.RawContentStream.ToArray());
$tempPath = Get-ScriptDirectory
$fileLocation = CreateTempFile $tempPath "migrationScript.ps1" $migrationScript

Creating the file:

$newFile = "$tempFolder"+"\"+"$fileName"
Write-Host "Creating temporary file $newFile"
[System.IO.File]::WriteAllText($newFile, $fileContents)

And then I invoke the downloaded file with

Invoke-Expression "& `"$fileLocation`" $migrationArgs"

This is working well, for what I need. However, the Invoke-Expression is not correctly reading the encoding of the file. It opens correctly in Notepad or Notepad++, but not in ISE (where I am executing the script right now).

Is there a way I can ensure the script is read correctly? It is necessary to support UTF8, as there is a possibility that the scripts will need to perform operations such as setting an AppSetting to a value that contains special characters.

EDIT: Behaviour is the same on "vanilla" non-ISE Powershell invocation.

Answer 1

As per @lit and @PetSerAI, the BOM is required for Powershell to work correctly.

My first attempt had not been successful, so I switched back to non-BOM, but, with the following steps, it worked:

1. Perform the Invoke-WebRequest with -ContentType "application/octet-stream;charset=utf-8"

2. Grab the Raw content (you will see it in Powershell as a series of numbers, which I assume are the ascii codes?) and convert its bytes with [system.Text.Encoding]::UTF8.GetString($resp.RawContentStream.ToArray()); to an array containing the characters you want.

3. When saving the file via .NET's WriteAllText, ensure you use UTF8, [System.IO.File]::WriteAllText($newFile, $fileContents, [System.Text.Encoding]::UTF8). In this case, UTF8 is understood to be UTF8 with a byte order mark, and is what Powershell needs.