Encoding problem when generating user in Microsoft Graph

Question

What character encoding should a POST body to /v1.0/users have?

I have problems with generating users with accented characters in the name with Microsoft Graph.

I use the following body:

{
    "accountEnabled":false,
    "displayName":"ADM Joost Müller",
    "userprincipalname":"[email protected]",
    "mailNickname":"admjmuller",
    "passwordProfile":{      
        "forceChangePasswordNextSignIn":true,
        "forceChangePasswordNextSignInWithMfa":true,
        "password":"Randompasswordgenerated"
    }
}

When calling POST to /v1.0/users with the above JSON in the body I get the following object back:

@odata.context    : https://graph.microsoft.com/v1.0/$metadata#users/$entity
id                : xxxxxxxx-xxxx-xxxx-xxxx-a72a188e6d9a
businessPhones    : {}
displayName       : ADM Joost M�ller
givenName         : 
jobTitle          : 
mail              : 
mobilePhone       : 
officeLocation    : 
preferredLanguage : 
surname           : 
userPrincipalName : [email protected]

Note the '�' in the displayname. This is NOT a display problem in Powershell, the Entra ID blade in the Azure portal also shows this character. I've already tried to forcibly convert the JSON to UTF-8:

$enc = [System.Text.Encoding]::UTF8
$jsonutf8 = $enc.getstring($enc.getbytes($json))

but that didn't solve the problem.

I've tried various search terms but can't find anything that points to a solution. It's probably something trivial but if it is, I'm using the wrong search terms...

The actual code:

function Encode {
  Param(
    [string]$text
  )
  $enc = [System.Text.Encoding]::Utf8
  return $enc.getstring($enc.getbytes($text))
}

function Invoke-GraphPost {
  [cmdletbinding()]
  Param(
    [parameter(Mandatory = $true)][string]$API,
    [parameter(Mandatory = $true)][string]$AccessToken,
    [parameter(Mandatory = $true)]$body,
    [parameter(Mandatory = $false)][string]$Apiversion = "v1.0"
  )
  $header = @{
    Authorization  = ("Bearer {0}" -f $AccessToken)
    'Content-Type' = 'application/json; charset=utf-8'
  }
  $bodyjson = Encode(ConvertTo-Json -InputObject $body -Compress)
  $URI = Encode(("https://graph.microsoft.com/{0}/{1}" -f $Apiversion, $API))
  try {
    $result = Invoke-RestMethod -Uri $URI -Method Post -Body $bodyjson -Headers $header
  }
  catch {
    $FailMessage = ("Unable to create graph POST request {0} with body {1}" -f $URI, $bodyjson)
    $module.failjson($FailMessage)
  }
  return $result
}

      $password = [System.Web.Security.Membership]::GeneratePassword(12, 4)
      $upnbase = New-EntraUserName -NameFormat "$adminprefix$format" -Voornaam $User.Roepnaam -Infix $User.Voorv_gebnaam -Achternaam $User.Geboortenaam -Domainname $admindomain
      $body = @{
        accountEnabled    = $false
        displayName       = ("ADM {0} {1}" -f $User.Roepnaam.Trim(), $User.Volledige_achternaam.Trim())
        mailNickname      = $upnbase
        userprincipalname = ("{0}@{1}" -f $upnbase, $admindomain)
        passwordProfile   = @{
          forceChangePasswordNextSignIn        = $true
          forceChangePasswordNextSignInWithMfa = $true
          password                             = $password
        }
      }
$newuser = Invoke-GraphPost -API users -AccessToken $token.access_token -body $body

where New-EntraUserName generates a unique username based on givenname, infix and surname (that exact code is irrelevant for the problem, for Joost Müller, the generated username is admjmuller, so without accent).

Joost

Answer 1

Your request body is not being submitted with the expected UTF-8 character encoding, for the following reasons:

• Your Encode function is a no-op: it converts a string to and back from a UTF-8 byte representation, so that you're ending up with the same .NET string (of type [string], which is internally composed of UTF-16 Unicode code units).

• Thus, you're passing a regular [string] instance to the -Body parameter and are therefore delegating the decision as to what character encoding to use to
  Invoke-RestMethod (the following applies analogously to Invoke-WebRequest as well):

  • While you're trying to indicate the desired encoding via a header field, - the 'Content-Type' = 'application/json; charset=utf-8' entry in the hashtable you're passing to -Header - PowerShell 7.3-, including Windows PowerShell, unfortunately does not honor this, and default to ISO-8559-1 encoding.

    • However, a charset attribute as part of the separate -ContentType parameter is honored (see below).

  • PowerShell (Core) 7.4+ now honors a header field too and in the absence of a charset attribute now defaults to UTF-8.

---

Solutions:

• PowerShell 7.4+:

  • Given the above, your code should work as-is - except that you don't need the redundant Encode call.

• PowerShell 7.3- and Windows PowerShell:

  • Option 1:

    • Omit the (pointless) Encode call.
    • Don't add a Content-Type entry to $header.
    • Instead, add a -ContentType 'application/json; charset=utf-8' argument to the Invoke-RestMethod call.

  • Option 2:

    • Perform the UTF-8 encoding yourself and pass a byte array to the -Body parameter:

       Invoke-RestMethod -Uri $URI -Method Post -Headers $header -Body (
         [Text.UTF8Encoding]::new().GetBytes(
           (ConvertTo-Json -InputObject $body -Compress)
         )
       )