Powershell script error when reading JSON file

Question

Im running this piece of code in powershell

 $url = "https://someURL/file.json"
$json = (New-Object System.Net.WebClient).DownloadString($url)

$data = $json | ConvertFrom-Json

$data | ConvertTo-Json -depth 100 | Out-File "C:\scripts\LW\data.json"

but when I run it I got this error: the length of the string exceeds the value set on the maxjsonlength property powershell

Any solution for this ?

Answer 1

I don't have a complete explanation, because I cannot reproduce the symptom,^[1] but what seems to be happening is:

• In Windows PowerShell (whose latest and last version is v5.1), ConvertFrom-Json uses the (deprecated) System.Web.Script.Serialization.JavaScriptSerializer .NET class.

• An instance of this class (if directly constructed, which ConvertFrom-Json must do) has a limit on the "maximum length of JSON strings", reflected in its MaxJsonLength property whose default value is "2097152 characters, which is equivalent to 4 MB of Unicode string data."

• It is unclear whether ConvertFrom-Json uses this default or sets a different value, but you seem to have run into the effective limit, and ConvertFrom-Json does not expose a way to increase that limit.

  • However, in Windows PowerShell v5.1 no artificial limit is imposed,^[2] given that ConvertFrom-Json there seemingly can even handle .NET strings of maximum length.^[1]

Therefore, you may have to find a different way to parse your JSON data; per the recommendations in the JavaScriptSerializer docs, there are at least two options (v5.1 PowerShell versions now build on .NET Framework v4.8):

For .NET Framework 4.7.2 and later versions, use the APIs in the System.Text.Json namespace for serialization and deserialization.
For earlier versions of .NET Framework, use Newtonsoft.Json.

Newtonsoft.Json is actually what underlies the PowerShell (Core) 7+ implementation of ConvertFrom-Json, though a move to System.Text.Json has at least been contemplated, but a PR to that effect was abandoned due to lack of resources - see GitHub PR #11198.

I presume that neither of these APIs is subject to the same length limitation as JavaScriptSerializer.

Note:

• Direct use of NewtonSoft.Json isn't PowerShell-friendly.

• Third-party module PSAdvancedJsonCmdlet - which I haven't personally used - describes itself as "Json cmdlets for Windows PowerShell 5.1, backporting from PowerShell 7", so it may be an option for you.

---

^{[1] You managed to download your data into a .NET string. .NET strings can only comprise about 1+ billion characters (16-bit code units), amounting to 2+ billion bytes (the actual string-length limit is close to 1 GB, which is equivalent to ([int]::MaxValue + 1) / 2). When I create a trivial JSON document (containing a JSON string value only) as a string with the maximum length, ConvertFrom-Json is able to parse it on my Window 11 machine running Windows PowerShell v5.1.22621.1778 (.NET Framework 4.8.9167.0); conceivably, however, earlier PowerShell versions had lower MaxJsonlength limits:
('"{0}"' -f ('x' * ([int]::MaxValue / 2 - 32 - 2 - 1)) | ConvertFrom-Json).Length}

^{[2] Mathias R. Jessen decompiled the Microsoft.PowerShell.Commands.Utility.dll assembly using ILSpy and found that MaxJsonLength is set to int.MaxValue in v5.1, which is a limit that at least a single string cannot even reach (see previous footnote).}