ActiveSupport::JSON.decode does not properly handle literal line breaks

Question

Is this the expected behavior? Note how the line break character gets lost.

ruby-1.9.2-p136 :001 > ActiveSupport::JSON.decode("{\"content\": \"active\n\nsupport\"}")
 => {"content"=>"active\nsupport"}

The same happens with unicode-escaped line breaks:

ruby-1.9.2-p136 :002 > ActiveSupport::JSON.decode("{\"content\": \"active\u000a\u000asupport\"}")
 => {"content"=>"active\nsupport"}

I'm using rails 3.0.3.

Answer 1

I eventually came across this ticket: https://rails.lighthouseapp.com/projects/8994/tickets/3479-activesupportjson-fails-to-decode-unicode-escaped-newline-and-literal-newlines

It seems this is a bug in ActiveSupport that will be fixed in Rails 3.0.5. For now I have patched activesupport and things are working as expected.

ruby-1.9.2-p136 :001 > ActiveSupport::JSON.decode("{\"content\": \"active\n\nsupport\"}")
 => {"content"=>"active\n\nsupport"}

Answer 2

To represent a newline in JSON data using double quotes, you must escape the newline:

ActiveSupport::JSON.decode("{\"content\": \"active\\n\\nsupport\"}")

Otherwise, you are inserting a newline into the JSON source and not the JSON data. Note that this would also work:

ActiveSupport::JSON.decode('{"content": "active\n\nsupport"}')

By using single quotes, you are no longer inserting a literal newline into the source of the JSON.

It is interesting to note the way ActiveSupport handles this by default (the default JSON backend is ActiveSupport::JSON::Backends::Yaml). By installing the json gem and changing the JSON backend to it (ActiveSupport::JSON.backend = 'JSONGem') and attempting to decode the same text (ActiveSupport::JSON.decode("{\"content\": \"active\\n\\nsupport\"}")) you get the following:

JSON::ParserError: 737: unexpected token at '{"content": "active

support"}'