1166558 - Charset should default to utf-8 for application/json

Status: RESOLVED DUPLICATE of bug 741776

(Core :: General, defect)

Description

[Ryan S. Scheel [:havvy]]

As per the JSON spec, a JSON file must be one of utf-8, utf-16, or utf-32. 

When viewing a JSON file that does not send the charset header, Firefox should assume that the charset is UTF-8 by default instead of ASCII.

When fixed, Felix's name on https://registry.npmjs.org/npm will show correctly. (To find it, just search "Felix Geisend". Currently it's Felix Geisendörfer.)

Comment 2

[Michelle Funches - QA]

@Reporter:
1) What version of Firefox was in use?
2) What is the operating system Windows, Linux, Macintosh...
3) Detailed test case

Appears fixed for future release. Tested in :
Version 	46.0a1
Build ID 	20160104030217
User Agent 	Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:46.0) Gecko/20100101 Firefox/46.0

Comment 3

[Ryan S. Scheel [:havvy]]

It's still happening in Firefox 43.0.4. I tried to test it on developer edition, but developer edition keeps failing to properly download the entire file, thus showing me syntax errors instead of the file.

Comment 4

[Michelle Funches - QA]

@Reporter: 

If you would like to confirm a fix is already in place, please test in Nightly and let me know if your results are different than mine.

I tested this in the Nightly version [46.0a1] [Nightly.Mozilla.Org]

Current Developer version is 45.x [45.0a2] I agree the fix is not in that future package/release.

Comment 6

[Michelle Funches - QA]

WFM no update from Reporter, closing as R-WFM

Comment 7

[Henri Sivonen (:hsivonen)]

This is not WORKSFORME. (For incomplete reports we have INCOMPLETE, but this bug has quite enough info.)

Comment 8

[Stijn de Witt]

Can this bug please be re-opened?

There are two issues:
This one, about Firefox not defaulting to UTF-8 for application/json, which is not fixed.
I just tested this in 46.0.1, the problem is still there, characters outside of ASCII get mangled.

And then there is 
Bug 741776 - Use UTF-8 and don't honor the charset parameter on application/json resources loaded as documents
Which is about the fact that Firefox honors the encoding argument, even though that's not strictly allowed by the spec. The spec says it should be ignored.

Personally I think this bug is far worse than the other one, because it actually interprets JSON in the *wrong* encoding. The other one actually allows us to work around this bug, but we loose spec compliance.

Note that Chrome is also defaulting to the wrong charset, which means ATM half of the world's browser marketshare is not processing application/json correctly.
https://bugs.chromium.org/p/chromium/issues/detail?id=438464&

Also see this StackOverflow thread as an example of real-life confusion over this. 
http://stackoverflow.com/questions/13096259/can-charset-parameter-be-used-with-application-json-content-type-in-http-1-1/13098683
IMHO the JSON spec is wrong and encoding should be honored, falling back to UTF-8 when none is provided. Because now UAs have to 'guess' at the encoding based on the JSON content which is just yuck.

Please re-open. This issue is not fixed and not a dupe.

Comment 9

[Henri Sivonen (:hsivonen)]

We aren't going to fix this in two steps with this description being addressed first and then bug 741776 later. When we do something, we should do bug 741776 comment 4 right away.