333859 - Consider sending query in UTF-8 by default

Status: RESOLVED WONTFIX

(Core :: Networking, defect)

Description

[Masatoshi Kimura [:emk]]

IE7 (at least Beta2 Preview) enables the corresponding setting by default.
I hope we could do the same thing on Fx3.
Probably we will have to compromise on Fx2.

Comment 1

[Jesse Ruderman]

Is this different from changing the default character coding?  I think Firefox usually sends form queries using the same character coding the page was interpreted with.

Comment 2

[Masatoshi Kimura [:emk]]

(In reply to comment #1)
> Is this different from changing the default character coding?  I think Firefox
> usually sends form queries using the same character coding the page was
> interpreted with.
For example, location bar. If you type in
http://www.google.com/search?q=<some non-ascii chars>
it will be encoded in UTF-8 only if network.standard-url.encode-query-utf8 is turned on.

Comment 3

[Masatoshi Kimura [:emk]]

Comment #2 is about Windows. It will be always encoded in UTF-8 on UTF-8 locale platforms.

Comment 4

[Masatoshi Kimura [:emk]]

"Send UTF-8 query strings" was removed from IE7 Beta 2 Build 7.0.5346.5 :(

Comment 5

[Curugwath]

The default setting breaks the World of Warcraft Armory (a Blizzard Entertainment site). Attempting to visit the url:

http://eu.wowarmory.com/character-sheet.xml?r=Moonglade&n=Áá

results in the name being changed by Firefox to '%C1%E1' and an error. Setting network.standard-url.encode-query-utf8 to true fixes the problem.

[Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9.0.1) Gecko/2008070208 Firefox/3.0.1 (.NET CLR 3.5.30729)]

Comment 6

[Mike Beltzner [:beltzner, not reading bugmail]]

Is there any reason *not* to do this? Carrying over blocking flag from duplicate bug 461304 ...

Comment 8

[:Gavin Sharp [email: [e-mail]]]

(In reply to comment #6)
> Is there any reason *not* to do this? Carrying over blocking flag from
> duplicate bug 461304 ...

It will break sites that don't expect UTF-8. It's not clear to me which option causes more problems. Bug 261929 has some extra context.

Comment 9

[Benjamin Smedberg]

I think the risk here is too much to take in 1.9.1 now.

Comment 10

[Fazal Majid]

The default behavior network.standard-url.encode-query-utf8=false as of Firefox 3.5) is not compliant with RFC3986, which mandates encoding with UTF-8 prior to urlencoding.

Comment 11

[Tim Guan-tin Chien [:timdream] (please needinfo)]

It turned out there are more issue with this pref if we don't change the current default. Bug 443588 demonstrated in a non-English Windows, an URL with query will be broken whenever one tries to accesses it using location bar.

Location bar renders all CJK characters when they were sent by UTF-8 (not affected by the pref), but sent the URL again in legacy system encoding when user press [enter] again (which is affected by the pref).

I would suggest we change the pref, rather than escaping CJK chars on location bar when they were actually sent using UTF-8.

Comment 12

[Boris Zbarsky [:bzbarsky]]

Note that changing the pref would break libcat.hkpl.gov.hk; see bug

Comment 13

[Boris Zbarsky [:bzbarsky]]

Er, see bug 647403.

Comment 14

[Masatoshi Kimura [:emk]]

WONTFIXing because HTML5 defines a willful violation of RFC 3986 and RFC 3987 about escaping a query component.
http://dev.w3.org/html5/spec/Overview.html#resolving-urls
We can't always encode query part in UTF-8 for Web compat.
For a location bar, bug 393246 would be more appropreate.