Closed Bug 647403 Opened 9 years ago Closed 8 years ago

Firefox 4 submits string encoded with Big5 as UTF-8

Categories

(Core :: Internationalization, defect)

All
Windows 7
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla11

People

(Reporter: jeff_tong, Assigned: emk)

References

()

Details

Attachments

(2 files, 3 obsolete files)

User-Agent:       Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0
Build Identifier: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0) Gecko/20100101 Firefox/4.0

The site is Hong Kong public library material search page which is encoded with traditional Big5 code (Traditional Chinese). Any Chinese character input in the page is treated as UTF-8. As a result, nothing was found with the browser.

Reproducible: Always

Steps to Reproduce:
1. enter any Traditional Chinese character
Actual Results:  
There is a warning message that no result is found.
Version: unspecified → 4.0 Branch
Assignee: nobody → smontagu
Component: General → Internationalization
Product: Firefox → Core
QA Contact: general → i18n
Version: 4.0 Branch → unspecified
I couldn't reproduce this bug. For example, I searched for 春秋 and was sent to the URL http://libcat.hkpl.gov.hk/webpac_cjk/wgbroker.exe?201104030406530250811+-access+top.books-page+search+open+BT+%ACK%AC%EE%23%23A:NONE%23NONE:NONE::%23%23

%ACK%AC%EE is equivalent to 春秋 encoded in Big5

Is network.standard-url.encode-query-utf8 set to true for you in about:config?
I have just checked. It is set to true but I swear I have never touched it and the problem can be reproduced on Win7 and WinXP machine.
> but I swear I have never touched it 

And none of your extensions did either?
I have found the problem since beta release.

I always tested whether the problems solved on different machines shortly after installation. None of extension was added until few days ago.
Are you using the localized version of Firefox? The zh-tw localization sets the pref to true.
Probably we should make a pref based on bug 393246 with fewer side effects for Localizers...
As I explained in bug 552273 comment #17, legacy Western sites are already "broken" on Japanese (and perhaps Taiwan?) locales. So bug 393246 patch has viatually zero side effect on those locales.
Attached patch patch (obsolete) — Splinter Review
I've added "browser.fixup.use-utf8" pref which is usable for localizers on some non-Western locales. If the pref is set to true, CreateFixupURI will always use UTF-8 as a default encoding. Only the location bar is affected by the pref.
I've also removed "network.standard-url.encode-query-utf8" pref because we will be unable to turn on the pref forever due to Web compat curse and RFC3987bis will formalize the current behavior. Now the query will be always encoded in the document charset.
Assignee: smontagu → VYV03354
Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true
Attachment #524051 - Flags: review?(smontagu)
(In reply to comment #5)
> Are you using the localized version of Firefox? The zh-tw localization sets the
> pref to true.

Sure. Traditional Chinese version (zh-tw).
Blocks: 472669
I have just tested Firefox 5.0 Traditional Chinese version. I am disappointed that the problem still remains.
Attached patch patch (obsolete) — Splinter Review
Attachment #572345 - Flags: review?(bzbarsky)
Comment on attachment 572345 [details] [diff] [review]
patch

Unbitrotted and switching the reviewer
Attachment #524051 - Flags: review?(smontagu)
Comment on attachment 572345 [details] [diff] [review]
patch

Please parenthesize the "aFixupFlags & FIXUP_FLAG_USE_UTF8" bit.

Also please make sure the l10n impact of this gets flagged; localizations that set the old pref probably need to set the new one.

r=me
Attachment #572345 - Flags: review?(bzbarsky) → review+
> Also please make sure the l10n impact of this gets flagged; localizations that set the old pref probably need to set the new one.
How can I flag for the l10n impact?

zh-TW is the only locale which modified the pref as far as I know.
https://mxr.mozilla.org/l10n-central/search?string=encode-query-utf8
Tim, please replace "network.standard-url.encode-query-utf8" with "browser.fixup.use-utf8" when this patch is landed. It will fix this bug and will not regress bug 443588.
Target Milestone: --- → Future
It's unwise to land this just before the aurora merge (meaning l10n freeze).
Target Milestone: Future → mozilla11
Attachment #524051 - Attachment is obsolete: true
Attachment #572345 - Attachment is obsolete: true
Keywords: checkin-needed
Attached patch patch for l10n-central/zh-TW (obsolete) — Splinter Review
double check before landing
Attachment #573190 - Flags: review?
Peter, anyone you want to review your change in particular?

Also, you probably want a different comment, as you're not working around something, you're merely adjusting a setting to a good local default, I guess?
(In reply to Axel Hecht [:Pike] from comment #18)
> Peter, anyone you want to review your change in particular?
> 
> Also, you probably want a different comment, as you're not working around
> something, you're merely adjusting a setting to a good local default, I
> guess?

Maybe you Pike, or emk could review for this? sorry I did not add requestee. 

And that's why I need to make sure in case there's still something missing.
Attachment #573190 - Attachment is obsolete: true
Attachment #573190 - Flags: review?
Attachment #573198 - Flags: review?(l10n)
Attachment #573198 - Flags: review?(VYV03354)
Comment on attachment 573198 [details] [diff] [review]
patch for l10n-central/zh-TW, v2

You don't have to change the comment. This is still a workaround for bug 443588.
This bug will be fixed regardless of your settings.
Otherwise looks good.
Attachment #573198 - Flags: review?(VYV03354) → review+
Comment on attachment 573198 [details] [diff] [review]
patch for l10n-central/zh-TW, v2

Ah, Axel is right about this is no longer a workaround because bug 443588 is Traditional Chinese specific.
Change the comment to "Fix bug 443588".
https://hg.mozilla.org/mozilla-central/rev/0246515e5fda
Status: ASSIGNED → RESOLVED
Closed: 8 years ago
Resolution: --- → FIXED
Blocks: 443588
Attachment #573198 - Flags: review?(l10n) → review+
You need to log in before you can comment on or make changes to this bug.