Wrong text encoding with browser.tabs.remote.separatePrivilegedContentProcess;true

VERIFIED FIXED in Firefox 63



8 months ago
8 months ago


(Reporter: darkspirit, Assigned: imjching)



Firefox 63
Dependency tree / graph

Firefox Tracking Flags

(firefox-esr60 unaffected, firefox61 unaffected, firefox62 unaffected, firefox63 fixed)



(2 attachments)



8 months ago
Non-standard configuration: browser.tabs.remote.separatePrivilegedContentProcess;true

German umlauts are busted.

(Jan Andre Ikenmeyer [:darkspirit] from bug 1482404 comment 2)
> https://screenshotscdn.firefoxusercontent.com/images/9b74c55a-8acd-48ed-9cfc-e5dc36d4ad91.jpg

> I just noticed text encoding bugs of german words (center bottom context menu + section title "Overview") and will try to narrow it down.

(Should this block the fission meta bug or something else instead?)
Hey jfkthame, can you think of any reason why the separate Activity Stream content process might be rendering this character differently?
Flags: needinfo?(jfkthame)


8 months ago
Blocks: 1472212
At some level, somewhere, there's confusion between a UTF-8 string and Latin-1. So the code units that are meant to make up a single multi-byte UTF-8 character are being interpreted as separate 8-bit characters. I don't know offhand where this is happening, though.

Interestingly, the titles of the actual tiles don't seem to be affected (see the bottom-right one), although the "Overview" heading is.

FWIW, after enabling separatePrivilegedContentProcess on macOS, I see a similar glitch when I go to manually add a "New Top Site": beneath the URL field, there's a link that reads "Use a custom image…". (It's supposed to be an ellipsis, U+2026, which in UTF-8 is the three bytes <E2 80 A6>; in Latin-1, those are <a-circumflex, ctrl-char-80, broken bar>, which is what we see.)
Flags: needinfo?(jfkthame)
I guess one key difference between the text that gets mangled and the tile titles (which work OK) is the source of the text: in the first case, it's part of the browser UI, which I guess gets loaded from resources in omni.jar or somewhere like that, while the titles originate from web content (and presumably get stashed in the profile).

So.... I wonder if the new process is not using the right encoding when it reads UI strings from the application package, for some reason?

Comment 4

8 months ago
Interesting. This should be the fix: https://searchfox.org/mozilla-central/rev/dc28b8bddfbb7bbc89de5d0fd6448589aa6a2991/browser/components/newtab/aboutNewTabService.js#164. At the moment, it is using the default character set, which is "ASCII", to read the strings. We will need to pass in "UTF-8" as the character set when we call `loadSubScript`: https://searchfox.org/mozilla-central/rev/dc28b8bddfbb7bbc89de5d0fd6448589aa6a2991/js/xpconnect/idl/mozIJSSubScriptLoader.idl#24-26.

Tested the fix locally and the strings were displayed correctly: https://imgur.com/SnjNuFR.

Comment 5

8 months ago
The default character set that is used to read sub-scripts using
`mozJSSubScriptLoader` is ASCII. Scripts for Activity Stream contains strings
which are encoded in UTF-8, and therefore, reading them in ASCII will display
the strings incorrectly.
Comment on attachment 8999413 [details]
Bug 1482422 - Use UTF-8 character set instead of ASCII when reading Activity Stream scripts with `mozJSSubScriptLoader`.

Mike Conley (:mconley) (:⚙️) has approved the revision.
Attachment #8999413 - Flags: review+

Comment 7

8 months ago
Pushed by mconley@mozilla.com:
Use UTF-8 character set instead of ASCII when reading Activity Stream scripts with `mozJSSubScriptLoader`. r=mconley
Iteration: --- → 63.4 - Aug 20
Priority: -- → P1
Blocks: 1479806


8 months ago
No longer blocks: 1479806

Comment 8

8 months ago
Last Resolved: 8 months ago
Resolution: --- → FIXED
Target Milestone: --- → Firefox 63

Comment 10

8 months ago
Verified fixed in Nightly 63 x64 20180813220525 de_DE @ Debian Testing (KDE, Xorg). Thank you!

Comment 11

8 months ago
Commit pushed to master at https://github.com/mozilla/activity-stream

chore(mc): Port Bug 1482422 - Use UTF-8 character set instead of ASCII when reading Activity Stream scripts with `mozJSSubScriptLoader`. r=mconley (#4335)
You need to log in before you can comment on or make changes to this bug.