1919205 - Internal stylesheet `plaintext.css` (UTF-8) isn't honored in rendering of plaintext document that uses UTF-16 page

Status: RESOLVED FIXED

(Core :: Networking, defect)

Description

[zhou]

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0

Steps to reproduce:

Open any plaintext document in UTF-16 encoding, e.g.:

1. (UTF-16LE) https://raw.githubusercontent.com/stain/encoding-test-files/master/utf16.txt
2. (UTF-16BE) https://raw.githubusercontent.com/unicode-org/icu/main/icu4c/source/test/testdata/encoded.utf16be

Actual results:

The stylesheet plaintext.css (resource://content-accessible/plaintext.css), which is in UTF-8, becomes mojibake and is completely broken, as can be seen in the style editor.

And of course, you get a warning in the console:

Ruleset ignored due to bad selector.

Expected results:

My guess is, the stylesheet somehow inherits the wrong encoding from the document, since the <link> element does not have the charset attribute (I know it is deprecated, but adding a charset="UTF-8" solves the issue).

Maybe internal resources should specify an encoding using Content-Type header, but I am not sure if that is possible, given they are local assets.

Comment 1

[Francesco Lodolo [:flod]]

Not completely sure this is about CSS Parsing, please move if there's a better product/component.

Comment 2

[Daniel Holbert [:dholbert]]

I can reproduce. At first glance I thought maybe this was just a display issue with DevTools, but it looks like the actual styles themselves aren't applied, so we're rejecting the stylesheet itself as well.

Screenshots coming up to demonstrate the bug a bit more.

Comment 3

[Daniel Holbert [:dholbert]]

Attachment: screenshot of bug (note the contents of the Style Editor in DevTools)

Comment 4

[Daniel Holbert [:dholbert]]

Attachment: screenshot demonstrating that the stylesheet really isn't getting picked up in the utf-16 document

Here's a screenshot comparing
https://raw.githubusercontent.com/stain/encoding-test-files/master/utf16.txt
...to a simpler plaintext document:
data:text/plain,hi
...in a fresh profile where I've set a dark color-scheme preference.

You can see a few things demonstrating that the utf16 document is missing the stylesheet (on top of the garbled style-editor view shown in my previous screenshot):

• The user's dark color-scheme preference isn't respected (because we're missing the color-scheme: light dark declaration in the stylesheet)
• the specified styles are all missing in the specified-style view.
• the computed styles of e.g. pre are different (text-wrap-mode is nowrap in the utf16 doc vs wrap in the simpler one), since we're missing the white-space: pre-wrap; style from plaintext.css (which is a shorthand that sets a few things, including textwrap-mode:wrap).

Comment 5

[Daniel Holbert [:dholbert]]

Attachment: testcase 1 (the utf16 doc)

Here's a copy of https://raw.githubusercontent.com/stain/encoding-test-files/master/utf16.txt for archival/reference.

Comment 6

[Daniel Holbert [:dholbert]]

I found a regression range for the rendering difference shown in my second screenshot (the styles being actually-honored-or-not).

Last good revision: b538ca7373143e97e0c5243dfaf6e941d9454d25 (2021-12-22)
First bad revision: bd494168b95a83344e778b5b4679b67101482118 (2021-12-23)
Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=b538ca7373143e97e0c5243dfaf6e941d9454d25&tochange=bd494168b95a83344e778b5b4679b67101482118

Here, the older "good" builds are parsing-and-honoring the stylesheet (i.e. they use a dark-color-scheme and show the correct computed style for the white-space property, from plaintext.css). The "good" builds do still show a garbled string in the DevTools style-editor, but I'm less concerned with the DevTools style-editor garbledness, since that's not as user-facing as whether or not we actually honor the styles for rendering purposes.

In that regression range, Bug 1745142 ("Communicate encoding commitment via speculative load queue") looks like the most-likely regressor. Hence, I'm tentatively reclassifying to that component.

Comment 7

[BugBot [:suhaib / :marco/ :calixte]]

Set release status flags based on info from the regressing bug 1745142

Comment 8

[Masatoshi Kimura [:emk]]

We are supposed to always treat resource: URLs as UTF-8-encoded.
https://searchfox.org/mozilla-central/rev/43450868e9f3e26da88a02837e5a43c93728ea95/parser/nsCharsetSource.h#44

Comment 9

[Daniel Holbert [:dholbert]]

The garbledness in the DevTools style-editor view here is a much older regression.

For that behavior-change, I get this regression-range:
Last good revision: 2694ff2ace6a (2015-06-19)
First bad revision: c319f262ce3e (2015-06-20)
Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=2694ff2ace6a&tochange=c319f262ce3e

Before that range, DevTools style-editor shows resource://gre-resources/plaintext.css (the path for this file at-that-time) just fine.
After that range, it shows the same garbledness.

However: even in the "good" builds there, the styles aren't actually honored -- computed-style for pre shows white-space being set to pre (rather than pre-wrap as specified in the stylesheet, which does show up for simpler plaintext documents). In other words, sufficiently-older builds are "bad" with respect to the regression range in comment 6. They switch from bad-to-good (for whether the styles are actually honored) with this fix-range:
First good revision: 567a8768593eb06a86deb263f94d9de2d3d3e8fa (2020-06-15)
Last bad revision: c68fe15a81fc2dc9fc5765f3be2573519c09b6c1 (2020-06-14)
Pushlog:
https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=c68fe15a81fc2dc9fc5765f3be2573519c09b6c1&tochange=567a8768593eb06a86deb263f94d9de2d3d3e8fa

(Maybe in there the fix would have been from bug 1599160? not sure.)

Comment 10

[Daniel Holbert [:dholbert]]

So, summing up, there were several historical behavior-changes here. Summing those up (with the changes-with-respect-to-previous-state highlighted in bold):

1. From ??? until 2015-06-19:

• DevTools style-editor view of plaintext.css looks fine.
• But plaintext.css styles don't make it into the computed style (and hence don't impact rendering).

2. From 2015-06-20 to 2020-06-14:

• DevTools style-editor view of plaintext.css looks garbled.
• plaintext.css styles still don't make it into computed style (and hence don't impact rendering).

3. From 2020-06-15 to 2021-12-22

• DevTools style-editor view of plaintext.css still garbled.
• plaintext.css styles do make it into computed style (and hence do impact the rendering).

4. From 2021-12-22 to present day:

• DevTools style-editor view of plaintext.css still garbled.
• plaintext.css styles still don't make it into computed style (and hence don't impact rendering).

(Note that the behaviors for #2 and #4 are the same; so we inadvertently fixed and then regressed something there.)

Comment 11

[Emilio Cobos Álvarez [:emilio]]

There are various potential fixes, easiest one would be adding a charset attr or @charset to the link, but probably we want the resource channel to claim utf-8 encoding which it probably doesn't right now.

Comment 12

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1919205 - Clean-up initialization of nsJARChannel. r=#necko-reviewers

I wrote this while debugging, so no reason not to do it I guess.

No behavior change.

Comment 13

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1919205 - Clean up CSS Loader encoding detection. r=#layout,#style

I wrote this while debugging, so no reason not to do it I guess.

No behavior change.

Comment 14

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1919205 - Make resource/chrome URIs default to UTF-8 encoding. r=#necko-reviewers

I could make this change scoped to stylesheets if needed, but
I don't see why?

Comment 15

[Emilio Cobos Álvarez [:emilio]]

Attachment: Bug 1919205 - Add a test for this bug. r=#layout,#style

Comment 16

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/a75df76f05ee
Clean-up initialization of nsJARChannel. r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/0f3b0b53604d
Clean up CSS Loader encoding detection. r=firefox-style-system-reviewers,zrhoffman
https://hg.mozilla.org/integration/autoland/rev/a47cc1e53989
Make resource/chrome URIs default to UTF-8 encoding. r=necko-reviewers,valentin
https://hg.mozilla.org/integration/autoland/rev/bede85b600ce
Add a test for this bug. r=firefox-style-system-reviewers,zrhoffman
https://hg.mozilla.org/integration/autoland/rev/b3b9b1a60847
apply code formatting via Lando

Comment 17

[Cristina Horotan [:chorotan]]

https://hg.mozilla.org/mozilla-central/rev/a75df76f05ee
https://hg.mozilla.org/mozilla-central/rev/0f3b0b53604d
https://hg.mozilla.org/mozilla-central/rev/a47cc1e53989
https://hg.mozilla.org/mozilla-central/rev/bede85b600ce
https://hg.mozilla.org/mozilla-central/rev/b3b9b1a60847