1984103 - div inside noscript does not end head for charset meta purposes

Status: NEW

(Core :: DOM: HTML Parser, defect, P4)

Description

[Calin Tanase]

Environment:
Operating system: Android 16/Windows 10
Firefox version: Firefox 141.0/142/144

Steps to reproduce:

1. Go to https://ochag-seversk.ru/
2. Observe the text on the page.

Expected Behavior:
The Cyrillic characters are rendered.

Actual Behavior:
The Cyrillic characters are not rendered, question mark symbols displayed instead.

Notes:

• Reproduces regardless of the status of ETP
• Reproduces in firefox-nightly, and firefox-release
• Does not reproduce in chrome

Created from https://github.com/webcompat/web-bugs/issues/172548

Comment 1

[Calin Tanase]

Attachment: image.png

Comment 2

[Sky Schubert [:skyschub]]

They claim it's utf-8 in their meta tag, but DevTools disagree:

The byte stream was erroneous according to the character encoding that was declared. The character encoding declaration may be incorrect.

Henri, this sounds like something you'd know about. :)

Comment 3

[Henri Sivonen (:hsivonen)]

(In reply to Dennis Schubert [:denschub] from comment #2)

They claim it's utf-8 in their meta tag

For security reasons, we don't automatically second-guess an explicit declaration. However, the Repair Text Encoding command in the View menu (or the toolbar if put there via customization) changes the encoding to detected windows-1251.

Does not reproduce in chrome

The interesting thing here is why this happens. Looking at Safari, it seems that Safari doesn't see the UTF-8 meta, so perhaps Chrome doesn't see it, either, so Chrome's detector takes effect.

So why don't Safari and Chrome see the meta?

Comment 4

[Henri Sivonen (:hsivonen)]

The meta is not within the first 1024 bytes, but it's still within head. Something makes Chrome and Safari use their 1024-byte limit instead of the looking though to the end of head if that's further away.

Comment 5

[Henri Sivonen (:hsivonen)]

One hypothesis is that there's a network packet boundary between the 1024-byte limit and the meta, but I haven't confirmed.

Most likely WONTFIX.

Comment 6

[Sky Schubert [:skyschub]]

That almost seems like a bug we should file with Google and Apple? I didn't even know about this behavior, and I think it's unexpected from a Web Developer point of view.

Comment 7

[B.J. Herbison]

(In reply to Henri Sivonen (:hsivonen) from comment #4)

The meta is not within the first 1024 bytes, but it's still within head. Something makes Chrome and Safari use their 1024-byte limit instead of the looking though to the end of head if that's further away.

The "something" is the HTML standard:

4.2.5.4 Specifying the document's character encoding
The following restrictions also apply:

• The element containing the character encoding declaration must be serialized completely within the first 1024 bytes of the document.

I don't know how long that restriction has been around, but I found a stack overflow comment mentioning it. The comment was created in 2009 and edited in 2013.

The page is badly written, but this is a Firefox bug.

Comment 8

[Daniel Veditz [:dveditz]]

The spec also says If the attribute is present, its value must be an ASCII case-insensitive match for the string "utf-8".—in which case why support it? There are a lot of "musts" in that spec that don't say what to do when real-world web pages inevitably disobey.

Comment 9

[Henri Sivonen (:hsivonen)]

(In reply to Henri Sivonen (:hsivonen) from comment #5)

One hypothesis is that there's a network packet boundary

For clarity, Gecko's behavior doesn't depend on packet boundaries but Chrome's does. "Fixing" this by introducing network packet boundary-dependent behavior to Gecko is not OK. I haven't confirmed if the network packet boundary thing is the explanation for the Chrome/Safari behavior here.

Comment 10

[Henri Sivonen (:hsivonen)]

The "something" is the HTML standard:
...
this is a Firefox bug.

In general (with the exception of whatever is going on here in Chrome and Safari), Firefox, Chrome, and Safari look for the encoding meta within head as "within head" was understood before the introduction of the template element, i.e. <template> is treated as ending head for this purpose, or within the first 1024 bytes, whichever is larger. The text you are reading is the requirements for authoring Web pages, not for how to consume them.

Comment 11

[Henri Sivonen (:hsivonen)]

Looking at this again, the most suspicious thing is <noscript><div><img src="https://mc.yandex.ru/watch/96441676" style="position:absolute; left:-9999px;" alt="" /></div></noscript>. If that's parsed in the "scripting disabled" mode, the <div> tag ends head.

And, indeed, I have already written comment about this: https://searchfox.org/firefox-main/rev/21a394033d607022640b4a32d85694aac8b19a54/parser/html/nsHtml5StreamParser.cpp#2054-2057

We could add another level of tokenization for the contents of noscript, but that would be complexity for relatively little gain.

S3 due to a workaround existing: The Repair Text Encoding command.
P4 due to the complexity vs. benefit situation.

If someone has the energy to contact this particular site, having the site swap the order of noscript and meta charset would help.

Comment 12

[Henri Sivonen (:hsivonen)]

(In reply to Henri Sivonen (:hsivonen) from comment #11)

If someone has the energy to contact this particular site, having the site swap the order of noscript and meta charset would help.

Help in the sense of making the site work the same way in all browsers and be equally unreadable. To actually make it readable, additionally windows-1251 needs to be declared in the meta instead of declaring UTF-8.