1727491 - Remove support for BOMless unlabeled Latin1 Supplement-range UTF-16LE|BE

Status: RESOLVED FIXED

(Core :: DOM: HTML Parser, task, P2)

Description

[Henri Sivonen (:hsivonen)]

Bug 631751 added sniffing for BOMless UTF-16 if the first kilobyte stays below U+0100. This was done for IE compatibility and has no basis in the spec. Neither WebKit nor Blink has this behavior. (I.e. new Edge doesn't have this behavior.)

Let's remove this oddity to get it out of the way before bug 1701828.

Demo: https://hsivonen.com/test/moz/bomless-utf-16le.htm

Risk: Failing to render some Windows-originating legacy HTML that rendered OK in IE, but is already broken in Safari, Chrome, and Edge.

Comment 1

[Henri Sivonen (:hsivonen)]

Attachment: Bug 1727491 - Remove support for BOMless unlabeled Latin1 Supplement-range UTF-16LE|BE.

Comment 2

[Henri Sivonen (:hsivonen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=ac126ecd4e9792c285acd5ffe4aa8b80bb6a7780

Comment 3

[Henri Sivonen (:hsivonen)]

https://treeherder.mozilla.org/#/jobs?repo=try&revision=49c4eca67e266a608f5c5386056a489d73ef4589

Comment 4

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/dc6b9ca8f3fa
Remove support for BOMless unlabeled Latin1 Supplement-range UTF-16LE|BE. r=emk

Comment 5

[Cristina Cozmuta (:CrissCozmuta)]

Backed out for causing mochitest failures on test_bug631751be.html. CLOSED TREE
Backout link : https://hg.mozilla.org/integration/autoland/rev/4a8481d5aaef1d2a919a9c05dd94789400fc4458
Push with failures : https://treeherder.mozilla.org/jobs?repo=autoland&selectedTaskRun=MCLnyNRZSjaYEAcB8nD4rg.0&resultStatus=testfailed%2Cbusted%2Cexception&revision=dc6b9ca8f3fa68c4ce6320fa19ae05ffec0eec9e&searchStr=mochitest
Link to failure log : https://treeherder.mozilla.org/logviewer?job_id=349921738&repo=autoland&lineNumber=4757

Comment 6

[Henri Sivonen (:hsivonen)]

Sigh. I should have run more test suites in my try runs.

Comment 7

[Henri Sivonen (:hsivonen)]

The tests that failed are tests for the patch being undone here. The reason why this patch made them detect as UTF-8 instead of windows-1252 is that the test runs from a file> URL.

Comment 8

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/c7c2c5f061b4
Remove support for BOMless unlabeled Latin1 Supplement-range UTF-16LE|BE. r=emk

Comment 9

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/c7c2c5f061b4