193094 - Incorrect characters displayed in text

Status: VERIFIED WORKSFORME

(Core :: Internationalization, defect)

Description

[Nigel Jewell]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3b) Gecko/20030211
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.3b) Gecko/20030211

When I view certain pages (such as that provided) I get strange characters
inserted into the text.

Reproducible: Sometimes

Steps to Reproduce:

Comment 1

[Nigel Jewell]

Attachment: Example of the characters displayed.

Comment 2

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Those are 0xFFFD glyphs, which indicate decoding errors, usually.

Comment 3

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

When you're viewing the page, what does the "View" -> "Character Coding" menu
show as selected?  And what does right-click -> "View Page Info" show as the
"Encoding"?

Comment 4

[Nigel Jewell]

View->Character Encoding = Western (ISO-8859-15)
PageInfo->Encoding = ISO-8859-15

Comment 5

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Ah, I think the problem is that the page in question is really Windows-1252,
which we use as the default, since we treat ISO-8859-1 as Windows-1252 (which is
a superset of it).  But we don't do that for ISO-8859-15.

(Did you change some default (such as Edit -> Preferences -> Languages ->
Default Character Coding) from ISO-8859-1 to ISO-8859-15?

In any case, ->i18n

Comment 6

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

(Note that I think we should explicitly call the default Windows-1252, and treat
anything that calls itself ISO-8859-1 as it should be treated.)

Comment 7

[Nigel Jewell]

Yeah, you are right, I had changed the encoding from ISO-8859-1 to ISO-8859-15.
 Changing it back (and restarting Mozzy) solves the problem for that page ...

Comment 8

[Simon Montagu :smontagu]

Resolving WORKSFORME per last comment. I think comment 6 is a fine suggestion,
and would make our encoding quirks model more consistent with our layout quirks
model, but I have no idea what compatibility problems it would create.

Comment 9

[Yuying Long]

Mark as verified.  Since this page doesn't has charset meta-tag, you can also
try to select auto-detect Universal to get page display correct.