316111 - Apostrophes appear as � (black diamond with white question mark)

Status: RESOLVED WORKSFORME

(Core :: Internationalization, defect)

Description

[Ken Rohrer]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20051106 SeaMonkey/1.5a
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20051106 SeaMonkey/1.5a

In the browser, apostrophes appear as � (black diamond with white question mark) even if the apostrophes are text-based. They also appear when I view the source code from the browser. When looking at the html in a text app, the apostrophes appear as a clear box (as if a line break), so it may be that they were made with Word apostrophe's. However, other browsers are able to view them OK.

Reproducible: Always

Steps to Reproduce:
1. Go to specified URL
2. View black diamonds
3. 

Actual Results:  
Apostrophes appear as black diamonds with white question marks.

Expected Results:  
Apostrophes should appear normally

Comment 1

[Ken Rohrer]

Some hyphens also appear this way. (view http://www.godrules.net/evolutioncruncher/CruncherTOC.htm)

Comment 2

[Jesse Ruderman]

This happens if Firefox treats an e.g. ISO-8859-1 page containing (isolated) extended characters as UTF-8.  Each extended character has the high bit set, but in UTF-8, a character by itself with the high bit set is always invalid.

You might have told Firefox to treat pages that don't specify a charset as UTF-8.

Comment 3

[Robert Madison]

This happens to me once in a while, but not every time.  I don't make any changes to my browser's settings that would cause this discrepancy.

Comment 4

[Jesse Ruderman]

Robert: when this happens, what encoding is Firefox using, and what is the correct encoding for the page?  (View > Character Encoding > ...)

Many pages don't specify an encoding, and when Firefox has to guess, it uses some random-seeming information to help it guess.  For example, if you follow a link, I think it uses the charset of the previous page to help it guess.

Comment 5

[tinymammoth]

Same thing happens with foreign characters:

http://french.about.com/library/weekly/bl-neuterpronoun.htm

Comment 6

[Jason Bassford]

No specific bug / patch referenced as the fix.

-> WORKSFORME