Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b5pre) Gecko/2008031713 Minefield/3.0b5pre To reproduce, load one of: data:text/html;charset=utf-8,<p>�</p> data:text/html;charset=utf-8,<p>%ee</p> Result: A hexbox containing the digits 'FFFD'. Expected: A black diamond containing a white question mark. This happens with most fonts, but not all. "font: -moz-field;" doesn't have this problem, but I don't know what font that is. Hexboxes (added in bug 372629) are appropriate for "this is a Telegu character and I don't know how to render it", but I expect question-mark-in-diamond for "the page is marked as UTF-8 but contains lone bytes with the high bit set".
This was added by the fix for bug 396732, we explicitly add U+FFFD to a list of codepoints that have no glyph, thus causing the hexbox. http://mxr.mozilla.org/mozilla/source/gfx/thebes/src/gfxQuartzFontCache.mm#793 This was added purely to improve performance of the binary-as-text situation, where lots of FFFD's show up. If we don't blacklist this, we spend a lot of time in the font matching code searching for this glyph. Definitely would rather have the diamond character, those hexboxes are ugly as sin. We could special case this within the font matching code to choose a font explicitly, since Lucida Grande is one of the few fonts that has a glyph for this and it's a system font, it's always available.
I don't think this should block. IM'd jdaggett and he seems to agree. Will take a patch.
Created attachment 313555 [details] [diff] [review] patch, v.0.1, hardwire Lucida Grande for U+FFFD Rather than a hexbox, in the system font fallback case use Lucida Grande explicitly. This font is a system font, guaranteed to be around, and is one of the few fonts that contain a glyph for the undefined character codepoint U+FFFD. In situations like displaying binary-as-text or pages with lots of encoding errors this helps us avoid a lot of time in system font fallback but still displays a glyph rather than just a hexbox.
Comment on attachment 313555 [details] [diff] [review] patch, v.0.1, hardwire Lucida Grande for U+FFFD Looks good to me
What about Windows? I see these FFFD hexboxes from time to time Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9pre) Gecko/2008041206 Minefield/3.0pre ID:2008041206
Egor, this bug ended up with a Mac-specific patch, so please file a new bug (with a testcase) for maybe doing something similar on Windows.
Filed bug 428773