If you think a bug might affect users in the 57 release, please set the correct tracking and status flags for Release Management.

U+FFFD replacement character is rendered as a hexbox instead of a question mark in a diamond

RESOLVED FIXED

Status

()

Core
Graphics
P4
normal
RESOLVED FIXED
10 years ago
9 years ago

People

(Reporter: Jesse Ruderman, Assigned: jtd)

Tracking

({regression, testcase})

Trunk
x86
Mac OS X
regression, testcase
Points:
---
Bug Flags:
blocking1.9 -

Firefox Tracking Flags

(Not tracked)

Details

(URL)

Attachments

(1 attachment)

(Reporter)

Description

10 years ago
Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.4; en-US; rv:1.9b5pre) Gecko/2008031713 Minefield/3.0b5pre

To reproduce, load one of:
  data:text/html;charset=utf-8,<p>&#xfffd;</p>
  data:text/html;charset=utf-8,<p>%ee</p>

Result: A hexbox containing the digits 'FFFD'.

Expected: A black diamond containing a white question mark.

This happens with most fonts, but not all.  "font: -moz-field;" doesn't have this problem, but I don't know what font that is.

Hexboxes (added in bug 372629) are appropriate for "this is a Telegu character and I don't know how to render it", but I expect question-mark-in-diamond for "the page is marked as UTF-8 but contains lone bytes with the high bit set".
Flags: blocking1.9?
(Assignee)

Comment 1

10 years ago
This was added by the fix for bug 396732, we explicitly add U+FFFD to a list of codepoints that have no glyph, thus causing the hexbox.

http://mxr.mozilla.org/mozilla/source/gfx/thebes/src/gfxQuartzFontCache.mm#793

This was added purely to improve performance of the binary-as-text situation, where lots of FFFD's show up.  If we don't blacklist this, we spend a lot of time in the font matching code searching for this glyph.

Definitely would rather have the diamond character, those hexboxes are ugly as sin.  We could special case this within the font matching code to choose a font explicitly, since Lucida Grande is one of the few fonts that has a glyph for this and it's a system font, it's always available.

(Assignee)

Updated

10 years ago
Assignee: nobody → jdaggett
(Reporter)

Updated

10 years ago
Blocks: 396732
I don't think this should block.  IM'd jdaggett and he seems to agree.  Will take a patch.
Flags: wanted1.9.0.x+
Flags: blocking1.9?
Flags: blocking1.9-
(Assignee)

Updated

10 years ago
Priority: -- → P4
(Assignee)

Comment 3

10 years ago
Created attachment 313555 [details] [diff] [review]
patch, v.0.1, hardwire Lucida Grande for U+FFFD

Rather than a hexbox, in the system font fallback case use Lucida Grande explicitly.  This font is a system font, guaranteed to be around, and is one of the few fonts that contain a glyph for the undefined character codepoint U+FFFD.  In situations like displaying binary-as-text or pages with lots of encoding errors this helps us avoid a lot of time in system font fallback but still displays a glyph rather than just a hexbox.
Attachment #313555 - Flags: superreview?(vladimir)
Attachment #313555 - Flags: review?(vladimir)
Comment on attachment 313555 [details] [diff] [review]
patch, v.0.1, hardwire Lucida Grande for U+FFFD

Looks good to me
Attachment #313555 - Flags: superreview?(vladimir)
Attachment #313555 - Flags: superreview+
Attachment #313555 - Flags: review?(vladimir)
Attachment #313555 - Flags: review+
(Assignee)

Comment 5

10 years ago
checked in
Status: NEW → RESOLVED
Last Resolved: 10 years ago
Resolution: --- → FIXED

Comment 6

10 years ago
What about Windows? I see these FFFD hexboxes from time to time

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9pre) Gecko/2008041206 Minefield/3.0pre ID:2008041206
(Reporter)

Comment 7

10 years ago
Egor, this bug ended up with a Mac-specific patch, so please file a new bug (with a testcase) for maybe doing something similar on Windows.

Comment 8

10 years ago
Filed bug 428773
Flags: wanted1.9.0.x+
You need to log in before you can comment on or make changes to this bug.