Closed Bug 560206 Opened 15 years ago Closed 15 years ago

Regression: some common unicode arrow characters/entities are now not rendered

Categories

(Core :: Layout: Text and Fonts, defect)

Product:
Core
Component:
Layout: Text and Fonts
Platform:
x86
macOS
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED INVALID

People

(Reporter: stephen_purcell, Unassigned)

References

(
URL
)

Details

User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.9 Safari/533.2 Build Identifier: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-GB; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3 Since upgrading from 3.5 to 3.6, certain common arrow characters are now left un-rendered. Examples include entities such as (but not limited to) → and ←, and the equivalent literal unicode characters. In Firefox 3.6.3 on OS X I see blanks in the 0-5 columns of the U+219x row here: http://en.wikipedia.org/wiki/Template:Unicode_chart_Arrows (Although Firebug's inspector reveals arrow characters in columns 2 and 3.) I originally came to notice this problem when the following commonly-used literal arrow character was not displayed on one of my sites: '►'. Via Adobe's browserlab, I've determined that both Firefox 3.0 and 3.5 render that particular character correctly on both Windows and OS X. I haven't been able to test this issue with 3.6.x on any other platforms. Reproducible: Always
CC-ing myself on this.
John, any idea what's going on here (or what to log to get more information)?
Also, is 3.6 still using ATSUI on 10.6, or does it use CoreText?
This sounds like bug 537444, which the reporter closed as working in 3.6.3, but I'm not aware that we really fixed anything related. So I think it's probably an erratic issue, dependent on available fonts, caches, or something like that. Maybe we can add some font-logging code to figure out what's going on here. It would be good to know what fonts end up getting chosen for those characters (as they may not be in the primary font of the page, so fallback kicks in).
FWIW, the wikipedia chart renders fine for me in 3.6.3 on OS X 10.6.2. But I was never able to reproduce bug 537444, either.
(In reply to comment #3) > Also, is 3.6 still using ATSUI on 10.6, or does it use CoreText? ATSUI, because it still runs on 10.4. (There's Core Text code in the tree, IIRC, but it's not actually present in our builds because we use the 10.4 SDK.)
FWIW, my install 3.6.3 came from an older version, + successive updates. I've just installed 3.6.3 from scratch on another household Mac (10.6.3), and that seems to work correctly (at least for the bold arrow char). Jonathan's suspicion of an erratic issue therefore seems to be supported. If you need further info from me, just post instructions here and I'll get back to you ASAP with my results.
I tried quitting FF, then removing the preferences .plist, ~/Library/Caches/Firefox and ~/Library/Application Support/Firefox. The characters remain hidden.
Could you try a "safe mode" boot with the problem machine? (Hold down the Shift key when the startup chime happens, until the screen indicates it is booting in safe mode.) If that makes a difference, it would indicate a possible issue with the system's (not Firefox's) font caches.
In safe mode, the chars display correctly. Having booted back into normal mode now, the arrows chars are once again missing. I can try cleaning my font cache with Onyx, but I'll hold off in case there's relevant info that would be lost.
Great, thanks - if you can live with the problem for a little while, I'll see if I can come up with some ideas to get more detail of what's going on.
Sure. Note also that as I write this, the chars in question are rendered correctly in Safari & Chrome on the same machine.
I've created a build that should provide a little extra information for these arrow characters, to tell us what font(s) it's using. Please download the Mac OS X .dmg from https://build.mozilla.org/tryserver-builds/jkew@mozilla.com-try-6521ed788573/ and try the version there on your "problem" machine. (It's called Namoroka rather than Firefox, as it's not an official branded version. You can run it directly from the mounted disk image.) When you view that Wikipedia page, you should get messages showing up in the Console Messages panel of Console.app (see /Applications/Utilities) reporting any font fallback choices for the U+219x arrows, and for U+25BA (the "bold arrow" on your page). Something like this: 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2190: found TimesNewRomanPSMT 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2191: found TimesNewRomanPSMT 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2192: found ZapfDingbatsITC 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2193: found TimesNewRomanPSMT 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2194: found ZapfDingbatsITC 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2195: found ZapfDingbatsITC 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2196: found SimSun 19/04/2010 19:09:48 [0x0-0x177177].org.mozilla.firefox fallback for U+2197: found SimSun (from my machine; your results may of course differ). Please let me know what your system reports, both for a "normal" boot where the arrows don't display and for a "safe" boot where they display correctly.
Here are the results for the wikipedia page. 1. Regular boot, incorrect display: Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2190: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2191: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2192: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2193: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2194: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2195: found loserboiallstar Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2196: found STSong Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2197: found STSong Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2198: found STSong Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+2199: found STSong Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+219a: found Menlo-Regular Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+219b: found Menlo-Regular Apr 19 20:51:19 mandala [0x0-0xd70d7].org.mozilla.firefox[0]: fallback for U+219c: found Menlo-Regular 2. Safe boot, correct display Apr 19 21:05:59 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2190: found TimesNewRomanPSMT Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2191: found TimesNewRomanPSMT Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2192: found ZapfDingbatsITC Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2193: found TimesNewRomanPSMT Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2194: found ZapfDingbatsITC Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2195: found ZapfDingbatsITC Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2196: found STSong Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2197: found STSong Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2198: found STSong Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+2199: found STSong Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219a: found Menlo-Regular Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219b: found Menlo-Regular Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219c: found Menlo-Regular Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219d: found Menlo-Regular Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219e: found Menlo-Regular Apr 19 21:06:00 mandala [0x0-0xe00e].org.mozilla.firefox[218]: fallback for U+219f: found Menlo-Regular Additionally, for the filled right arrow mentioned earlier, and which is used prominently on http://www.looktothestars.org/, the regular-boot results in the following similarly incorrect fallback: Apr 19 21:15:51 mandala [0x0-0x27027].org.mozilla.firefox[288]: fallback for U+25ba: found loserboiallstar The problem fallback font in this case is the following free TrueType font, one of many I downloaded at some point into my user fonts library: http://maxfonts.com/fonts/l/loserboi-allstar.font Many of these decorative fonts contain only the most commonly used characters, and I've no idea what features the font may have which might prompt its use as a fallback. I presume that in a safe boot, the user's own font library is not activated.
Yes, in a safe boot your personal fonts are ignored, I believe. I downloaded the font you mention, and looked at it with FontForge, and also dumped it to a text file with ttx. Both of these methods of inspection confirm the same thing: the font claims to support a large number of characters, including these arrows and many other symbols, accented letters, etc. (i.e., they're included in the 'cmap' character-to-glyph mapping table), but aside from the basic Latin alphabet, digits, and a bit of punctuation, the glyphs are actually blank. So basically it's a lousy font. It claims to support about 650 characters, of which roughly 80 are really there and the rest are all blanks.
By the way, if you download Firefox 3.5.9 (see http://www.mozilla.com/en-US/firefox/all-older.html) and run it on that same machine and user account, with the bad font present, do the arrows show up? If so, that would indicate a change in how font fallback is behaving, which I'd be interested to look into.
(In reply to comment #14) > Many of these decorative fonts contain only the most commonly used characters, Which is fine, provided they don't CLAIM to support lots of others! > and I've no idea what features the font may have which might prompt its use as > a fallback. Firefox decides whether a font is usable for fallback on a character-by-character basis, depending on the characters present in the 'cmap' table of the fonts. It can't really do much about it if a font "supports" the character in question, but has the wrong glyph image (whether a complete blank, as in this case, or a totally unrelated shape).
(In reply to comment #16) > By the way, if you download Firefox 3.5.9 (see > http://www.mozilla.com/en-US/firefox/all-older.html) and run it on that same > machine and user account, with the bad font present, do the arrows show up? If > so, that would indicate a change in how font fallback is behaving, which I'd be > interested to look into. The wikipedia arrows page does not work for me in 3.5.9, but the other block arrow character works fine -- since it was that character which made me notice the problem, I guess that explains why I remembered everything being fine up to the 3.6 release.
I'm wondering to myself if it's possible to distinguish system-installed fonts from user-installed fonts, since for symbols such as these (and possibly any character not available in the current font), it might be preferable to initially fall back to the system-wide fonts, since that would likely produce the most consistent results for any given page.
I was wondering something similar. We could either try to determine at runtime which fonts come from the system directories (could be expensive?), or we could have a predefined list of known "standard" fonts on the platform, and try those first when fallback is needed. Actually, although the first alternative could work on OS X, it'd be less useful on Windows, where all fonts are typically installed into c:\windows\fonts so there's no easy way to distinguish "system" and "user" fonts.
Closing this as INVALID, as it's not really a regression or even arguably a Mozilla bug, but a font error. (It was an interesting discussion/investigation, though - thanks for your testing, Steve!) I've filed bug 560472 to consider prioritizing standard system fonts for fallback; I think that could be a worthwhile enhancement that would make such problems less likely to occur.
Status: UNCONFIRMED → RESOLVED
Closed: 15 years ago
Resolution: --- → INVALID
Fair enough. I guess I'll have to delete the font, and hope for the best. Trial and error with the test build you provided would eventually let me weed out all the bad fonts; there's no other practical way for an end user to fix the issue, short of going back to 3.5. I like the idea of predefined initial fallback fonts on a per-platform basis, and so I'm happy to see the follow-up bug filed. I'm more of a developer than a designer, and "real" web designers who care most about how their pages are rendered are also the most likely to have bad fonts installed, so I guess this problem will manifest itself disproportionately frequently for the most picky users.
(In reply to comment #22) > Fair enough. I guess I'll have to delete the font, and hope for the best. Trial > and error with the test build you provided would eventually let me weed out all > the bad fonts; there's no other practical way for an end user to fix the issue, > short of going back to 3.5. I'm afraid so. (Note that that test build only reports fallbacks for the very limited range of arrow characters we were interested in. Reporting ALL fallbacks could lead to huge amounts of console spew in some cases - Wikipedia pages, with their links to other-language entries, are an example where font fallback can be very frequent.) > I'm more of a developer than a designer, and "real" web designers who care most > about how their pages are rendered are also the most likely to have bad fonts > installed, so I guess this problem will manifest itself disproportionately > frequently for the most picky users. Perhaps so - although I'd like to suppose that "real" web designers should be using "real" fonts from professional-quality sources that are unlikely to have such problems. Unfortunately many of the free or cheap fonts out there - not all, of course - are poorly built and can lead to a variety of problems. Designers who care strongly about how their pages are rendered should also be explicitly specifying appropriate fonts for the characters they use, rather than relying on fallback to pick something and hoping for the best. Still, I agree that it's an issue we should try to address as best we can.
+1 on all counts. (**** fonts, allow me to introduce you to Mr. Trashcan...) Thanks for your patience and effort in tracking this down.
You need to log in before you can comment on or make changes to this bug.