1279693 - Unicode characters rendered differently in firefox 47 compared to 46

Status: RESOLVED FIXED

(Core :: Graphics: Text, defect)

Description

[Defias R. Ingleader]

Attachment: dongs.png

User Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:47.0) Gecko/20100101 Firefox/47.0
Build ID: 20160604131506

Steps to reproduce:

Attempt to view a donger: ヽ༼ຈل͜ຈ༽ﾉ


Actual results:

Left side of the face overlaps the eye, right arm overlaps the right side


Expected results:

Donger looks as it did previously, see attachment

Comment 1

[Loic]

regression range:
https://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=d8da9bc56468f58be505e61e72e78bff1d2b4b61&tochange=2b0aa1cffeeaabb524b9d2936321af18c7445fbc


Jonathan Kew — Bug 1249861 - Update harfbuzz to release 1.2.2 from upstream. r=jrmuizel

Is it a valid regression?

Comment 2

[Jonathan Kew [:jfkthame]]

I'm not sure. I suspect it's font-dependent, as I can't seem to reproduce the result in the screenshot (on either my OS X or Win10 systems). It may be a particular Tibetan font (as it's the Tibetan glyphs that seem to be mis-spaced) that doesn't work well with the updated harfbuzz, but to diagnose whether this is a font bug or a code bug, we'll need to identify exactly what fonts are being used in the poor rendering.

Comment 3

[Loic]

Attachment: donger-ff33-47.png

Small other bug I see too: the "mooth" of the donger is not placed under the "nose", but on the left side (with old versions or 47, whatever):

Comment 4

[Jonathan Kew [:jfkthame]]

Loic, when you reproduce this in FF47, what specific fonts are involved? To check, set devtools.fontinspector.enabled to 'true', then use Inspect Element on the element with the text in it, and click the Fonts panel to find out what faces font-fallback has chosen. Thanks!

Comment 5

[Loic]

If I select the entire donger:

1) with devtools.fontinspector.enabled=true or the add-on "fontinfo":
Courier New
Lao UI
MS PGothic
Microsoft Himalaya

2) with the add-on "Font Finder":
Font being rendered: "Courier New"

Comment 6

[Jonathan Kew [:jfkthame]]

OK, thanks. The problematic Tibetan characters will be coming from Microsoft Himalaya. What version of Windows are you on, and what is the version of the Himalaya font? (I think you can find this if you look in Control Panel / Fonts and check the font's Properties window.)

Comment 7

[Loic]

In Win 7 folder %windir%\fonts:

Nom de la police : Microsoft Himalaya
Version : Version 5.00
Disposition OpenType, - signé numériquement, TrueType Contours

Comment 8

[Jonathan Kew [:jfkthame]]

Thanks. I just checked, and confirmed that this is still broken in the Himalaya font shipped with Win8 and Win8.1, but looks to be fixed in Win10.

So it's a font bug, exposed by a harfbuzz change (whereby it relies on the font's GDEF table instead of just Unicode character properties in the Tibetan shaper). A similar problem affects some Latin fonts; see bug 1279925.

Comment 9

[Milan Sreckovic [:milan] (needinfo for best results)]

Is this bug a wontfix, or do we look for a workaround?  I don't think we would try to merge two versions of harfbuzz and runtime switch between the old and the new behaviour.

Comment 10

[Krasnaya Ploshchad’]

(In reply to Milan Sreckovic [:milan] from comment #9)
> Is this bug a wontfix, or do we look for a workaround?  I don't think we
> would try to merge two versions of harfbuzz and runtime switch between the
> old and the new behaviour.

HarfBuzz got resolved in version 1.3.0.

Comment 11

[Ryan VanderMeulen [:RyanVM]]

We updated Harfbuzz to version 1.3.0 in bug 1269343. Can anybody confirm that everything renders as expected now in a current Beta release?

Comment 12

[Loic]

It's not fixed in FF51+, the issue is still remaining.

Comment 13

[Ryan VanderMeulen [:RyanVM]]

Behdad, any ideas what might have broken with harfbuzz here? This is one of a few regressions that go back to the 1.2.2 release (see also: bug 1279875 and bug 1279925).

Comment 14

[Behdad Esfahbod]

(In reply to Ryan VanderMeulen [:RyanVM] from comment #13)
> Behdad, any ideas what might have broken with harfbuzz here? This is one of
> a few regressions that go back to the 1.2.2 release (see also: bug 1279875
> and bug 1279925).

In Windows 10+, Tibetan is routed through Universal Shaping Engine.  So if we change our Tibetan shaping, it will be in that direction.  Which means we are going to respect GDEF.  So, no change here.

We already blacklisted GDEF of two versions of Himalaya from Windows 8 and 8.1:

      /* sha1sum:73da7f025b238a3f737aa1fde22577a6370f77b0  himalaya.ttf from Windows 8 */ 
      || (192 == gdef_len && 7254 == gpos_len && 12638 == gsub_len) 
      /* sha1sum:6e80fd1c0b059bbee49272401583160dc1e6a427  himalaya.ttf from Windows 8.1 */ 
      || (192 == gdef_len && 7254 == gpos_len && 12690 == gsub_len) 

So if this is still happening, one needs to check which font is having the problem and send me the blacklisting data for it.  Otherwise, close as WONTFIX.

Comment 15

[Ryan VanderMeulen [:RyanVM]]

Loic, can you screenshot what you're seeing? On Win10 for me, #c0 looks pretty darn close to the FF33 rendering in attachment 8762566 [details] (vs. the FF47 version).

Comment 16

[Loic]

Attachment: screen-FF50-54.png

Not fixed on my in win7.

Comment 17

[Jonathan Kew [:jfkthame]]

(In reply to Loic from comment #16)
> Not fixed on my in win7.

Not very surprising, given that...

> We already blacklisted GDEF of two versions of Himalaya from Windows 8 and
> 8.1:

...but presumably the version that ships on Win7 is different.

So Loic, can you provide the corresponding details for the Win7 font? Specifically, we need the length of the three OpenType Layout tables (GDEF, GPOS, GSUB), which will be reported (for example) if you have fonttools[1] installed and run

  ttx -l himalaya.ttf

It would be nice to also have the sha1 checksum of the file, to record for documentation purposes, but that's not critical if you don't have a sha1sum tool available.

Comment 18

[Jonathan Kew [:jfkthame]]

[1] https://github.com/fonttools/fonttools

Comment 19

[Loic]

Attachment: himalaya.ttf

himalaya.ttf
SHA1 eb8afadd28e9cf963e886b23a30b44ab4fd83acc 

ttx -l himala
Listing table info for "himalaya.ttf":
    tag     checksum   length   offset
    ----  ----------  -------  -------
    DSIG  0x527D34D0     7000   603104
    GDEF  0x2D122F1A      180   582612
    GPOS  0x57672522     7254   582792
    GSUB  0x15CB2D5C    13054   590048
    LTSH  0xDA09C7D4     1486     6080
    OS/2  0x3F491956       96      472
    cmap  0xD6ECB913      788    31320
    cvt   0x181A072A      224    33984
    fpgm  0x76BD44C4     1571    32108
    gasp  0x00210009       16   582596
    glyf  0x625BDD34   520190    40140
    hdmx  0x5E74D31B    23752     7568
    head  0xE2AD66E5       54      348
    hhea  0x08B207CC       36      404
    hmtx  0x9A0EFAE1     5510      568
    kern  0x061E052E      666   560332
    loca  0x14F8F0DA     5932    34208
    maxp  0x07E90814       32      440
    name  0xD914AD65     2274   561000
    post  0x3A99F5FC    19317   563276
    prep  0x44AEF097      302    33680

Comment 20

[Jonathan Kew [:jfkthame]]

Thanks! I've just pushed a build at https://treeherder.mozilla.org/#/jobs?repo=try&revision=21f67a239d6a30393f85a29ec31412691274bf44 where this version of the font is added to the blacklist. Once that build completes, could you please give it a try and confirm whether it fixes things on Win7 for you?

Comment 21

[Loic]

With https://archive.mozilla.org/pub/firefox/try-builds/[e-mail]-21f67a239d6a30393f85a29ec31412691274bf44/try-win32/firefox-53.0a1.en-US.win32.zip

Rendering is fine: http://i.imgur.com/YjOk9ST.png

Comment 22

[Jonathan Kew [:jfkthame]]

Attachment: Add the Win7 version of himalaya.ttf to the GDEF blacklist in harfbuzz

I've submitted this upstream (https://github.com/behdad/harfbuzz/pull/407), so eventually it'll come back to us as part of a harfbuzz update, but in the meantime I think we should just apply the patch locally.

Comment 23

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/a467f9800f8dc0b48dca0e9a4121b43681cab71a
Bug 1279693 - Add the Win7 version of himalaya.ttf to the GDEF blacklist in harfbuzz. r=jrmuizel

Comment 24

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/a467f9800f8d

Comment 25

[Loic]

Jonathan, is there a reason why Microsoft doesn't push an update on Win7/8 to fix such an issue with bad fonts like that?

Comment 26

[Jonathan Kew [:jfkthame]]

Stability. I wouldn't expect them to push an update to older systems unless it was a serious security vulnerability or something like that.

Comment 27

[Ryan VanderMeulen [:RyanVM]]

Seems like a pretty trivial fix for backporting. Please request Aurora/Beta approval on it when you get a chance :)

Comment 28

[Jonathan Kew [:jfkthame]]

Comment on attachment 8830718 [details] [diff] [review]
Add the Win7 version of himalaya.ttf to the GDEF blacklist in harfbuzz

Approval Request Comment
[Feature/Bug causing the regression]: some glyphs from the Tibetan font on Win7 render poorly with recent harfbuzz, due to buggy font tables

[User impact if declined]: bad spacing/positioning of some characters in the Himalaya font shipped on Win7 

[Is this code covered by automated tests?]: no

[Has the fix been verified in Nightly?]: yes, comment 21

[Needs manual test from QE? If yes, steps to reproduce]: no

[List of other uplifts needed for the feature/fix]: none

[Is the change risky?]: no

[Why is the change risky/not risky?]: trivial patch, just adding another font version to a harfbuzz blacklist

[String changes made/needed]: none

Comment 29

[Julien Cristau [:jcristau]]

Comment on attachment 8830718 [details] [diff] [review]
Add the Win7 version of himalaya.ttf to the GDEF blacklist in harfbuzz

fix font rendering issue on win7, let's get this in aurora53 and beta52.

Comment 30

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-aurora/rev/06e8f4459eb4

Comment 31

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/releases/mozilla-beta/rev/0f53157d8586