1165788 - default sans-serif Khmer fonts not getting used for non language-tagged Khmer content

Status: VERIFIED FIXED

(Core :: Graphics: Text, defect)

Description

[Mathieu Pellerin]

Nightly suffered from a significant regressed in the last week or two, whereas the browser does _not_ respect the default system font fallback / substitution (not sure what the exact term is here) when drawing unicode ranges outside of the web page declared font family.

This regression impacts on social media users (twitter, facebook, etc.) who are now seeing their native languages drawn with a non-system default font. 

I suspect it's a regression of bug 1056479, which was committed on May 14, 2015.

Comment 1

[Mathieu Pellerin]

Attachment: a before (proper) vs now (broken) font substitution screen capture

Comment 3

[Mathieu Pellerin]

Ekanan, hmm, it's not obvious to me why this has been resolved as duplicated of a bug which relates to the thickness of a font. 

This bug is about picking the wrong font family altogether, not about applying wrong metrics to a font.

Comment 4

[Ekanan Ketunuti]

I think it's the same as bug 1165590 that marked as a dupe of bug 1165611.


> This bug is about picking the wrong font family altogether, not about
> applying wrong metrics to a font.

as such, reopening. sorry.

Comment 5

[John Daggett (:jtd)]

Mathieu, which distro are you using?

Could you open up a terminal and enter this:

  fc-match -s ":lang=km" | head -10

Comment 6

[Mathieu Pellerin]

John,

I'm using Ubuntu 15.04.

Here's what the above-stated command returns:
KhmerOS.ttf: "Khmer OS" "Regular"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
Verdana.ttf: "Verdana" "Normal"
Arial.ttf: "Arial" "Normal"
n019003l.pfb: "Nimbus Sans L" "Regular"
Waree.ttf: "Waree" "Book"
Laksaman.ttf: "Laksaman" "Regular"

This is correct. The default font for Khmer language on Ubuntu (and other distributions) is "Khmer OS". It was honored by Firefox/Nightly until very recently (I can't download a pre bug 1165590 build right now, but chances are pretty high its the culprit).

Comment 7

[Virtual_ManPL [:Virtual] 🇵🇱 - (please needinfo? me - so I will see your comment/reply/question/etc.)]

[Tracking Requested - why for this release]: Regression

Comment 8

[John Daggett (:jtd)]

Workaround for now -- flip back to the gfxPangoFontGroup code:

  gfx.font_rendering.fontconfig.fontlist.enabled ==> false

Comment 9

[Karl Tomlinson (:karlt back Feb 10)]

font.name.serif.x-khmr etc. are not predefined.

Does this work around the problem?:

Preferences -> Content -> Advanced ->
Beside "Serif:", select something other than "serif" (e.g. Khmer OS), then open the dropdown again and select serif again.
Beside "Sans-serif:", repeat but with "sans-serif".

If that still doesn't help, does setting both to "sans-serif" restore the expected font?

Comment 10

[Mathieu Pellerin]

Karl, it doesn't help.

Toggling gfx.font_rendering.fontconfig.fontlist.enabled to false brings back the default system font. It's a workaround, but certainly not a long term solution :)

Comment 11

[John Daggett (:jtd)]

Mathieu, I suspect you have other Khmer fonts on your system and for some reason those are getting picked up.  Could you post the results of running the commands below:

  fc-list ":lang=km"

I just installed Ubuntu 15.04 and everything on the test page ends up in Khmer OS. One thing to note is that the content on that facebook page is not language tagged and the fontlist is "helvetica, arial, lucida grande, sans-serif".

If you send me a list of the Khmer fonts you have installed I can try to reproduce what's going on in your environment.

Comment 12

[Mathieu Pellerin]

John,

Here's the output for fc-list ":lang=km":

/usr/share/fonts/truetype/ttf-khmeros/KhmerOSsiemreap.ttf: Khmer OS Siemreap:style=Regular
/home/webmaster/.fonts/KHMKAMPONGTRACH.ttf: Khmer Kampongtrach:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmetalchrieng.ttf: Khmer OS Metal Chrieng:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuolpali.ttf: Khmer OS Muol Pali:style=Regular
/usr/share/fonts/truetype/ttf-khmeros-core/KhmerOS.ttf: Khmer OS:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSfreehand.ttf: Khmer OS Freehand:style=Regular
/usr/share/fonts/truetype/ttf-khmeros-core/KhmerOSsys.ttf: Khmer OS System:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuol.ttf: Khmer OS Muol:style=Regular
/home/webmaster/.fonts/KHMWATPHNOM.ttf: Khmer Wat Phnom:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuollight.ttf: Khmer OS Muol Light:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSbokor.ttf: Khmer OS Bokor:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSfasthand.ttf: Khmer OS Fasthand:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSbattambang.ttf: Khmer OS Battambang:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOScontent.ttf: Khmer OS Content:style=Regular
/home/webmaster/.fonts/khmer Z2.volt.ttf: Khmer M2:style=Regular

The font that's incorrectly used instead of the default system font is the last one (i.e. "/home/webmaster/.fonts/khmer Z2.volt.ttf: Khmer M2:style=Regular"). I'll attach the font to this bug.

Comment 13

[Mathieu Pellerin]

Attachment: khmer Z2.volt.ttf

Font (located in my .fonts home directory) that is wrongly being used instead of the default system font.

Comment 14

[Mathieu Pellerin]

In terms of ubuntu packages, but fonts-khmeros-core and fonts-khmeros are installed.

Comment 15

[Karl Tomlinson (:karlt back Feb 10)]

(In reply to Karl Tomlinson (ni?:karlt) from comment #9)
> Does this work around the problem?:

Sorry, I missed a step here.
 
> Preferences -> Content -> Advanced ->

Beside "Fonts for:", select "Khmer".

> Beside "Serif:", select something other than "serif" (e.g. Khmer OS), then
> open the dropdown again and select serif again.
> Beside "Sans-serif:", repeat but with "sans-serif".
> 
> If that still doesn't help, does setting both to "sans-serif" restore the
> expected font?

And if neither of those help, then please try with both set to "Khmer OS".

In about:config, you should be able to search for khmr and see new preferences font.name.sans-serif.x-khmr and font.name.serif.x-khmr.

Comment 16

[Mathieu Pellerin]

Karl,

Ok, I had missed the Fonts for "Khmer" bit. 

If I set the serif and sans-serif font for "Khmer" to Khmer OS, Firefox does go back to rendering text using Khmer OS font.

That said, this remains a workaround for a regression that - if left unfixed - will affect a great load of people around the globe :)

Comment 17

[John Daggett (:jtd)]

(In reply to Mathieu Pellerin from comment #16)

> That said, this remains a workaround for a regression that - if left unfixed
> - will affect a great load of people around the globe :)

Mathieu, are you seeing problems with pages other than those containing Khmer?

I'm actually not able to reproduce the problem you're seeing, even with Ubuntu 15.04, the khmeros font packages and your attached font installed.

Guessing you've already tried this but just to be sure, have you tried rebuilding font caches?

  sudo fc-cache -fv

The other thing is do you have any local fontconfig conf files?

I'd like to fix this ASAP but I'm still not quite sure what precisely is happening here. Thanks for your patience.

Comment 18

[Mathieu Pellerin]

John, I wouldn't know if pages featuring characters other than Khmer lead to same issue as I wouldn't be used to the default system font for another given language. But, whatever is causing Firefox not to pick the default font for a given language - in this specific case Khmer - I'm pretty sure it'll happen for other languages.

Ok, I've tried sudo fc-cache -fv, and reloaded Firefox. The result is the same, the wrong font is used.

I don't believe I have a local fontconfig conf file. Where would it be located?

Comment 19

[Karl Tomlinson (:karlt back Feb 10)]

(In reply to Mathieu Pellerin from comment #16)
> If I set the serif and sans-serif font for "Khmer" to Khmer OS, Firefox does
> go back to rendering text using Khmer OS font.
> 
> That said, this remains a workaround for a regression that - if left unfixed
> - will affect a great load of people around the globe :)

Yes, definitely.  But your experiments are helpful in diagnosing the cause, thank you.

I filed bug 1167072 for one issue involved here.

If changing these back to "serif" and "sans-serif" does not provide the correct behaviour, then there is still another bug to diagnose.

I suspect bug 390900 is also involved here.

Comment 20

[John Daggett (:jtd)]

Mathieu, could you try out the tryserver build below?

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-674c9657cc2f/

This is a test fix for bug 1167072 which, I think, may be the underlying cause of the problem you're seeing.

Comment 21

[Mathieu Pellerin]

John, progress! :)

It partially fixes things. That is, the system font is now used, when elements have a lang="km" tag attached to it. However, for elements that do not define the language (such as Facebook), Nightly still regresses as it doesn't respect / pick the default system font for a given language.

Comment 22

[Mathieu Pellerin]

Attachment: small test case (the three paragraphs should use the same Khmer font)

John, I've created a small test case which might be of help to your debugging efforts. It's basically a page with three <p> elements, one with a style tag, a lang tag, and no tag at all.

Under Firefox, prior to this regression, all three paragraphs render using the same font. So does Chromium, etc.

Comment 23

[John Daggett (:jtd)]

Mathieu, one more request. Could I ask you to run with logging enabled? That results would help me narrow what to investigate.

1. Set up new profile
2. In Preferences > General, switch to 'Show my windows from last time'
3. Navigate to the test facebook URL
4. Quit
5. Run firefox from the command line with logging enabled:

export NSPR_LOG_MODULES=textrun:5
export NSPR_LOG_FILE=textrun.out
/path/to/firefox-bin -P <profilename>

The results will be saved to a file 'textrun.out' (maybe more than one with e10s enabled). If you could attach those here that would be a big help.

Comment 24

[Mathieu Pellerin]

Attachment: textrun.out.child-1

Comment 25

[John Daggett (:jtd)]

Mathieu, thanks for attaching the logfile, that's a big help. The problem here is that the pref fonts aren't getting used for Khmer fallback, due to a combination of a couple other bugs. When pref fonts with glyph coverage can't be found, we do system fallback. System fallback simply looks at all fonts that support Khmer. In your case, the last font looked at happens to be one that's very different from what you were expected. The pref font will be better as it's the font that fontconfig suggests for serif/sans-serif/monospace for a given language.

I went ahead and pushed the fixes for those to inbound but here's a trybuild to test with:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/[e-mail]-829cfaed76d9/

Please let me know whether this fixes your problem or not.

Comment 26

[Mathieu Pellerin]

John, yay, this regression is gone when using the latest mozilla-central tinderbox build; Khmer language now renders using the default system font when no Khmer font is specified. Excellent, thanks for the quick response.

Closing as resolved -> fixed.

Comment 27

[John Daggett (:jtd)]

(In reply to Mathieu Pellerin from comment #26)
> John, yay, this regression is gone when using the latest mozilla-central
> tinderbox build; Khmer language now renders using the default system font
> when no Khmer font is specified. Excellent, thanks for the quick response.

Excellent!! Thanks for all your help tracking this down Mathieu! Sorry for the bother.

Comment 28

[Ritu Kothari (:ritu) (Inactive, please n-i to RyanVM, jcristau, or pascal)]

John, this bug was marked as "fixed" but I do not see a patch associated. Was this fixed as a side effect of another bug? Should we resolve this as a duplicate in that case?

Adding a tracking flag for QE team can verify this fix.

Comment 29

[John Daggett (:jtd)]

It was resolved by changes in bug 1164735 and bug 1167072. So I don't think it's strictly a duplicate. I think it's fine to leave this as "Fixed", similar to bug 1165590.

Comment 30

[Vasilica Tamas, QA [:vtamas]]

I was unable to reproduce this issue on Firefox 41.0a1 from 2015-05-17 under Ubuntu 14.04 32-bit and Ubuntu 13.10 64-bit.
I’ve only reproduced Bug 1165590 which I confirm that is fixed in Firefox 43.0a1 (2015-08-30).

Mathieu, since you've reproduced this bug, could you please verify if it's now fixed in FIrefox 41 Beta 5 https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/41.0b5-candidates/build1/ ?

Comment 31

[Mathieu Pellerin]

It's fixed. Apologies for not having commented on this before now.