default sans-serif Khmer fonts not getting used for non language-tagged Khmer content

VERIFIED FIXED

Status

()

Core
Graphics: Text
--
major
VERIFIED FIXED
3 years ago
2 years ago

People

(Reporter: Mathieu Pellerin, Assigned: jtd)

Tracking

(Depends on: 1 bug, {regression})

41 Branch
Unspecified
Linux
regression
Points:
---
Dependency tree / graph
Bug Flags:
qe-verify +

Firefox Tracking Flags

(firefox40 unaffected, firefox41+ verified)

Details

(URL)

Attachments

(4 attachments)

(Reporter)

Description

3 years ago
Nightly suffered from a significant regressed in the last week or two, whereas the browser does _not_ respect the default system font fallback / substitution (not sure what the exact term is here) when drawing unicode ranges outside of the web page declared font family.

This regression impacts on social media users (twitter, facebook, etc.) who are now seeing their native languages drawn with a non-system default font. 

I suspect it's a regression of bug 1056479, which was committed on May 14, 2015.
Blocks: 1056479
(Reporter)

Comment 1

3 years ago
Created attachment 8606834 [details]
a before (proper) vs now (broken) font substitution screen capture

Updated

3 years ago
Status: NEW → RESOLVED
Last Resolved: 3 years ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1165611
(Reporter)

Comment 3

3 years ago
Ekanan, hmm, it's not obvious to me why this has been resolved as duplicated of a bug which relates to the thickness of a font. 

This bug is about picking the wrong font family altogether, not about applying wrong metrics to a font.

Comment 4

3 years ago
I think it's the same as bug 1165590 that marked as a dupe of bug 1165611.


> This bug is about picking the wrong font family altogether, not about
> applying wrong metrics to a font.

as such, reopening. sorry.
Status: RESOLVED → REOPENED
Resolution: DUPLICATE → ---

Updated

3 years ago
Status: REOPENED → NEW
(Assignee)

Comment 5

3 years ago
Mathieu, which distro are you using?

Could you open up a terminal and enter this:

  fc-match -s ":lang=km" | head -10
(Reporter)

Comment 6

3 years ago
John,

I'm using Ubuntu 15.04.

Here's what the above-stated command returns:
KhmerOS.ttf: "Khmer OS" "Regular"
DejaVuSans.ttf: "DejaVu Sans" "Book"
DejaVuSans-Bold.ttf: "DejaVu Sans" "Bold"
DejaVuSans-Oblique.ttf: "DejaVu Sans" "Oblique"
DejaVuSans-BoldOblique.ttf: "DejaVu Sans" "Bold Oblique"
Verdana.ttf: "Verdana" "Normal"
Arial.ttf: "Arial" "Normal"
n019003l.pfb: "Nimbus Sans L" "Regular"
Waree.ttf: "Waree" "Book"
Laksaman.ttf: "Laksaman" "Regular"

This is correct. The default font for Khmer language on Ubuntu (and other distributions) is "Khmer OS". It was honored by Firefox/Nightly until very recently (I can't download a pre bug 1165590 build right now, but chances are pretty high its the culprit).
[Tracking Requested - why for this release]: Regression
Severity: critical → major
status-firefox40: --- → unaffected
tracking-firefox41: --- → ?
Keywords: regression
Version: Trunk → 41 Branch
(Assignee)

Comment 8

3 years ago
Workaround for now -- flip back to the gfxPangoFontGroup code:

  gfx.font_rendering.fontconfig.fontlist.enabled ==> false
(Assignee)

Updated

3 years ago
Assignee: nobody → jdaggett
See Also: → bug 1165590
font.name.serif.x-khmr etc. are not predefined.

Does this work around the problem?:

Preferences -> Content -> Advanced ->
Beside "Serif:", select something other than "serif" (e.g. Khmer OS), then open the dropdown again and select serif again.
Beside "Sans-serif:", repeat but with "sans-serif".

If that still doesn't help, does setting both to "sans-serif" restore the expected font?
(Reporter)

Comment 10

3 years ago
Karl, it doesn't help.

Toggling gfx.font_rendering.fontconfig.fontlist.enabled to false brings back the default system font. It's a workaround, but certainly not a long term solution :)
(Assignee)

Comment 11

3 years ago
Mathieu, I suspect you have other Khmer fonts on your system and for some reason those are getting picked up.  Could you post the results of running the commands below:

  fc-list ":lang=km"

I just installed Ubuntu 15.04 and everything on the test page ends up in Khmer OS. One thing to note is that the content on that facebook page is not language tagged and the fontlist is "helvetica, arial, lucida grande, sans-serif".

If you send me a list of the Khmer fonts you have installed I can try to reproduce what's going on in your environment.
(Assignee)

Updated

3 years ago
Summary: nightly doesn't respect default system font fallback / substitution for Unicode range outside of used font family → default sans-serif Khmer fonts not getting used for non language-tagged Khmer content
(Assignee)

Updated

3 years ago
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 12

3 years ago
John,

Here's the output for fc-list ":lang=km":

/usr/share/fonts/truetype/ttf-khmeros/KhmerOSsiemreap.ttf: Khmer OS Siemreap:style=Regular
/home/webmaster/.fonts/KHMKAMPONGTRACH.ttf: Khmer Kampongtrach:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmetalchrieng.ttf: Khmer OS Metal Chrieng:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuolpali.ttf: Khmer OS Muol Pali:style=Regular
/usr/share/fonts/truetype/ttf-khmeros-core/KhmerOS.ttf: Khmer OS:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSfreehand.ttf: Khmer OS Freehand:style=Regular
/usr/share/fonts/truetype/ttf-khmeros-core/KhmerOSsys.ttf: Khmer OS System:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuol.ttf: Khmer OS Muol:style=Regular
/home/webmaster/.fonts/KHMWATPHNOM.ttf: Khmer Wat Phnom:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSmuollight.ttf: Khmer OS Muol Light:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSbokor.ttf: Khmer OS Bokor:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSfasthand.ttf: Khmer OS Fasthand:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOSbattambang.ttf: Khmer OS Battambang:style=Regular
/usr/share/fonts/truetype/ttf-khmeros/KhmerOScontent.ttf: Khmer OS Content:style=Regular
/home/webmaster/.fonts/khmer Z2.volt.ttf: Khmer M2:style=Regular

The font that's incorrectly used instead of the default system font is the last one (i.e. "/home/webmaster/.fonts/khmer Z2.volt.ttf: Khmer M2:style=Regular"). I'll attach the font to this bug.
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 13

3 years ago
Created attachment 8607333 [details]
khmer Z2.volt.ttf

Font (located in my .fonts home directory) that is wrongly being used instead of the default system font.
(Reporter)

Comment 14

3 years ago
In terms of ubuntu packages, but fonts-khmeros-core and fonts-khmeros are installed.
(In reply to Karl Tomlinson (ni?:karlt) from comment #9)
> Does this work around the problem?:

Sorry, I missed a step here.
 
> Preferences -> Content -> Advanced ->

Beside "Fonts for:", select "Khmer".

> Beside "Serif:", select something other than "serif" (e.g. Khmer OS), then
> open the dropdown again and select serif again.
> Beside "Sans-serif:", repeat but with "sans-serif".
> 
> If that still doesn't help, does setting both to "sans-serif" restore the
> expected font?

And if neither of those help, then please try with both set to "Khmer OS".

In about:config, you should be able to search for khmr and see new preferences font.name.sans-serif.x-khmr and font.name.serif.x-khmr.
(Reporter)

Comment 16

3 years ago
Karl,

Ok, I had missed the Fonts for "Khmer" bit. 

If I set the serif and sans-serif font for "Khmer" to Khmer OS, Firefox does go back to rendering text using Khmer OS font.

That said, this remains a workaround for a regression that - if left unfixed - will affect a great load of people around the globe :)
(Assignee)

Comment 17

3 years ago
(In reply to Mathieu Pellerin from comment #16)

> That said, this remains a workaround for a regression that - if left unfixed
> - will affect a great load of people around the globe :)

Mathieu, are you seeing problems with pages other than those containing Khmer?

I'm actually not able to reproduce the problem you're seeing, even with Ubuntu 15.04, the khmeros font packages and your attached font installed.

Guessing you've already tried this but just to be sure, have you tried rebuilding font caches?

  sudo fc-cache -fv

The other thing is do you have any local fontconfig conf files?

I'd like to fix this ASAP but I'm still not quite sure what precisely is happening here. Thanks for your patience.
(Reporter)

Comment 18

3 years ago
John, I wouldn't know if pages featuring characters other than Khmer lead to same issue as I wouldn't be used to the default system font for another given language. But, whatever is causing Firefox not to pick the default font for a given language - in this specific case Khmer - I'm pretty sure it'll happen for other languages.

Ok, I've tried sudo fc-cache -fv, and reloaded Firefox. The result is the same, the wrong font is used.

I don't believe I have a local fontconfig conf file. Where would it be located?
Depends on: 1167072
(In reply to Mathieu Pellerin from comment #16)
> If I set the serif and sans-serif font for "Khmer" to Khmer OS, Firefox does
> go back to rendering text using Khmer OS font.
> 
> That said, this remains a workaround for a regression that - if left unfixed
> - will affect a great load of people around the globe :)

Yes, definitely.  But your experiments are helpful in diagnosing the cause, thank you.

I filed bug 1167072 for one issue involved here.

If changing these back to "serif" and "sans-serif" does not provide the correct behaviour, then there is still another bug to diagnose.

I suspect bug 390900 is also involved here.
Depends on: 390900
(Assignee)

Comment 20

3 years ago
Mathieu, could you try out the tryserver build below?

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jdaggett@mozilla.com-674c9657cc2f/

This is a test fix for bug 1167072 which, I think, may be the underlying cause of the problem you're seeing.
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 21

3 years ago
John, progress! :)

It partially fixes things. That is, the system font is now used, when elements have a lang="km" tag attached to it. However, for elements that do not define the language (such as Facebook), Nightly still regresses as it doesn't respect / pick the default system font for a given language.
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 22

3 years ago
Created attachment 8609994 [details]
small test case (the three paragraphs should use the same Khmer font)

John, I've created a small test case which might be of help to your debugging efforts. It's basically a page with three <p> elements, one with a style tag, a lang tag, and no tag at all.

Under Firefox, prior to this regression, all three paragraphs render using the same font. So does Chromium, etc.
(Assignee)

Comment 23

3 years ago
Mathieu, one more request. Could I ask you to run with logging enabled? That results would help me narrow what to investigate.

1. Set up new profile
2. In Preferences > General, switch to 'Show my windows from last time'
3. Navigate to the test facebook URL
4. Quit
5. Run firefox from the command line with logging enabled:

export NSPR_LOG_MODULES=textrun:5
export NSPR_LOG_FILE=textrun.out
/path/to/firefox-bin -P <profilename>

The results will be saved to a file 'textrun.out' (maybe more than one with e10s enabled). If you could attach those here that would be a big help.
(Assignee)

Updated

3 years ago
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 24

3 years ago
Created attachment 8610006 [details]
textrun.out.child-1
Flags: needinfo?(nirvn.asia)
(Assignee)

Comment 25

3 years ago
Mathieu, thanks for attaching the logfile, that's a big help. The problem here is that the pref fonts aren't getting used for Khmer fallback, due to a combination of a couple other bugs. When pref fonts with glyph coverage can't be found, we do system fallback. System fallback simply looks at all fonts that support Khmer. In your case, the last font looked at happens to be one that's very different from what you were expected. The pref font will be better as it's the font that fontconfig suggests for serif/sans-serif/monospace for a given language.

I went ahead and pushed the fixes for those to inbound but here's a trybuild to test with:

http://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/jdaggett@mozilla.com-829cfaed76d9/

Please let me know whether this fixes your problem or not.
(Assignee)

Updated

3 years ago
Flags: needinfo?(nirvn.asia)
(Assignee)

Updated

3 years ago
Depends on: 1164735
(Reporter)

Comment 26

3 years ago
John, yay, this regression is gone when using the latest mozilla-central tinderbox build; Khmer language now renders using the default system font when no Khmer font is specified. Excellent, thanks for the quick response.

Closing as resolved -> fixed.
Status: NEW → RESOLVED
Last Resolved: 3 years ago3 years ago
Flags: needinfo?(nirvn.asia)
Resolution: --- → FIXED
(Assignee)

Comment 27

3 years ago
(In reply to Mathieu Pellerin from comment #26)
> John, yay, this regression is gone when using the latest mozilla-central
> tinderbox build; Khmer language now renders using the default system font
> when no Khmer font is specified. Excellent, thanks for the quick response.

Excellent!! Thanks for all your help tracking this down Mathieu! Sorry for the bother.

Updated

3 years ago
status-firefox41: affected → fixed
John, this bug was marked as "fixed" but I do not see a patch associated. Was this fixed as a side effect of another bug? Should we resolve this as a duplicate in that case?

Adding a tracking flag for QE team can verify this fix.
tracking-firefox41: ? → +
Flags: qe-verify+
Flags: needinfo?(jdaggett)
(Assignee)

Comment 29

3 years ago
It was resolved by changes in bug 1164735 and bug 1167072. So I don't think it's strictly a duplicate. I think it's fine to leave this as "Fixed", similar to bug 1165590.
Flags: needinfo?(jdaggett)
I was unable to reproduce this issue on Firefox 41.0a1 from 2015-05-17 under Ubuntu 14.04 32-bit and Ubuntu 13.10 64-bit.
I’ve only reproduced Bug 1165590 which I confirm that is fixed in Firefox 43.0a1 (2015-08-30).

Mathieu, since you've reproduced this bug, could you please verify if it's now fixed in FIrefox 41 Beta 5 https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/41.0b5-candidates/build1/ ?
Flags: needinfo?(nirvn.asia)
(Reporter)

Comment 31

2 years ago
It's fixed. Apologies for not having commented on this before now.
Flags: needinfo?(nirvn.asia)

Updated

2 years ago
Status: RESOLVED → VERIFIED
status-firefox41: fixed → verified
You need to log in before you can comment on or make changes to this bug.