416475 - non-deterministic font selection for Latin words in non-Latin text

Status: RESOLVED FIXED

(Core :: Graphics, defect, P2)

Description

[Karl Tomlinson (:karlt)]

Attachment: testcase

pango_itemize selects a best font for Latin text but CreateGlyphRunsFast just uses the font for the document language.

It seems that the path used to select the font for the word cache may be inconsistent.

On first load of the testcase, the Latin font is used.  On reload the Japanese font is used.  After an inactive period repainting uses the Latin font again.

font.name-list.*.ja should be empty and font.name.*.ja should be set to fontconfig generic names to reproduce, which will be the default after attachment 299736 [details] [diff] [review] is backed out due to bug 416068.

Comment 1

[Karl Tomlinson (:karlt)]

Attachment: snapshot on first load

Comment 2

[Karl Tomlinson (:karlt)]

Attachment: snapshot after reload

Comment 3

[Karl Tomlinson (:karlt)]

Attachment: only use fast path if we think it will use the same font as the itemizing path

I'm assuming that for non-western languages most of the text is positioned with pango_itemize, so I'm hoping that putting the Latin words through this path will not affect page load time too much.  I couldn't detect any adverse affect when measuring load time on some Japanese and Chinese pages.

Comment 4

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

I think that we should not check the lang value for this bug. You should not believe it. It might be often non-suitable value.

Comment 5

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 302251 [details] [diff] [review]
only use fast path if we think it will use the same font as the itemizing path

We don't really want to do it this way. We need a general fix that ensures textrun construction doesn't depend on context across space boundaries.

Comment 6

[Karl Tomlinson (:karlt)]

(In reply to comment #4)
> It might be often non-suitable value.

wrt getting consistent behavior between the fast and itemizing paths, it actually doesn't matter whether the language is correct.  The check just tests (in a conservative manner) whether the itemizing and fast paths will choose the same font with the provided language.

With an appropriate fix for bug 416725 that chooses fast or itemizing based on individual words, we won't need this patch for deterministic behavior.
The patch does however provide consistent behavior between fast and itemizing paths.

If the langGroup is incorrect (e.g. a user with a non-Western locale views a Latin page that doesn't specify locale), then that is perhaps a good reason why the characters should be analysed to determine script, perhaps by using pango_itemize.

Another approach to consider if we want consistent behavior between itemizing and fast paths is whether we might want to choose fonts for LTR 8-bit words always using the x-western langGroup.  IIUC that would give the predominantly preferred behavior for Latin text in CJK pages?

This approach is probably not really the right thing to do for Cyrillic and Greek pages though as it would be best if the same (Cyrillic or Greek) font were used for punctuation and numerals (I'm assuming these scripts use similar code points for punctuation and numerals.)  The effect might not be noticed often though as DejaVu fonts support Latin, Cyrillic and Greek.

Comment 7

[Masayuki Nakano [:masayuki] (he/him)(JST, +0900)]

(In reply to comment #6)
> (In reply to comment #4)
> > It might be often non-suitable value.
> 
> wrt getting consistent behavior between the fast and itemizing paths, it
> actually doesn't matter whether the language is correct.  The check just tests
> (in a conservative manner) whether the itemizing and fast paths will choose the
> same font with the provided language.

O.K.

> With an appropriate fix for bug 416725 that chooses fast or itemizing based on
> individual words, we won't need this patch for deterministic behavior.
> The patch does however provide consistent behavior between fast and itemizing
> paths.

I'm not sure you're right.

E.g., An Japanese paragraph has Chinese text and the paragraph is specified a Japanese font. And the Chinese text has non-Japanese Character. In this case, the characters which can be rendered by Japanese font should be rendered by Japanese font. And remains characters should be rendered Chinese font.

However, maybe, pango decides the language of the words which has Chinese characters is Chinese and rendering with Chinese font.

> If the langGroup is incorrect (e.g. a user with a non-Western locale views a
> Latin page that doesn't specify locale), then that is perhaps a good reason why
> the characters should be analysed to determine script, perhaps by using
> pango_itemize.

Yeah.

> Another approach to consider if we want consistent behavior between itemizing
> and fast paths is whether we might want to choose fonts for LTR 8-bit words
> always using the x-western langGroup.  IIUC that would give the predominantly
> preferred behavior for Latin text in CJK pages?
>
> This approach is probably not really the right thing to do for Cyrillic and
> Greek pages though as it would be best if the same (Cyrillic or Greek) font
> were used for punctuation and numerals (I'm assuming these scripts use similar
> code points for punctuation and numerals.)  The effect might not be noticed
> often though as DejaVu fonts support Latin, Cyrillic and Greek.

This is wrong approach as you think so.

Comment 8

[Damon Sicore (:damons)]

We wouldn't hold the release for this.  wanted1.9.0.x+

Comment 9

[Karl Tomlinson (:karlt)]

The patch in bug 416725 fixes this.