1273154 - Firefox can not correctly rendering 'NARROW NO-BREAK SPACE' (U+202F) in page title, tag title and some pages.

Status: RESOLVED FIXED

(Core :: Graphics: Text, defect)

Description

[siqinbilige]

Attachment: correct_mongolian_words.png

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0
Build ID: 20160502172042

Steps to reproduce:

1. access google site https://www.google.co.jp
2. search mongolian words "ᠳᠠᠷᠢᠬᠢᠨ ᠤ ᠠᠵᠢᠯᠯᠠᠭ᠎ᠠ".
[1833][1820][1837][1822][182C][1822][1828][202F][1824][0020]
[1820][1835[1822][182F][182F][1820][182D]᠎[180E][1820]"



Actual results:

Firefox can not correctly rendering NNBSP(202F) in page title, tag title and some pages.
Windows7_Firefox_46.png
Ubuntu_1404_Firefox_41.png



Expected results:

The NNBSP(202F) in mongolian words rendering correctly anywhere like correct_mongolian_words.png

Comment 1

[siqinbilige]

Attachment: Ubuntu_1404_Firefox_41.png

Comment 2

[siqinbilige]

Attachment: Windows7_Firefox_46.png

Comment 3

[siqinbilige]

In the google search result page, the NNBSP(202F) may be replaced by white space ( by google search engine ) ?

Comment 4

[Jonathan Kew [:jfkthame]]

The problem arises when the content is not explicitly styled with the appropriate Mongolian font, but is relying on font fallback to find it for the Mongolian characters. That generally works OK; but there's a strong possibility that U+202F will be found in some other font from the preferences, before we get to system fallback and find the Mongolian font. And that means it will interrupt the Mongolian text run and get shaped separately.

To fix this, I think we should treat NNBSP similarly to cluster-extender characters in gfxFontGroup::FindFontForChar, so that where possible, we propagate the font selection from the previous character. That will avoid breaking the run here, and permit proper shaping.

Comment 5

[Jonathan Kew [:jfkthame]]

Attachment: Avoid breaking a font run at U+202F (NNBSP) if possible, because Mongolian shaping depends on the text run continuing across it

This fixes the Mongolian shaping failure, provided the system has a suitable font available (Mongolian Baiti on modern Windows works, for example). Difficult to reftest this in a 'universal' way because of the dependency on available fonts, but we could have a Windows-only reftest, at least; I'll put something together.

Comment 6

[Jeff Muizelaar [:jrmuizel]]

Comment on attachment 8753830 [details] [diff] [review]
Avoid breaking a font run at U+202F (NNBSP) if possible, because Mongolian shaping depends on the text run continuing across it

Review of attachment 8753830 [details] [diff] [review]:
-----------------------------------------------------------------

Sounds reasonable. A Windows only ref test would be fine.

Comment 7

[Jonathan Kew [:jfkthame]]

Attachment: Reftest for Mongolian shaping across NNBSP

Here's a reftest that should cover this on Windows. Without the patch, the NNBSP will be assigned Times New Roman in the testcase, and thus break the Mongolian text run; but with the patch we get correct rendering, as if the Mongolian font were explicitly requested.

Comment 8

[siqinbilige]

For your reference, there are two case about usage of 202F in mongolian.
1. the word start with 202F.
2. the word include 202F.
(there is no word end with 202F in mongolian)

like 202F_usage_in_mongolian.png

Comment 9

[siqinbilige]

Attachment: 202F_usage_in_mongolian.png

Comment 10

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/07c833b5d64d97aeda5de0223a1010e429acda56
Bug 1273154 - Avoid breaking a font run at U+202F (NNBSP) if possible, because Mongolian shaping depends on the text run continuing across it. r=jrmuizel

https://hg.mozilla.org/integration/mozilla-inbound/rev/97aef1a40d0f7fdc00fa54e600590d176820a42c
Bug 1273154 - Reftest for Mongolian shaping across NNBSP. r=jrmuizel

Comment 11

[Jonathan Kew [:jfkthame]]

(In reply to siqinbilige from comment #8)
> For your reference, there are two case about usage of 202F in mongolian.
> 1. the word start with 202F.
> 2. the word include 202F.
> (there is no word end with 202F in mongolian)

Interesting, thank you. I wasn't aware it could occur at the very beginning of the word. In the case where a word _starts_ with U+202F, the fix here will not help; to ensure correct display, it will still be necessary to ensure the text is styled with a suitable Mongolian font.

I'll think about whether we can do something further to improve this.

Comment 12

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

Backed out for reftest failure in 1273154-1.html on Windows 8:

https://hg.mozilla.org/integration/mozilla-inbound/rev/cfe3bbf2dc6beec52c4f5f939987673f97d51619
https://hg.mozilla.org/integration/mozilla-inbound/rev/b8458855ddccef7de155b79523ac1ff6fe142c4a

Push with failures: https://treeherder.mozilla.org/#/jobs?repo=mozilla-inbound&revision=97aef1a40d0f7fdc00fa54e600590d176820a42c
Failure log: https://treeherder.mozilla.org/logviewer.html#?job_id=28205809&repo=mozilla-inbound

Comment 13

[Jonathan Kew [:jfkthame]]

Ugh, we got a line-height discrepancy depending on the order of the fonts. :( OK, we can avoid that by unconditionally forcing a more generous overall line-height on the testcase, so the actual metrics of the first-listed font for the test span doesn't affect it.

Comment 14

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b9481ab085785e98b17ea7a3f41f8059bddb5d32
Bug 1273154 - Avoid breaking a font run at U+202F (NNBSP) if possible, because Mongolian shaping depends on the text run continuing across it. r=jrmuizel

https://hg.mozilla.org/integration/mozilla-inbound/rev/2d7d35f47ccd95d2bd713d3f790d6b8b49e7af21
Bug 1273154 - Reftest for Mongolian shaping across NNBSP. r=jrmuizel

Comment 15

[Jonathan Kew [:jfkthame]]

Attachment: followup - Rework NNBSP handling in FindFontForChar to make word-initial NNBSP adopt the font that will be used for the following letter

Given comment 8, noting that word-initial NNBSP may also occur and need to be shaped with the Mongolian text, I suggest we rework the fix here so as to handle this case as well.

Comment 16

[Jonathan Kew [:jfkthame]]

Attachment: Reftest for Mongolian shaping with word-initial NNBSP

And this adds an extra reftest to cover the word-initial case.

Comment 17

[Carsten Book [:Tomcat]]

https://hg.mozilla.org/mozilla-central/rev/b9481ab08578
https://hg.mozilla.org/mozilla-central/rev/2d7d35f47ccd

Comment 18

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/f24ab718413961570a3c370396f7f297920229f7
Bug 1273154 followup - Rework NNBSP handling in FindFontForChar to make word-initial NNBSP adopt the font that will be used for the following letter. r=jrmuizel

https://hg.mozilla.org/integration/mozilla-inbound/rev/51b1f2343ad9544921e68859dfbe9d3c79b00951
Bug 1273154 - Reftest for Mongolian shaping with word-initial NNBSP. r=jrmuizel

Comment 19

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/b59da510fb4ee028d01be2f56f8645787568e06a
Backout 51b1f2343ad9 and f24ab7184139 (bug 1273154 followups) for reftest crash on linux

Comment 20

[Jonathan Kew [:jfkthame]]

https://hg.mozilla.org/integration/mozilla-inbound/rev/2dc65c74bfa6c40479deb070b912b7bd2d151550
Bug 1273154 - Re-land followup to handle word-initial NNBSP shaping in Mongolian, with its reftest. r=jrmuizel

Comment 21

[Wes Kocher (:KWierso) (Not reading bugmail; email directly if needed)]

https://hg.mozilla.org/mozilla-central/rev/2dc65c74bfa6