Bengali font vowel incorrect on mac osx lion

RESOLVED FIXED in Firefox 39

Status

()

Core
Layout: Text
RESOLVED FIXED
7 years ago
3 years ago

People

(Reporter: swagat, Assigned: jfkthame)

Tracking

(Blocks: 1 bug)

Trunk
mozilla39
x86
Mac OS X
Points:
---
Dependency tree / graph
Bug Flags:
in-testsuite +

Firefox Tracking Flags

(firefox39 fixed)

Details

(URL)

Attachments

(4 attachments, 1 obsolete attachment)

(Reporter)

Description

7 years ago
Created attachment 559780 [details]
Screen Shot 2011-09-11 at 9.48.24 PM.png

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Build ID: 20110902133214

Steps to reproduce:

using mac os x lion which has support for bengali fonts, chrome and safari display bengali fonts correctly. but firefox does not .


Actual results:

best described in screenshot


Expected results:

best described in screenshot
(Reporter)

Comment 1

7 years ago
Created attachment 559786 [details]
the correct rendering
(Reporter)

Updated

7 years ago

Updated

7 years ago
Component: General → Layout: Text
Product: Firefox → Core
QA Contact: general → layout.fonts-and-text
(Assignee)

Comment 2

7 years ago
Please confirm what font is being used for the Bengali text in this example. You can use the fontinfo addon[1] to determine this.

[1] https://addons.mozilla.org/en-US/firefox/addon/fontinfo/
(Reporter)

Comment 3

7 years ago
the font being used is Bangla Sangam MN.
(Reporter)

Comment 4

7 years ago
bug is in firefox 7 branch too.
(Assignee)

Comment 5

7 years ago
Note that the "correct" and "incorrect" screenshots show different fonts. Please compare results of Firefox and other browsers using the SAME font.
(Reporter)

Comment 6

7 years ago
used the SAME font,this time. 
1)all three set to Bangla Sangam MN font: firefox- incorrect, safari,chrome- correct
2)all three set to Bangla MN font: firefox- incorrect, safari,chrome- correct

these are the only two fonts available in lion by default for bengali script
(Reporter)

Comment 7

7 years ago
Bug still present in firefox 10. using Bangla MN font .
(Reporter)

Comment 8

6 years ago
bug is present in firefox 11 too. Also, what is the process to mark the status of the bug as "confirmed" from "unconfirmed"
Version: 6 Branch → 11 Branch
(Assignee)

Comment 9

6 years ago
Confirming, as I can reproduce the problem on Lion here. However, this is an OS X bug, not a Gecko bug; to demonstrate that it is not limited to Firefox, copy the example text

না না, গতকাল বাংলাদেশী বাজার থেকে কিনে আনলাম. মমতা তিস্তার জল দিলে পারত. বাংলাদেশী ইলিশ অনেক বেশি ভালো

into a new window in TextEdit.app, and use the Fonts panel to format the text in a fairly large font such as Bangla Sangam MN 24pt. Then gradually change the width of the window so that the text wraps to different widths. Note how the last word ভালো, which contains the "split vowel" ো, sometimes renders correctly and sometimes wrongly, depending on the exact content of the line.

I haven't managed to reproduce the problem in Safari, though I'm not sure why. (Given the erratic nature of it (as shown by TextEdit), perhaps they just got lucky!)
Status: UNCONFIRMED → NEW
Ever confirmed: true

Updated

6 years ago
Version: 11 Branch → Trunk
(Reporter)

Comment 10

6 years ago
I can reproduce the problem in TextEdit. Weirdly safari/chrome never displays the same problem for a large sample of tests. I will file a bug report with apple.
(Reporter)

Comment 11

6 years ago
TextEdit.app in os x 10.8 mountain lion now correctly renders the word ভালো  which contains the "split vowel" ো . However Firefox 15.0.1 continues to render it incorrectly.
(Reporter)

Comment 12

5 years ago
(In reply to Jonathan Kew (:jfkthame) from comment #9)
> Confirming, as I can reproduce the problem on Lion here. However, this is an
> OS X bug, not a Gecko bug; to demonstrate that it is not limited to Firefox,
> copy the example text
> 
> না না, গতকাল বাংলাদেশী বাজার থেকে কিনে আনলাম. মমতা তিস্তার জল দিলে পারত.
> বাংলাদেশী ইলিশ অনেক বেশি ভালো
> 
> into a new window in TextEdit.app, and use the Fonts panel to format the
> text in a fairly large font such as Bangla Sangam MN 24pt. Then gradually
> change the width of the window so that the text wraps to different widths.
> Note how the last word ভালো, which contains the "split vowel" ো, sometimes
> renders correctly and sometimes wrongly, depending on the exact content of
> the line.
> 
> I haven't managed to reproduce the problem in Safari, though I'm not sure
> why. (Given the erratic nature of it (as shown by TextEdit), perhaps they
> just got lucky!)

can we confirm this as a gecko bug?using os x mavericks(10.9.1), textedit.app displays the split vowel correctly. Firefox is the only browser which displays this incorrectly.
(Assignee)

Comment 13

5 years ago
I don't currently have 10.9 for testing, but I still believe this must be an OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the text by calling Core Text (an OS X component) and just renders whatever glyphs that returns. The failure to handle the split vowel properly must be happening within Core Text.
(Reporter)

Comment 14

5 years ago
(In reply to Jonathan Kew (:jfkthame) from comment #13)
> I don't currently have 10.9 for testing, but I still believe this must be an
> OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the
> text by calling Core Text (an OS X component) and just renders whatever
> glyphs that returns. The failure to handle the split vowel properly must be
> happening within Core Text.

Oh. The webkit based browsers(opera/chrome/safari) display correctly.
(Assignee)

Comment 15

5 years ago
(In reply to swagat from comment #14)
> (In reply to Jonathan Kew (:jfkthame) from comment #13)
> > I don't currently have 10.9 for testing, but I still believe this must be an
> > OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the
> > text by calling Core Text (an OS X component) and just renders whatever
> > glyphs that returns. The failure to handle the split vowel properly must be
> > happening within Core Text.
> 
> Oh. The webkit based browsers(opera/chrome/safari) display correctly.

The erratic behavior of TextEdit on 10.7 suggested to me that the problem is very sensitive to exactly how the underlying AAT shaping code gets used in a particular application - sometimes it worked correctly, sometimes it failed.

Interestingly, though, I can no longer seem to reproduce the behavior in TextEdit on 10.7. (Perhaps something has been changed during a system update?)

It does seem possible that there's a subtle bug in how Gecko interfaces with Core Text, and that's leading to the incorrect rendering here. It'd be a bit surprising given that TextEdit used to suffer (intermittently) from the same issue, but it'd be worth trying to trace more carefully exactly what happens in gfxCoreTextShaper during the shaping of the problem sequences.
(Assignee)

Comment 16

5 years ago
Test string: U+0995,U+0995,U+09CB using Bangla Sangam MN.ttf

(A) Using standalone hb-view tool with debugging printfs added:

CFShow(attr_string) immediately before the call to CTLineCreateWithAttributedString:

\u0995\u0995\u09cb 0x7fa0e50070f0 {NSFont=CTFont <name: BanglaSangamMN, size: 2048.000000, matrix: 0x0>
CTFontDescriptor <attributes: <CFBasicHash 0x7fa0e50074f0 [0x7fff7c0c5fa0]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff7be0ab38 [0x7fff7c0c5fa0]>{contents = "NSFontNameAttribute"} = <CFString 0x7fa0e5007270 [0x7fff7c0c5fa0]>{contents = "BanglaSangamMN"}
}
>} Len 3

Resulting glyph run in the CTLine (printing glyph IDs and original string indices):

Glyphs: 128@0  168@2     128@1  602@2
        bn_ka  bn_ekaar  bn_ka  bn_aakaar

Note how the character U+09CB at index 2 has been "split" into two glyphs, the bn_ekaar that is moved before the consonant, and bn_aakaar that remains in the original post-consonant position.


(B) Within Firefox, adding debugging print statements in gfxCoreTextShaper.cpp:

CFShow(attrStringObj) immediately before the call to CTLineCreateWithAttributedString:

\u0995\u0995\u09cb 0x12340eac0 {NSFont=CTFont <name: BanglaSangamMN, size: 500.000000, matrix: 0x0>
CTFontDescriptor <attributes: <CFBasicHash 0x12268ad00 [0x7fff7c0c5fa0]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff7be0ab38 [0x7fff7c0c5fa0]>{contents = "NSFontNameAttribute"} = <CFString 0x1189369e0 [0x7fff7c0c5fa0]>{contents = "BanglaSangamMN"}
}
>} Len 3

Resulting glyph run in the CTLine:

Glyphs: 128@0  128@1  170@2     602@2
        bn_ka  bn_ka  bn_okaar  bn_aakaar

Note that U+09CB (matra o) has been "split" to produce the extra bn_aakaar glyph for the right-hand part, but the bn_okaar glyph has not been changed to bn_ekaar, which would be the appropriate glyph for the left-hand part of the vowel. And therefore it has also not been reordered to the pre-consonant position.


Conclusion: given the exact same Unicode string, and the same AAT font, CTLineCreateWithAttributedString is sometimes failing to shape the split vowel U+09CB properly. It looks like it's specifically the feature "Stand Alone Matras" in the Bangla font that isn't being applied correctly. There's no significant difference in the CFAttributedString parameters being passed to the Core Text function; why is it behaving inconsistently?

I suspect there may be something like an uninitialized variable used within the Core Text shaping code, so that its behavior depends on what happens to be in some particular register or memory location when it is called.

Updated

5 years ago
Blocks: 953231
(Assignee)

Updated

3 years ago
Duplicate of this bug: 1145515
(Assignee)

Comment 19

3 years ago
I've just been looking into this again, and concluded that (at least on OS X 10.9), the issue here is that the Bangla MN and Bangla Sangam MN fonts (as well as their Kannada counterparts, see bug 728557 and bug 953231) depends on the "Line Initial Smart Swash" feature being enabled. If this feature is turned off, the shaping of certain split vowels fails.

In Gecko, we shape with these line-edge features disabled because they can affect measurement, and at the time of shaping we don't yet know where the line breaks will be.

This is a font bug, IMO: line-initial (and -final) swashes are a typographic option that should not affect the basic shaping needed for linguistically-correct rendering. Nevertheless, given that this applies to the default fonts Apple ships for these scripts, I think we should try to work around the problem. Patch coming.
(Assignee)

Updated

3 years ago
Blocks: 728557
(Assignee)

Comment 20

3 years ago
Created attachment 8580670 [details] [diff] [review]
Work around buggy AAT fonts for Bengali and Kannada scripts

Try job in progress: https://treeherder.mozilla.org/#/jobs?repo=try&revision=bba4e84d2923. Will flag for review after checking test results.
(Assignee)

Comment 21

3 years ago
Created attachment 8580716 [details] [diff] [review]
Work around buggy AAT fonts for Bengali and Kannada scripts
Attachment #8580716 - Flags: review?(jdaggett)
(Assignee)

Updated

3 years ago
Attachment #8580670 - Attachment is obsolete: true
(Assignee)

Updated

3 years ago
Assignee: nobody → jfkthame
Status: NEW → ASSIGNED
(Assignee)

Comment 22

3 years ago
FTR, I reported the problem with the abuse of registered smart-swash feature selectors in Bangla and Kannada fonts as rdar://20257092.

Comment 23

3 years ago
Comment on attachment 8580716 [details] [diff] [review]
Work around buggy AAT fonts for Bengali and Kannada scripts

Review of attachment 8580716 [details] [diff] [review]:
-----------------------------------------------------------------

r+ Unfortunate that we have to clutter up the code like this.

If there's an easy way to make a reftest, that would be nice to include.

>+    if (IsBuggyIndicScript(aScript)) {
>+        // To work around buggy Indic AAT fonts shipped with OS X,
>+        // we re-enable the Line Initial Smart Swashes feature that is needed
>+        // for "split vowels" to work in at least Bengali and Kannada fonts.
>+        // See bug reports such as 686225, 728557, 953231, 1145515
>+        tempCTFont =
>+            CreateCTFontWithFeatures(::CTFontGetSize(mCTFont),
>+                                     aShapedText->DisableLigatures()
>+                                         ? GetIndicDisableLigaturesDescriptor()
>+                                         : GetIndicFeaturesDescriptor());

I think it would be useful to list here the specific fonts that appear to contain this bug.
Attachment #8580716 - Flags: review?(jdaggett) → review+
(Assignee)

Comment 24

3 years ago
Created attachment 8582302 [details] [diff] [review]
Reftest for Indic shaping with buggy OS X (AAT) fonts

It's difficult to positively reftest the precise shaping of a bunch of Indic text, but we can at least test that this specific bug no longer occurs. The examples here fail on current trunk, and pass with the patch; they should harmlessly pass on any other platform, too, regardless of available fonts.
Attachment #8582302 - Flags: review?(jdaggett)

Updated

3 years ago
Attachment #8582302 - Flags: review?(jdaggett) → review+
https://hg.mozilla.org/mozilla-central/rev/23e60e2a05a5
https://hg.mozilla.org/mozilla-central/rev/bfbca6c64c1b
Status: ASSIGNED → RESOLVED
Last Resolved: 3 years ago
status-firefox39: --- → fixed
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla39

Comment 28

3 years ago
Jonathan, thanks for figuring out this workaround. 

For the benefit of any Firefox ESR users (I think this fix won't make it into the next ESR which is based on Firefox 38), let me just mention that one can also download and install Google's "Noto Sans Bengali" font, which is rendered properly in Firefox. This can be found at 
https://www.google.com/get/noto/
Once the font is installed, go to the Content tab in Firefox's Preferences dialog box, click the "Advanced..." button under Fonts & Colors, select Bengali under the "Fonts for" dropdown, and replace any references to Bangla MN Fonts with the Noto Sans Bengali font you just installed.
You need to log in before you can comment on or make changes to this bug.