Closed Bug 686225 Opened 8 years ago Closed 5 years ago

Bengali font vowel incorrect on mac osx lion

Categories

(Core :: Layout: Text and Fonts, defect)

x86
macOS
defect
Not set

Tracking

()

RESOLVED FIXED
mozilla39
Tracking Status
firefox39 --- fixed

People

(Reporter: swagat.wp, Assigned: jfkthame)

References

(Blocks 1 open bug, )

Details

Attachments

(4 files, 1 obsolete file)

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:6.0.2) Gecko/20100101 Firefox/6.0.2
Build ID: 20110902133214

Steps to reproduce:

using mac os x lion which has support for bengali fonts, chrome and safari display bengali fonts correctly. but firefox does not .


Actual results:

best described in screenshot


Expected results:

best described in screenshot
Attached image the correct rendering
Component: General → Layout: Text
Product: Firefox → Core
QA Contact: general → layout.fonts-and-text
Please confirm what font is being used for the Bengali text in this example. You can use the fontinfo addon[1] to determine this.

[1] https://addons.mozilla.org/en-US/firefox/addon/fontinfo/
the font being used is Bangla Sangam MN.
bug is in firefox 7 branch too.
Note that the "correct" and "incorrect" screenshots show different fonts. Please compare results of Firefox and other browsers using the SAME font.
used the SAME font,this time. 
1)all three set to Bangla Sangam MN font: firefox- incorrect, safari,chrome- correct
2)all three set to Bangla MN font: firefox- incorrect, safari,chrome- correct

these are the only two fonts available in lion by default for bengali script
Bug still present in firefox 10. using Bangla MN font .
bug is present in firefox 11 too. Also, what is the process to mark the status of the bug as "confirmed" from "unconfirmed"
Version: 6 Branch → 11 Branch
Confirming, as I can reproduce the problem on Lion here. However, this is an OS X bug, not a Gecko bug; to demonstrate that it is not limited to Firefox, copy the example text

না না, গতকাল বাংলাদেশী বাজার থেকে কিনে আনলাম. মমতা তিস্তার জল দিলে পারত. বাংলাদেশী ইলিশ অনেক বেশি ভালো

into a new window in TextEdit.app, and use the Fonts panel to format the text in a fairly large font such as Bangla Sangam MN 24pt. Then gradually change the width of the window so that the text wraps to different widths. Note how the last word ভালো, which contains the "split vowel" ো, sometimes renders correctly and sometimes wrongly, depending on the exact content of the line.

I haven't managed to reproduce the problem in Safari, though I'm not sure why. (Given the erratic nature of it (as shown by TextEdit), perhaps they just got lucky!)
Status: UNCONFIRMED → NEW
Ever confirmed: true
Version: 11 Branch → Trunk
I can reproduce the problem in TextEdit. Weirdly safari/chrome never displays the same problem for a large sample of tests. I will file a bug report with apple.
TextEdit.app in os x 10.8 mountain lion now correctly renders the word ভালো  which contains the "split vowel" ো . However Firefox 15.0.1 continues to render it incorrectly.
(In reply to Jonathan Kew (:jfkthame) from comment #9)
> Confirming, as I can reproduce the problem on Lion here. However, this is an
> OS X bug, not a Gecko bug; to demonstrate that it is not limited to Firefox,
> copy the example text
> 
> না না, গতকাল বাংলাদেশী বাজার থেকে কিনে আনলাম. মমতা তিস্তার জল দিলে পারত.
> বাংলাদেশী ইলিশ অনেক বেশি ভালো
> 
> into a new window in TextEdit.app, and use the Fonts panel to format the
> text in a fairly large font such as Bangla Sangam MN 24pt. Then gradually
> change the width of the window so that the text wraps to different widths.
> Note how the last word ভালো, which contains the "split vowel" ো, sometimes
> renders correctly and sometimes wrongly, depending on the exact content of
> the line.
> 
> I haven't managed to reproduce the problem in Safari, though I'm not sure
> why. (Given the erratic nature of it (as shown by TextEdit), perhaps they
> just got lucky!)

can we confirm this as a gecko bug?using os x mavericks(10.9.1), textedit.app displays the split vowel correctly. Firefox is the only browser which displays this incorrectly.
I don't currently have 10.9 for testing, but I still believe this must be an OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the text by calling Core Text (an OS X component) and just renders whatever glyphs that returns. The failure to handle the split vowel properly must be happening within Core Text.
(In reply to Jonathan Kew (:jfkthame) from comment #13)
> I don't currently have 10.9 for testing, but I still believe this must be an
> OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the
> text by calling Core Text (an OS X component) and just renders whatever
> glyphs that returns. The failure to handle the split vowel properly must be
> happening within Core Text.

Oh. The webkit based browsers(opera/chrome/safari) display correctly.
(In reply to swagat from comment #14)
> (In reply to Jonathan Kew (:jfkthame) from comment #13)
> > I don't currently have 10.9 for testing, but I still believe this must be an
> > OS X bug. The Bangla Sangam MN font is an AAT font, so Gecko "shapes" the
> > text by calling Core Text (an OS X component) and just renders whatever
> > glyphs that returns. The failure to handle the split vowel properly must be
> > happening within Core Text.
> 
> Oh. The webkit based browsers(opera/chrome/safari) display correctly.

The erratic behavior of TextEdit on 10.7 suggested to me that the problem is very sensitive to exactly how the underlying AAT shaping code gets used in a particular application - sometimes it worked correctly, sometimes it failed.

Interestingly, though, I can no longer seem to reproduce the behavior in TextEdit on 10.7. (Perhaps something has been changed during a system update?)

It does seem possible that there's a subtle bug in how Gecko interfaces with Core Text, and that's leading to the incorrect rendering here. It'd be a bit surprising given that TextEdit used to suffer (intermittently) from the same issue, but it'd be worth trying to trace more carefully exactly what happens in gfxCoreTextShaper during the shaping of the problem sequences.
Test string: U+0995,U+0995,U+09CB using Bangla Sangam MN.ttf

(A) Using standalone hb-view tool with debugging printfs added:

CFShow(attr_string) immediately before the call to CTLineCreateWithAttributedString:

\u0995\u0995\u09cb 0x7fa0e50070f0 {NSFont=CTFont <name: BanglaSangamMN, size: 2048.000000, matrix: 0x0>
CTFontDescriptor <attributes: <CFBasicHash 0x7fa0e50074f0 [0x7fff7c0c5fa0]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff7be0ab38 [0x7fff7c0c5fa0]>{contents = "NSFontNameAttribute"} = <CFString 0x7fa0e5007270 [0x7fff7c0c5fa0]>{contents = "BanglaSangamMN"}
}
>} Len 3

Resulting glyph run in the CTLine (printing glyph IDs and original string indices):

Glyphs: 128@0  168@2     128@1  602@2
        bn_ka  bn_ekaar  bn_ka  bn_aakaar

Note how the character U+09CB at index 2 has been "split" into two glyphs, the bn_ekaar that is moved before the consonant, and bn_aakaar that remains in the original post-consonant position.


(B) Within Firefox, adding debugging print statements in gfxCoreTextShaper.cpp:

CFShow(attrStringObj) immediately before the call to CTLineCreateWithAttributedString:

\u0995\u0995\u09cb 0x12340eac0 {NSFont=CTFont <name: BanglaSangamMN, size: 500.000000, matrix: 0x0>
CTFontDescriptor <attributes: <CFBasicHash 0x12268ad00 [0x7fff7c0c5fa0]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff7be0ab38 [0x7fff7c0c5fa0]>{contents = "NSFontNameAttribute"} = <CFString 0x1189369e0 [0x7fff7c0c5fa0]>{contents = "BanglaSangamMN"}
}
>} Len 3

Resulting glyph run in the CTLine:

Glyphs: 128@0  128@1  170@2     602@2
        bn_ka  bn_ka  bn_okaar  bn_aakaar

Note that U+09CB (matra o) has been "split" to produce the extra bn_aakaar glyph for the right-hand part, but the bn_okaar glyph has not been changed to bn_ekaar, which would be the appropriate glyph for the left-hand part of the vowel. And therefore it has also not been reordered to the pre-consonant position.


Conclusion: given the exact same Unicode string, and the same AAT font, CTLineCreateWithAttributedString is sometimes failing to shape the split vowel U+09CB properly. It looks like it's specifically the feature "Stand Alone Matras" in the Bangla font that isn't being applied correctly. There's no significant difference in the CFAttributedString parameters being passed to the Core Text function; why is it behaving inconsistently?

I suspect there may be something like an uninitialized variable used within the Core Text shaping code, so that its behavior depends on what happens to be in some particular register or memory location when it is called.
Blocks: 953231
Duplicate of this bug: 1145515
I've just been looking into this again, and concluded that (at least on OS X 10.9), the issue here is that the Bangla MN and Bangla Sangam MN fonts (as well as their Kannada counterparts, see bug 728557 and bug 953231) depends on the "Line Initial Smart Swash" feature being enabled. If this feature is turned off, the shaping of certain split vowels fails.

In Gecko, we shape with these line-edge features disabled because they can affect measurement, and at the time of shaping we don't yet know where the line breaks will be.

This is a font bug, IMO: line-initial (and -final) swashes are a typographic option that should not affect the basic shaping needed for linguistically-correct rendering. Nevertheless, given that this applies to the default fonts Apple ships for these scripts, I think we should try to work around the problem. Patch coming.
Blocks: 728557
Try job in progress: https://treeherder.mozilla.org/#/jobs?repo=try&revision=bba4e84d2923. Will flag for review after checking test results.
Attachment #8580670 - Attachment is obsolete: true
Assignee: nobody → jfkthame
Status: NEW → ASSIGNED
FTR, I reported the problem with the abuse of registered smart-swash feature selectors in Bangla and Kannada fonts as rdar://20257092.
Comment on attachment 8580716 [details] [diff] [review]
Work around buggy AAT fonts for Bengali and Kannada scripts

Review of attachment 8580716 [details] [diff] [review]:
-----------------------------------------------------------------

r+ Unfortunate that we have to clutter up the code like this.

If there's an easy way to make a reftest, that would be nice to include.

>+    if (IsBuggyIndicScript(aScript)) {
>+        // To work around buggy Indic AAT fonts shipped with OS X,
>+        // we re-enable the Line Initial Smart Swashes feature that is needed
>+        // for "split vowels" to work in at least Bengali and Kannada fonts.
>+        // See bug reports such as 686225, 728557, 953231, 1145515
>+        tempCTFont =
>+            CreateCTFontWithFeatures(::CTFontGetSize(mCTFont),
>+                                     aShapedText->DisableLigatures()
>+                                         ? GetIndicDisableLigaturesDescriptor()
>+                                         : GetIndicFeaturesDescriptor());

I think it would be useful to list here the specific fonts that appear to contain this bug.
Attachment #8580716 - Flags: review?(jdaggett) → review+
It's difficult to positively reftest the precise shaping of a bunch of Indic text, but we can at least test that this specific bug no longer occurs. The examples here fail on current trunk, and pass with the patch; they should harmlessly pass on any other platform, too, regardless of available fonts.
Attachment #8582302 - Flags: review?(jdaggett)
Attachment #8582302 - Flags: review?(jdaggett) → review+
https://hg.mozilla.org/mozilla-central/rev/23e60e2a05a5
https://hg.mozilla.org/mozilla-central/rev/bfbca6c64c1b
Status: ASSIGNED → RESOLVED
Closed: 5 years ago
Flags: in-testsuite+
Resolution: --- → FIXED
Target Milestone: --- → mozilla39
Jonathan, thanks for figuring out this workaround. 

For the benefit of any Firefox ESR users (I think this fix won't make it into the next ESR which is based on Firefox 38), let me just mention that one can also download and install Google's "Noto Sans Bengali" font, which is rendered properly in Firefox. This can be found at 
https://www.google.com/get/noto/
Once the font is installed, go to the Content tab in Firefox's Preferences dialog box, click the "Advanced..." button under Fonts & Colors, select Bengali under the "Fonts for" dropdown, and replace any references to Bangla MN Fonts with the Noto Sans Bengali font you just installed.
You need to log in before you can comment on or make changes to this bug.