428317 - [gu-IN] Some combinations not rendering properly on Mac system!

Status: RESOLVED FIXED

(Core :: Graphics, defect, P3)

Description

[Ankit Patel]

During the testing phase of RC1 for Firefox 3, found few combinations not appearing correctly on Mac system.

Here are the screenshots of those combinations which can help to identify the issue:
Mac:
http://indianoss.sourceforge.net/Firefox_screenshots_on_Mac/
Linux:
http://indianoss.sourceforge.net/Firefox_screenshots_on_Linux/

No idea whether it's font or rendering issue, but I guess it should block gujarati being released for Mac system.

Comment 1

[Axel Hecht [:Pike]]

Ankit, mind pasting the actual string into this bug?

And I guess that Core GFX Mac is a better component for this.

Comment 2

[Ankit Patel]

1. http://mxr.mozilla.org/l10n/source/gu-IN/security/manager/chrome/pippki/certManager.dtd#98
2. http://mxr.mozilla.org/l10n/source/gu-IN/browser/chrome/browser/preferences/content.dtd#34
3. http://mxr.mozilla.org/l10n/source/gu-IN/toolkit/chrome/passwordmgr/passwordManager.dtd#41
4. http://mxr.mozilla.org/l10n/source/gu-IN/toolkit/chrome/passwordmgr/passwordmgr.properties#66

Messages highlighted in the screenshots are from the following files respectively...

Comment 3

[Simon Montagu :smontagu]

Attachment: Minimized testcase.

This testcase includes some analysis of the problematical sequences for the benefit of people (like me!) who don't read Gujarati fluently.

Comment 4

[Simon Montagu :smontagu]

Since there is no problem on Safari, I'm assuming this is a rendering bug rather than a font problem. I'm guessing that we are somehow dropping glyphs from the clusters.

Comment 5

[John Daggett (:jtd)]

Attachment: screenshot, testcase FF3 trunk vs. Safari (right)

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

John, if you want to work on this please assign it to yourself, otherwise assign it to me :-).

Comment 7

[John Daggett (:jtd)]

Attachment: another testcase with gujarati codepoint display

Comment 8

[Ankit Patel]

Test cases look correct to me...

Comment 9

[Smokey Ardisson (offline for a while; not following bugs - do not email)]

Kicking out of the dead Gfx:Mac ;)

Comment 10

[Axel Hecht [:Pike]]

John, is this something we can get on 1.9.0.1? 1.9.1 would be good for sure.

Comment 11

[Shreevatsa R]

This problem happens with the Devanagari script as well ( bug 429064 ) on OS X (but not on Windows and Linux).
Interestingly, there are some vowel signs and some consonants with which the problem does not occur. (See following attachhment, along the lines of Attachment 315721 [details].)

Comment 12

[Shreevatsa R]

Attachment: Testcase for Devanagari, almost all vowel signs and consonants

Testcase for Devanagari, almost all vowel signs and consonants.

Comment 13

[Shreevatsa R]

This bug happens with the Kannada script as well.

[Namely, it works "in the case of RA+VIRAMA+consonant, but in the
case of RA+VIRAMA+consonant+vowel the consonant disappears."]

Comment 14

[Simon Montagu :smontagu]

Attachment: Kannada testcase

Comment 15

[Simon Montagu :smontagu]

Attachment: Patch

I don't like this much, but attaching it as a basis for discussion...

Attachment 315358 [details] 315721 is no good for testing right now because of bug 462387, but attachment 333182 [details] and attachment 345611 [details] work well with this patch. Ideally we want a more robust solution which can recognize arbitrary-length sequences of reordered glyphs.

Comment 16

[Simon Montagu :smontagu]

http://hg.mozilla.org/mozilla-central/rev/b1bf4dd217be

Comment 17

[Daniel Veditz [:dveditz]]

If you want this on the 1.9.0.x branch please request approval on a patch that will apply to the branch.

Comment 18

[Simon Montagu :smontagu]

Comment on attachment 345614 [details] [diff] [review]
Patch

The patch applies to 1.9.0.x branch as-is. It's well baked on trunk, and has had beta exposure too.

Comment 19

[Simon Montagu :smontagu]

I forgot to mention the most important reason we want this on 1.9.0.x: it will enable us to release the gu-IN localization for Mac, which we haven't yet done.

Comment 21

[Daniel Veditz [:dveditz]]

Comment on attachment 345614 [details] [diff] [review]
Patch

Approved for 1.9.0.6, a=dveditz for release-drivers.

Comment 22

[Al Billings [:abillings - ex-MoCo]]

This appears to be fixed for 1.9.0.6 with Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6pre) Gecko/2009011304 GranParadiso/3.0.6pre.

I notice that the testcase in comment 14 renders as squares in 1.9.0.5, 1.9.0.6 and the nightly 1.9.1 build. I'm assuming that this doesn't invalidate the fix but I'd like someone to confirm.

Comment 23

[philippe (part-time)]

(In reply to comment #22)
 
> I notice that the testcase in comment 14 renders as squares in 1.9.0.5, 1.9.0.6
> and the nightly 1.9.1 build. I'm assuming that this doesn't invalidate the fix
> but I'd like someone to confirm.
You probably don't have the necessary fonts installed. It works fine here (10.5.6, Gecko 1.9.06pre and up)

Comment 24

[Al Billings [:abillings - ex-MoCo]]

All right. I'll mark it as verified for 1.9.0.6 since you covered the last scenario there.

Comment 25

[Shreevatsa R]

Please reopen; it's still not fixed in general. (Tested with Firefox 3.0.7 in which this bug is marked as fixed, also tested in latest Firefox nightly as of 04-Mar-2009.)

It now works with RA+VIRAMA+consonant+vowel, but not with RA+virama+consonant+consonant+vowel. Here's a testcase: in the word irkvem, इर्क्वेम्, the first consonant disappears, and it appears identical to "इर्वेम्". More generally, in RA+VIRAMA+consonant1+consonant2+...+consonantN+vowel, all consonants except consonantN disappear.

(Should this be filed as a new bug, because it's more general?)

Comment 26

[Ankit Patel]

Hi Shrivatsa,

The test text you put here is from Devnagari not Gujarati (gu-IN)...

and please open another bug for that...

Thanks!
Ankit

(In reply to comment #25)
> Please reopen; it's still not fixed in general. (Tested with Firefox 3.0.7 in
> which this bug is marked as fixed, also tested in latest Firefox nightly as of
> 04-Mar-2009.)
> 
> It now works with RA+VIRAMA+consonant+vowel, but not with
> RA+virama+consonant+consonant+vowel. Here's a testcase: in the word irkvem,
> इर्क्वेम्, the first consonant disappears, and it appears identical to
> "इर्वेम्". More generally, in
> RA+VIRAMA+consonant1+consonant2+...+consonantN+vowel, all consonants except
> consonantN disappear.
> 
> (Should this be filed as a new bug, because it's more general?)

Comment 27

[Simon Montagu :smontagu]

(In reply to comment #25)

> It now works with RA+VIRAMA+consonant+vowel, but not with
> RA+virama+consonant+consonant+vowel. Here's a testcase: in the word irkvem,
> इर्क्वेम्, the first consonant disappears, and it appears identical to
> "इर्वेम्". More generally, in
> RA+VIRAMA+consonant1+consonant2+...+consonantN+vowel, all consonants except
> consonantN disappear.

I see that in Safari your example irkvem, इर्क्वेम् is displayed with the repha over the va. Is that correct? In general, can there be sequences in Indic languages RA+VIRAMA+consonant1+consonant2+...+consonantN+vowel of arbitrary length, or can we assume a maximum value for N?

Comment 28

[Shreevatsa R]

Yes, the repha should be displayed over the va, and over the last consonant in general. (Safari's rendering is correct.)

In principle there is no limit on how many consonants there can be, but in practice there's probably a limit based on what words exist in the different languages (also, exotic proper nouns!), what can actually be pronounced, etc. (And the limit might be surprisingly high: I encountered today the actual word kārtsnyam: कार्त्स्न्यम् which has 5 consonants in a row including the ra and is still pronounceable. It doesn't have a vowel following them, though, so it renders fine.)

Ideally the rendering would always put the repha on the last consonant no matter how many (Safari manages to do this, as does Firefox on Linux/Windows), but I guess a limit of, say, 10 consonants would probably cover everything meaningful anyone is likely to encounter :)

Comment 29

[Shreevatsa R]

(In reply to comment #28)
> Ideally the rendering would always put the repha on the last consonant no
> matter how many (Safari manages to do this, as does Firefox on Linux/Windows),
> but I guess a limit of, say, 10 consonants would probably cover everything
> meaningful anyone is likely to encounter :)

On the other hand, it's probably not a good idea to use such a limit, because the bug will still be present, just less likely to be encountered: this would make it all the more puzzling when someone has to face it, less likely to be reported, etc.

Comment 30

[Jonathan Kew [:jfkthame]]

(In reply to comment #28)

> (And the limit might be surprisingly high: I encountered today the actual word
> kārtsnyam: कार्त्स्न्यम् which has 5 consonants in a row including the ra and
> is still pronounceable. It doesn't have a vowel following them, though, so it
> renders fine.)

The rendering doesn't look fine to me; the /ta/ is missing.

AFAICT, this problem doesn't depend on the presence of a vowel, only on the number of glyphs in the consonant cluster. (Most of kārtsnyam survives because of combining the characters into a single conjunct glyph.)

For a simple demonstration of the issue, just type

ra virama ka virama ka virama ka virama ka
(र्क्क्क्क -- should have 4 /ka/s but only two are visible; copy/paste to TextEdit to prove that they're present)

right here in the comment box; after the second /ka/, additional ones simply disappear.

The CoreText rendering path which we hope to add for 1.9.2 (bug 389074) resolves this, but we still need a fix in the ATSUI path as well for pre-10.5 systems.

Simon, are you prepared to reopen this bug or do you want a new one instead?

Comment 31

[Simon Montagu :smontagu]

I'd rather a new bug, and with a clearer description. The original bug applied to Gujarati, Devanagari, Kannada and possibly other Indic scripts, and I expect the remaining issue does as well.

Comment 32

[Shreevatsa R]

(In reply to comment #31)
> I'd rather a new bug, and with a clearer description. The original bug applied
> to Gujarati, Devanagari, Kannada and possibly other Indic scripts, and I expect
> the remaining issue does as well.
   
I wasn't sure if a new bug was already reported, so I've created Bug 481948. I'm not sure if I'm the right person to give a clearer description. :)