204286 - support Devanagari rendering with existing hindi shaper on Xft build

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[Jungshik Shin]

This is the fifth application of my patch to fix bug 176290. Other applications
include MathML (bug 128153), pre-1933 orthography Korean (bug 176315), Thai(bug
203052), and
Tamil (bug 204039). 

In this case, there's very little to do (for Devanagari) 
other than landing the patch for bug 176290 because the covnerter we need was
already written by Prabhat. All that need be done[1] is just get
some truetype fonts with Sun-Unicode-India-0 encoding (in which
nominal glyphs for Indic script letters are at their Unicode code points
with additional presentation forms in the PUA) and add entries for them
to fontEncoding.properties file like this:

# SunIndic fonts.
# glyph arrangement : Nominal glyphs for Indic scripts at their Unicode code
#                     points along with presentation forms in PUA
# coverage : US-ASCII, Devanagari, Bengali, Gurmukhi, Gujarati, Tamil,
#            Telugu, Kanada, Malayalam
encoding.saraswati5.ttf = x-sun-unicode-india-0.wide

[1] This is only the case for Devanagari. We still need converters
for Bengali, Gurmukhi, Gujarati, Telugu, Kanada, Malayalam. For Tamil, 
we have a separate converter (bug 204039) for TSCII-encoded fonts.

Comment 1

[Jungshik Shin]

Adding a couple of more. 

Comment 2

[Jungshik Shin]

It turned out that we needs a thin wrapper over x-sun-unicode-india-0 converter
as is the case for Thai (bug 203052). x-sun-unicode-india-0 encoder appears
to generate 'mix-width' output (for US-ASCII, it produces 8bit chars
while for chars beyond 0x100 - or 0x80 -, it generates 16bit shorts.)

Comment 3

[Jungshik Shin]

Attachment: A Devanagari test case

A set of test cases. All of them were taken from examples given in
Unicode 3.0 section 9.1 (p. 214 - p.220).
I may update it with more examples and put it up at
http://jshin.net/moztest/hindi2u.html

Comment 4

[Jungshik Shin]

Attachment: a screenshot

This shot was taken of the test case (attachment 12383 [details]) with
two TTFs (differing only in weight) in x-sun-unicode-india-0 encoding.
I slightly modified SunIndic encoder, but that shouldn't affect this shot
because the change is for US-ASCII characters. Nonetheless, I'll upload
my patch. To get this screenshot, you have to apply the latest patch 
to fix bug 176290 and add an entry for (a) TTF(s) in SunIndic encoding
to fontEncoding.properties file.

Comment 5

[Jungshik Shin]

Attachment: a patch

After writting a thin wrapper over SunIndic encoder, I realized that
I didn't have to if we can just change SunIndic encoder slightly. Currently,
it returns single byte sequences for US-ASCII characters. If it's safe
to extend US-ASCII to UCS2BE (afaik, the only other place it's used is
nsFontMetricsGTK/Xlib)
by zero-padding, I think it's better to modify it than to subclass it.

Prabhat, can I do that?  BTW, as is probably well known to you, the screenshot
revealed
a few problems with the current Hindi shaper including ZWNJ/ZWJ handling
(they're
depicted as spacing glyphs from some other fonts on my system), Vocalic R,
and a case involving CANDRABINDU.

Comment 6

[Prabhat Hegde]

hi JungShik,

No problem. You're right the shaper currently does'nt have ZWJ & ZNWJ :-)
http://lxr.mozilla.org/seamonkey/source/intl/ctl/src/nsUnicodeToSunIndic.cpp#259
I will look at it.

Also did you get a chance to look at my patch in :
http://bugzilla.mozilla.org/show_bug.cgi?id=203406

That should fix ChandraBindu and some other problems that i had found besides
eliminating the poor handling of non-CTL.

Comment 7

[Jungshik Shin]

Hi Prabhat,
I applied your patch to bug 203406 and now ChandraBindu is taken care of as you
mentioned.
Somehow 'KAd + ZWJ + SSAl' and 'KA + Virima + ZWJ' (== KAh) also work. It's strange
because nsUnicodeToSunIndic doesn't yet even cover ZWJ/ZWNJ. 

Instead of my patch (attachement 122383. there's a flaw anyway), I modified
GetPresentationForm()
in the following way:
  
  - added an optional PRBool aIsWide parameter which is PR_FALSE by default.
(nsILE.h, nsULE.h, nsULE.cpp)
  - modified the loop in nsULE::GetPresentationForm() as following:

    if (tmpGlyphs->num_glyphs > 0) {
      guint i=0, glyphCt=0;
      for (i=0; i < tmpGlyphs->num_glyphs; i++, glyphCt++) {
        if (aIsWide)
          aGlyphs[glyphCt++]=(unsigned char)
                             ((tmpGlyphs->glyphs[i].glyph & 0xFF00) >> 8);
        aGlyphs[glyphCt]=(unsigned char)(tmpGlyphs->glyphs[i].glyph & 0x00FF);
      }
      *aOutLength = (PRSize)glyphCt;
    }

  - In nsUnicodeToSunIndic::Convert, GetPresentationForm() is called with the last
    parameter aIsWide set to PR_TRUE. No change is necessary in
nsUnicodeToTIS620::Convetert
    because by default aIsWide is PR_FALSE.

Comment 8

[Jungshik Shin]

Attachment: a patch to be applied on top of Prabhat's patch for bug 203406

This has to be applied after applying attachment 121744 [details] [diff] [review].
Prabhat, I changed GetPresentationForm() as mentioned earlier.
Is there any problem that my change will cause in nsTextFrame?

Comment 9

[Jungshik Shin]

Comment on attachment 122542 [details] [diff] [review]
a patch to be applied on top of Prabhat's patch for bug 203406


>--- src/nsULE.h.bak	2003-05-06 12:37:36.000000000 +0900
>+++ src/nsULE.h	2003-05-06 12:39:45.000000000 +0900
>@@ -59,7 +59,8 @@
>                                  PRUint32        aLength,
>                                  const char      *aFontCharset,
>                                  char            *aGlyphs,
>-                                 PRSize          *aOutLength);
>+                                 PRSize          *aOutLength,
>+                                 PRBool          aIsWide = PR_TRUE);
^^^^^^^

The default value of aIsWide should be PR_FALSE.

Comment 10

[Jungshik Shin]

*** Bug 202667 has been marked as a duplicate of this bug. ***

Comment 11

[Prabhat Hegde]

Attachment: Incorporates Jungshiks patch

Merged (id=122542) and updated credits.

Comment 12

[Alok Kumar]

Interesting development:
<http://www.eng.buffalo.edu/%7Edsjoshi/mozilla-xft-ctl.html>
Announced here:
<http://sourceforge.net/mailarchive/message.php?msg_id=5737100>

Comment 13

[J.M.N.]

Re: Comment #12

I have visited the CTL enabled Mozilla page and found the link :

http://www.eng.buffalo.edu/%7Edsjoshi/mozilla-xft-with-ctl.png

This looks good but some of the alignments appear to be a bit off.

Regarding the announcement message on SourceForge :

http://sourceforge.net/mailarchive/message.php?msg_id=5737100

The address doesn't work.  The error message is :

"Message Not Found. This message does not (any longer) exist. "

Is this the correct address?

Comment 14

[Jungshik Shin]

J.M.N, That's probably because the patch doesn't modify text-dimension-related
part. Only the drawing part is taken care of (I just took a brief look and I'm
not sure though).

Anyway, this bug is not suitable for the discussion of the patch. Let's discuss
it at bug 215219

Comment 15

[Jungshik Shin]

This was fixed with the check-in of the patch for bug 203406 that included the 
patch for this bug.