Last Comment Bug 218887 - (uniscribe) Use Uniscribe APIs in GFX:Win and layout/editor
(uniscribe)
: Use Uniscribe APIs in GFX:Win and layout/editor
Status: RESOLVED DUPLICATE of bug 322938
: intl
Product: Core Graveyard
Classification: Graveyard
Component: GFX: Win32 (show other bugs)
: 1.8 Branch
: x86 Windows ME
: -- normal with 5 votes (vote)
: ---
Assigned To: win32
: Hixie (not reading bugmail)
:
Mentors:
: 234781 377345 (view as bug list)
Depends on: 157967 241485
Blocks: 185600 60546 140013 202351 240914 245157 248361 270012 278291 281483 293511 294579 321735 343405
  Show dependency treegraph
 
Reported: 2003-09-10 19:40 PDT by Jungshik Shin
Modified: 2014-04-26 02:23 PDT (History)
22 users (show)
See Also:
QA Whiteboard:
Iteration: ---
Points: ---


Attachments
experiment using uniscribe in win gfx (9.16 KB, patch)
2004-02-02 09:18 PST, tor
no flags Details | Diff | Splinter Review
updated patch (12.98 KB, patch)
2004-02-02 13:31 PST, tor
no flags Details | Diff | Splinter Review
Malayalam test case (8.61 KB, text/html)
2004-02-03 10:16 PST, Jungshik Shin
no flags Details
rendering of http://bugzilla.mozilla.org/attachment.cgi?id=138332&action=view (21.02 KB, image/png)
2004-02-10 11:44 PST, tor
no flags Details
uniscribe rendering of Malayalam test (35.07 KB, image/png)
2004-02-11 16:14 PST, tor
no flags Details
another experimental patch (10.97 KB, patch)
2004-02-16 13:10 PST, tor
no flags Details | Diff | Splinter Review
checkpoint (11.48 KB, patch)
2004-03-02 15:54 PST, tor
no flags Details | Diff | Splinter Review
use different layout/gfx space conversion (15.17 KB, patch)
2004-03-30 11:52 PST, tor
no flags Details | Diff | Splinter Review

Description Jungshik Shin 2003-09-10 19:40:16 PDT
This is a Windows counterpart of bug  215219, bug 214715(Linux) and bug
121540(MacOS). At the moment, I'm filing this bug as a place holder to offer a
forum for discussion. 

Currently, Mozilla relies on ExtTextOutW and friends for text drawing. That more
or less works even for complex scripts as far as rendering is concerned, but
that doesn't work for selection(copy'n'pasting) and editing (see bug 79286, bug
100173, bug 122552, bug 157534). Moreover, even rendering doesn't work on Win
9x/ME (which is why I'm settign the platform to  Windows ME. see bug 166520, for
instance). With custom font encoding support, some of this problem has been
fixed for Tamil, Korean, Thai and Devanagari (the last one only with CTL
enabled), but that solution is not general enough.  

Given all these, it's pretty obvious that we need to use Uniscribe APIs (as is
done by MS IE) (in the long run).
Comment 1 Rajkumar S 2003-09-11 12:44:55 PDT
> Given all these, it's pretty obvious that we need to use Uniscribe APIs (as is
> done by MS IE) (in the long run).

I feel that mozilla should support OS like win98 and win2k as they are going to
be popular for a long time to come. Here in India very few people are going to
upgrade to XP or heigher. Even the current XP do not have Malayalam support. So
we may not want to force people to upgrade their OS to get local language suport. 

Is it possible to get pango ported in Windows? I am no expert in it, but just a
question, and use that if Uniscribe is not available?

raj
Comment 2 Jungshik Shin 2003-09-11 18:47:04 PDT
I'm afraid you missed my point of filing this bug. Using Uniscribe APIs would
enable Mozilla to support complex scripts on Win 9x/ME as well as on Win 2k/XP.
(it's not Win 9x/ME/2k vs Win XP but Win9x/ME vs Win 2k/XP. Win 2k and XP are
basically the same OS which is very different from Win 9x/ME. It's easy to be
fooled by MS' strange versioning of its OS'). 

Well, the complex script support of Uniscribe is available on Win 9x/ME _only_
if MS IE 6 is installed. So, we have to tell users to install MS IE6 to enable
Mozilla to render complex scripts on Win 9x/ME. This is a downside of using
Uniscribe. We can do a run-time detection of the presence of Uniscribe, but that
wouldn't help us with cutting down the code size, which is another objective of
using Uniscribe APIs.   

As for Pango, Win32 port is available, but it's implemented with Uniscribe APIs
internally. Even if it's not, I guess making the default build of Mozilla-Win
dependent on Pango would be out of question.
Comment 3 Mike Kaply [:mkaply] 2003-11-10 08:42:39 PST
I don't think requiring the users to get the Uniscribe DLL for Win9X/ME is a bad
thing if Mozilla supported Uniscribe.

Does anyone have a sense for how big this effort would be?

We (IBM) might be interested in doing some work on this.
Comment 4 Jungshik Shin 2003-11-10 09:13:55 PST
I have little idea. It could be a pretty extensive work touching both gfx-win
and layout/content. gfx-win part should be easier than fixing nsText*.cpp.
We may phase in changes a little by little.

Another possibility in complex script support is using SIL Graphite. Frank Tang
has worked on it. See http://sila.mozdev.org. 

BTW, Uniscribe's Korean support is worse than Mozilla's current support based on
custom-encoded fonts.  

Another BTW, we have to be careful not to hurt MathML support (that relies on
custom font encoding). On Linux, going all the way to Pango could 'sacrifice'
MathML support.
Comment 5 Jungshik Shin 2004-01-08 19:19:16 PST
After working on bug 215219 (Gfx-Gtk2 counterpart of this bug using Pango), I
realized that a similar approach can be applied to this bug because Uniscribe
and Pango have something common in their APIs.
(I'll take a look at ATSUI APIs on Mac OS X as well)  When I filed this bug, I
intended to use Uniscribe for text selection and cursor movement as well as for
text-drawing/width measurement. However, now I plan to use Uniscribe only for
the text drawing and width measurement. For text selection and cursor movement,
an XP-solution (bug 229896) can be made to work more or less.  

This may not be the best approach, but seems to be the path with the least
'resistance'.
Comment 6 Jungshik Shin 2004-01-08 19:25:17 PST
Just like Xft has an API to use glyph indices instead of Unicode characters for
text drawing, ExTextOutW on Win32 has a flag to indicate glyphs are being passed
over (ETO_GLYPH_INDEX).

With this approach (using Uniscribe to only a limited extent), we can keep
MathML and custom-encoded font support (necessary because Uniscribe's Korean
handling is worse than Mozilla's : bug 176315. also handy in rare cases where
Uniscribe is not available.) intact while takind advantage of Uniscribe's
capability to render complex scripts. 



Comment 7 tor 2004-02-02 09:18:56 PST
Created attachment 140421 [details] [diff] [review]
experiment using uniscribe in win gfx

A quick experiment to hook up uniscribe to the windows font code, complete
with debug spew.  The attempt to bridge the difference between layout's
idea of character spacing and uniscribe glyph based spacing is a complete
hack and doesn't work right.
Comment 8 tor 2004-02-02 13:31:15 PST
Created attachment 140436 [details] [diff] [review]
updated patch

Spacing now somewhat working - spaced strings don't match the length
layout is expecting (since it tries adding space based on characters),
but seems like the right amount is being added.
Comment 9 Jungshik Shin 2004-02-03 08:35:20 PST
tor, great ! attachment 138332 [details] has a test case for justified text rendering.
I've just built a firebird with your patch applied and it seems like your patch
is at about the same status as my patch for Xft with pango was around last
September (patch v 0.3 to bug 215219). Anyway, it's a good start. 

BTW, although it's not necessary at this stage, we later have to add a test for
the presence of Uniscribe and have to fall back if it's not present. 
Comment 10 Jungshik Shin 2004-02-03 10:16:24 PST
Created attachment 140515 [details]
Malayalam test case

You need to install Thoolika opentype font to test this page.

http://www.supersoftweb.com/Unicode.htm
Comment 11 Jungshik Shin 2004-02-03 11:12:05 PST
I found that MS IE's (or perhaps Uniscribe's) notion of 'grapheme cluster' is
different from that of Pango (I relied on in bug 215219). MS IE renders the
second and the third columns of attachment 140515 [details] differently in some cases. My
patched version of Xft+gtk2 on Linux renders them identically because Pango's
notion of grapheme clusters in Indic scripts is the same as mine. However, this
is not a matter of who is right  and who is wrong because even the Unicode
Technical committee  gave up defining what constitutes 'atomic' units  in Indic
scripts. 
Comment 12 tor 2004-02-10 11:44:28 PST
Created attachment 141067 [details]
rendering of http://bugzilla.mozilla.org/attachment.cgi?id=138332&action=view

Rendering of the pango spacing testcase with my current uniscribe work
(later than the attached patch).  The justification case looks a little
cramped, but seems to reflect the narrow interword spacing that layout
is passing down to gfx.
Comment 13 Jungshik Shin 2004-02-10 19:24:33 PST
It's working ! Unicode characters making up a single grapheme cluster are held
together correctly regardless of the letter spacing specified. I have some doubt
about 'justified' case in the screenshot, but that's minor. Why don't you
install Thoolika font for Malayalam and test attachment 140515 [details] as well? The font
is available at http://www.supersoftweb.com/Unicode.htm
Comment 14 tor 2004-02-11 16:14:26 PST
Created attachment 141195 [details]
uniscribe rendering of Malayalam test

After fixing a couple problems it gets close to the correct rendering,
missing only testcases four and five.  I'll investigate further
tomorrow, unless you happen to know offhand of a bug in the Thoolika
font causing this.
Comment 15 tor 2004-02-16 13:10:19 PST
Created attachment 141555 [details] [diff] [review]
another experimental patch
Comment 16 tor 2004-02-18 08:56:24 PST
*** Bug 234781 has been marked as a duplicate of this bug. ***
Comment 17 tor 2004-03-02 15:54:57 PST
Created attachment 142786 [details] [diff] [review]
checkpoint

Different spacing conversion than the pango bridge patch, though I don't
think either really create the spacing that would result if we could
get text frame to work with graphemes.
Comment 18 tor 2004-03-18 09:35:33 PST
jshin, could you try the latest patch and let me know if you see problems?
Thanks.
Comment 19 Jungshik Shin 2004-03-18 10:14:27 PST
I'm building now on Windows (having just one machine with two OS' makes it
rather hard ..).  BTW, I'm afraid your current patch allocates too often. Why
don't you take a look at nsAutoBuffer and use it if/where possible? 
Comment 20 tor 2004-03-18 10:18:38 PST
Right now I'm more interested in getting the functionality right before
doing code cleanup.
Comment 21 Jungshik Shin 2004-03-18 17:56:00 PST
Comment on attachment 142786 [details] [diff] [review]
checkpoint

I've just tested a few test cases and it seems to work well. Thanks !

>+DEFINES	+= -DSTRICT -DDEBUG_tor

I guess you're gonna clean this up later, but even now, I wonder why you have
to hard-code this. If your Windows login name is 'tor', DEBUG_tor is
automatically defined for debug builds, isn't it? 

>+        int val = -1;
>+        for (int j=0; j<itemLength; j++) {
>+          if (val != clusters[j]) {
>+            val = clusters[j];
>+            spacing[j] = aSpacing[items[i].iCharPos + j];

We can do a little better than this, can't we? Of course, once bug 229896 is
fixed and layout makes use of new APIs (made in the fix), we don't have to
worry less about this.
Comment 22 tor 2004-03-30 11:52:19 PST
Created attachment 145115 [details] [diff] [review]
use different layout/gfx space conversion
Comment 23 tor 2004-04-19 11:41:10 PDT
I don't suppose anyone lurking in this bug happens to know about font
substitution and uniscribe, do they?  On the site
http://tdil.mit.gov.in/hindi_site/homepage.htm the unisribe code ends
up in problems because ScriptShape can't find a glyph it wants in the
chosen font (though it seems the right fonts are installed because
IE and a mozilla with tree font code look fine) and returns an error.

Comment 24 Greg K. 2004-06-03 06:46:28 PDT
As with bug 121540 for Mac, this bug may depend on bug 157967.
Comment 25 Sukh 2005-01-02 11:14:13 PST
Any update on this?  I've been waiting for quite a while to see when the issues
with Indic text on Windows will be fixed.  If anyone has any updates please let
me know.

Thanks!
Comment 26 Jungshik Shin 2005-05-17 19:21:40 PDT
*** Bug 294579 has been marked as a duplicate of this bug. ***
Comment 27 Satish 2005-07-17 23:34:15 PDT
Telugu font rendering not proper with Firefox 1.0.4 under Windows XP. The
combination of letters (especially Vattulu) breaks.

Here is a page for seeing some Telugu text.

http://nosaku.blogspot.com

The text is fine with "Pango enabled Firefox 1.0.4 build" on Linux. But not
proper in the Firefox 1.0.4 on Windows XP (and 2000 too).

Any work going on on this?

Satish.
Comment 28 H P Nadig 2007-04-13 14:23:14 PDT
https://bugzilla.mozilla.org/show_bug.cgi?id=377345 is the same as this. 
Comment 29 Simon Montagu :smontagu 2007-04-18 01:04:45 PDT
*** Bug 377345 has been marked as a duplicate of this bug. ***
Comment 30 Simon Montagu :smontagu 2007-06-03 02:55:03 PDT

*** This bug has been marked as a duplicate of bug 322938 ***

Note You need to log in before you can comment on or make changes to this bug.