468200 - Arabic space is rendered incorrectly

Status: RESOLVED WORKSFORME

(Core :: Layout: Text and Fonts, defect)

Description

[arifkarim]

User-Agent:       Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Avant Browser; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.0.04506; eSobiSubscriber 2.0.4.16; .NET CLR 1.1.4322)
Build Identifier: Firefox 3.0 and later

I am using an arabic/urdu opentype font called: ''Alvi Nastaliq''. This font uses two types of spaces. One for the latin characters, which is the default space, and the other for the arabic/urdu characters.

Now see both spaces are rendered correctly in old Firefox (2.0):
http://i169.photobucket.com/albums/u218/arifkarim/b020.gif

and Internet Explorer 7:
http://i169.photobucket.com/albums/u218/arifkarim/b022.gif

but new Firefox (3.0) and later is creating major space rendering problems:
http://i169.photobucket.com/albums/u218/arifkarim/b021.gif

Strange thing is that only Firefox (3) on Windows is making such problems. Linux users have reported no strange rendering behaviour in Firefox (3).

I hope this problem would be solved soon enough...

Reproducible: Always

Steps to Reproduce:
Just download Alvi Nastaliq font:
http://www.fileden.com/files/2008/5/23/1926615/Alvi%20Nastaleeq.rar 

and check urself...

Comment 1

[Simon Montagu :smontagu]

I see this on Linux also.

Assuming that the font is using contextual substitution for the space character this might be tricky to fix, since we don't measure spaces in context, but cache the width of the space glyph in the font metrics.

Comment 2

[Jonathan Kew [:jfkthame]]

It's not quite as bad as that; the font does not use contextual substitution for the space, it has a simple non-contextual substitution in the 'ccmp' feature for Arabic script.

This will still require some thought; we currently measure the width of the space during font setup, and cache that in the font's Metrics structure, which means we're assuming the space has a constant width.

Comment 3

[Simon Montagu :smontagu]

Attachment: Testcase

Comment 4

[Jonathan Kew [:jfkthame]]

As a first approximation, we could fix this by having two space-width fields in the font metrics, one for LTR text and one for RTL, and choosing the appropriate width in gfxTextRun::SetSpaceGlyph() according to whether the TEXT_IS_RTL flag is set. However, this still leaves a couple of issues unanswered:

(1) How to measure a "right-to-left space".... in this particular case, we need to measure in the context of an Arabic-script run, but for other fonts it might be Hebrew or Thaana or another script. The various platform font implementations may need different approaches to this, as they may not allow us to explicitly specify the script when measuring a character.

(2) As implied by this, there could theoretically be many different script-specific spaces with different widths, so simply distinguishing LTR from RTL might not be sufficient. How general do we want to be? (IMO, we could reasonably ignore this aspect until we run across a real-life font where it matters.)

In addition to alternate space widths, we might also want to retrieve script-specific glyph IDs for the space, although I'm not sure this really matters; we can simply "impose" a different width on the default glyph.

Comment 5

[arifkarim]

Thnx to Both Simon And Jonathan for taking interest in this crucial matter. I still wonder why the old firefox 2.0 is rendering both arabic and latin spaces correctly:
http://i169.photobucket.com/albums/u218/arifkarim/b020.gif

Is there some sort of new standard applied in the new version?

I have the source of this font and I have used all types of possible connections. The ccmp table is not working for alternate space, neither the contextual substitution. I even tried using the kern table in volt to fix this issue, but the result is always the same: Firefox 3 always uses the default space charachter as the space for whole font!!! While the old school Firefox 2 and i IE7 is rendering completely fine....

Plz check out what has been changed in this new version of Firefox 3, regarding its spaces... THnx

Comment 6

[Jonathan Kew [:jfkthame]]

The issue is that Gecko does not process entire lines of text through the OpenType layout mechanism, it splits the text into whitespace-separated runs, processes each such word (or other fragment) in isolation, and caches the resulting glyphs for performance reasons.

So neither contextual substitution nor kerning can work for the space at present, because spaces are not measured or drawn as a normal part of the text run along with their surrounding characters. However, we may be able to handle the use of 'ccmp' for this (i.e., a non-contextual substitution, not dependent on the actual surrounding characters but only on the current script).

Comment 7

[Simon Montagu :smontagu]

I am the last person to minimize the importance of rendering text as expected. However, this bug is not "critical" or "major" as defined at https://bugzilla.mozilla.org/page.cgi?id=fields.html#bug_severity

Comment 8

[arifkarim]

Thnx Jonathan... As u suggested; I used the ccmp table to define an alternate space for only the arabic script, like this:
http://i169.photobucket.com/albums/u218/arifkarim/b034.gif

However the result was the same....  :(
Should I use a different code?

And To Simon: I checked ur testcase on Firefox 3. The second line was rendered correctly! How did u do it?

Comment 9

[Jonathan Kew [:jfkthame]]

No, it won't work at the moment; as I said earlier, Gecko just measures the width of the space once, when it first initializes the font, and then uses that value. But it may be possible for us to make it use the alternate width from the 'ccmp' substitution when processing a run of Arabic words; I have some thoughts on how we could implement this, so if I can find the time I'll try to develop a patch.

Comment 10

[Jonathan Kew [:jfkthame]]

In the test case, the second line explicitly uses the narrow "6-per-em" space character, not the normal space. In this particular font, that character maps to the same glyph as the 'ccmp' substitution for normal space in the Arabic script, although that is of course a font-specific detail; there's no reason to suppose that other fonts would use this particular width.

Comment 11

[arifkarim]

U are right Jonathan Kew. I have done many tests and come to conclusion that new Firefox 3s rendering engine has lost the true usage of different spaces in unicode. I tried all of the different unicode spaces given in this page:
http://www.cs.tut.fi/~jkorpela/chars/spaces.html

AND most of the (widht) spaces failed to work :(. Strangely they are all working fine in Firefox 2.0. I think its a major problem that unicode spaces are not rendering correctly in new Firefox builds. Therefore I am changing its severity back to major.

Comment 12

[Jonathan Kew [:jfkthame]]

I don't think I can agree with this. The question of how the various Unicode space characters (mostly in the U+200x range) are displayed is quite separate from the original issue raised in this bug, which concerns the script-specific variation of the standard space U+0020 by means of an OpenType glyph substitution.

If there is a general problem with the various Unicode "space" characters, this should be filed as a separate bug. However, I looked at the example page mentioned, and the various spaces worked fine using recent trunk builds on both OS X and Linux. Note that the results may depend on the fonts you have installed, as these are simply characters to be rendered from the selected font, and not all fonts include them (which means font fallback behaviour could be involved).

We could consider simulating the various widths of "space" when the font in use doesn't explicitly support them; that would also be a separate bug, if it's worth pursuing.

Comment 13

[arifkarim]

Thnx John... I did some more tests and found out that the problem occurs only when one tries to substitute unicode spaces! When I used ''6-em'' space directly, instead of usual space, it was rendering correctly in both latin and arabic scripts. But when I substituted it with the ''normal space'', then I got same problems mentioned above. It seems gecko engine is rendering words as whole and spaces as whole, which is completely conflicting with the standars of the opentype!

In opentype; a space is defined as a character, which has the same behaviour as all other charachters... I have no idea what firefox programmers have in their mind right now, and how much they are going to interchange the standards set by opentype for all languages...

Anyhow, I would ask: Should I post this problem as a seperat bug? Cause this problem is general and appears everytime u try to substitute spaces... Suprisingly it was not happening in Firefox 2 :)

Comment 14

[Negritas Sergiu, Desktop QA]

Hello! I have tried to reproduce the issue with firefox 97.0a1(2021-12-07) on Windows 10 unfortunately I wasn't able to reproduce the issue.

Marking this as RESOLVED-WORKSFORME since the issue isn't reproducible anymore and no activity has been done in the past 10 years.