619286 - ZWNJ and ZWJ affect letterspacing

Status: RESOLVED FIXED

(Core :: Layout: Text and Fonts, defect)

Description

[j. 'mach' wust]

User-Agent:       Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:2.0b7) Gecko/20100101 Firefox/4.0b7
Build Identifier: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:2.0b7) Gecko/20100101 Firefox/4.0b7

The characters ZWNJ and ZWJ are supposed not to affect letterspacing, just like other zero width characters such as the SHY. This bug only surfaces when letterspacing is increased. On Firefox 3.6, ZWNJ and ZWJ would not affect letterspacing. On Firefox 4 beta, they do.

Reproducible: Always

Steps to Reproduce:
1. Visit any page with a ZWNJ/ZWJ in text with increased letterspacing.

Actual Results:  
In text with increased letterspacing, ZWNJ and ZWJ affect letterspacing.

Expected Results:  
ZWNJ and ZWJ should not affect letterspacing (just like the SHY which continues to work fine, or just like on Firefox 3.6).

When the ZWJ is hardcoded into a GSUB rule that substitutes the ZWJ by a ligature, the letterspacing is not affected. However, as per Unicode's implementation notes, this hardcoding is only necessary for the ZWJ in those cases where ligatures are possible. In the case of the ZWNJ, such a hardcoding is not necessary. In the case of a ZWJ or a ZWNJ in arbitrary places, hardcoding is impossible. But still, these cases should be displayed correctly.

This bug is quite similar to https://bugzilla.mozilla.org/show_bug.cgi?id=253143 but it affects Mac OS, not Windows XP, and Firefox 4 beta.

Note that this bug also affects Ubuntu Linux, although there are additional issues with the ZWJ and with ligatures. I don't know whether Windows systems are also affected.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Is this coming from the switch to harfbuzz?

Comment 2

[Jonathan Kew [:jfkthame]]

(In reply to comment #1)
> Is this coming from the switch to harfbuzz?

It's not simply a harfbuzz issue, as setting gfx.font_rendering.harfbuzz.level=0 doesn't make any difference. But it could still be related to the refactoring of font backends that has gone on in association with harfbuzz integration.

I'm able to reproduce the issue here with the testcase given, but it does not occur with other fonts that I've tried. Will investigate further...

Comment 3

[j. 'mach' wust]

I hope it's not a bug of the testcase font. I have done some quick tests with the Google font previewer, by copying «Zeit‌zone Benut‍zer» (Code: Zeit‌zone Benut‍zer) for instance and then increasing the letterspacing with the spacing slider. In all fonts I've tried, the ZWNJ and ZWJ would affect the increased spacing. Try for instance here:

http://code.google.com/webfonts/preview#font-family=Droid+Sans

Additionally, any letter that is preceded by the ZWJ will not get the proper font, unless the font has a ZWJ character! Is this a known bug? It is appearently not an issue of the Google font previewer. Other browsers do not show this odd behaviour, and the odd behaviour persists when I copy the font to my server:

http://unifraktur.sourceforge.net/testcases/zwj_test.html

Comment 4

[Jonathan Kew [:jfkthame]]

(In reply to comment #3)
> I hope it's not a bug of the testcase font.

I don't think it is - I think it's a bug in our handling of the "default-ignorable" characters such as ZWJ and ZWNJ, and will be fixed along with bug 618870. (Because the CompressedGlyph::SetMissing function is actually used for default-ignorables as well as missing glyphs.)

> Additionally, any letter that is preceded by the ZWJ will not get the proper
> font, unless the font has a ZWJ character! Is this a known bug?

This occurs because our font-matching code tries to ensure that ZWJ is processed using the same font as the adjacent characters, so that there is the possibility of OpenType lookups (or other shaping behavior) being applied. So, for example, if the default font is a standard Latin font, but the page includes a run of Arabic or Indic text with embedded ZWJ characters, we want font-matching to keep the ZWJ in the same font as the surrounding letters *even though* it may be supported in the primary (Latin) font. But then, if your font *doesn't* have ZWJ, and so we are forced to fall back to something else, that fallback propagates to the next character as well.

I agree the behavior you get in this situation is unfortunate; please file a separate bug on that, and we'll see if there's some way to get the best of both worlds.

Comment 5

[j. 'mach' wust]

(In reply to comment #4)
> please file a separate bug on that, and we'll see if there's
> some way to get the best of both worlds.

<a href="https://bugzilla.mozilla.org/show_bug.cgi?id=619511">Bug 619511 - ZWJ may affect the font of the following character</a>

It's great to know that you really know how to fix such things – I just stumble upon the bugs and can do nothing but report them!

Comment 6

[Jonathan Kew [:jfkthame]]

I believe this has been fixed by the patch for bug 618870.