88588 - directional formatting characters cause spaces on Linux

Status: RESOLVED WORKSFORME

(Core :: Layout: Text and Fonts, defect)

Description

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

On Linux (maybe only after the fix to bug 83958, but I'm not sure), directional
formatting characters cause extra spaces in the document in some cases.  I'm not
completely sure what the pattern is.

To reproduce:
 * load http://www.people.fas.harvard.edu/~dbaron/css/test/bidi2_charcode

Bug appears on:
 * System with RedHat 6.2 default english install, optimized build on 0.9.2
branch after landing bug 83958.  (Also appears on trunk.)

Bug does not appear on:
 * Windows 2000, default US-English install, same builds (works perfectly)

Could not be tested on:
 * MacOS 9.1, default US-English install, since bidi reordering is basically
completely broken (although slightly differently before hyatt's style landing
and after the fix for bug 83958).  However, many of the formatting characters
appear as '?'.  I assume the Mac problems are covered by bug 80577.

Comment 1

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Attachment: screenshot of bug

Comment 2

[tenthumbs]

If you set NS_FONT_DEBUG=3 on linux, you'll see that mozilla does not
treat U+202B specially. It attempts to find it in a font. If mozilla
can't find the glyph (perhaps the usual case) then it goes into its
substitution drill and usually comes up with a space. If mozilla does
find the glyph, it's happy. Hopefilly that glyph is zero size.

So, whether or not you see extra spaces depends on exactly which fonts
you have. In fact, mozilla treats all the formatting characters, as well
as such things like emdash and endash, as normal characters. Rather
silly if you ask me.

Comment 3

[tenthumbs]

I used an older nightly so the bidi stuff isn't right but the attached
screenshot shows what happens if certain fonts do not exist. In my case, it's
 -altsys-caslon-medium-r-normal--14-*-0-0-p-*-iso10646-1
which is about the only font I have that has a U+202B glyph.

Comment 4

[tenthumbs]

Attachment: PNG example of presence or absence of fonts

Comment 5

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

We shouldn't even be sending the characters that far, though.  I thought the
StripBidiControlCharacters added for bug 83958 would fix that, but apparently
not.  (It did fix a similar problem on Windows, though.)

Comment 6

[Lina Kemmel]

The "spaces" you see are probably result of text measurement with formatting 
characters. We remove those characters before drawing, but not before 
measurement. The problem doesn't appear on W2K, since the latter has for sure 
fonts with appropriate glyphs, while Linux - does not. (On W2K, with the build 
before http://bugzilla.mozilla.org/show_bug.cgi?id=83958, formatting codes are 
displayed, but definitely occupy zero-width space.)

However, it make sense to add complex fix for (or before) text measurement 
taking into account also Arabic shaping (see 
http://bugzilla.mozilla.org/show_bug.cgi?id=74998).

Comment 7

[Lina Kemmel]

To be more precise: W2K seems to know to give zero width to BIDI control 
characters (as opposed to what I previously said - no matter, whether this 
information is retrieved from the font or set internally from the OS itself).

Comment 8

[Lina Kemmel]

dbaron: I tried to skip over BIDI controls while measuring text, and currently
see on Linux, RH 7.1, only 1 excessive space produced by
<p class="rtl">&#x202e;IHG&#x202b;DEF&#x202c;CBA&#x202c;</p>
the rest of your testcase looks fine.

Comment 9

[Lina Kemmel]

Attachment: Temporary patch

Comment 10

[Zach Lipton [:zach]]

Mass-move all BiDi Hebrew and Arabic qa to me, [e-mail]. 
Thank you Gilad for your service to this component, and best of luck to you 
in the future.

Sholom. 

Comment 11

[Simon Montagu :smontagu]

Marking WORKSFORME after testing on SuSE 6.4 with a 20011114+ build (after the
checkin on bug 36163). If anyone still sees this on another system, please reopen.

Comment 12

[David Baron :dbaron: (⌚️UTC-5, no longer working on Mozilla)]

Reopening.  Still looks pretty much the same on my RedHat 7.1 box with a bunch
of TrueType fonts installed.

Comment 13

[Lina Kemmel]

dbaron, which build are you using? i've tried 20011117 build, and
http://www.people.fas.harvard.edu/~dbaron/css/test/bidi2_charcode looks just
fine (using TTF arial).

Comment 14

[Lina Kemmel]

I tested on RedHat 7.1.

Comment 15

[Simon Montagu :smontagu]

dbaron, are you still seeing this problem on your system? It might have been
fixed by bug 138097.

Comment 16

[Simon Montagu :smontagu]

This has been WORKSFORME for ages. Marking in-testsuite+ because most of the tests in layout/reftests/bidi depend on directional formatting characters not having any visual effect.