190278 - With Xft, Mozilla only displays the last word in Hebrew sequences [missing/invisible lines of text] [text doesn't show up until highlighted]

Status: RESOLVED WORKSFORME

(Core Graveyard :: GFX: Gtk, defect)

Description

[Ilya Konstantinov]

Given a basic document with a sequence of Hebrew words, only the last one gets
drawn, e.g.:

[HEBREW] [HEBREW] [HEBREW] [ENGLISH] [ENGLISH] [ENGLISH]

gets drawn as (logically; ignoring BiDi since its irrelevant here):

[blank.] [blank.] [HEBREW] [ENGLISH] [ENGLISH] [ENGLISH]

(where [HEBREW] denotes a Hebrew word, [ENGLISH] denotes an English word and
[blank.] denotes a blank space whose width is the width of the original word
which was supposed to be drawn there)

Comment 1

[Ilya Konstantinov]

Attachment: Testcase

Comment 2

[Ilya Konstantinov]

Attachment: Screenshot

This screenshot shows the rendering of the testcase on my machine.

Comment 3

[Ilya Konstantinov]

Attachment: Screenshot

Strangely, if you keep one of the hidden words selected, it reveals it, and the
other hidden word as well.

Comment 4

[Christopher Blizzard (:blizzard)]

On my system your test case displays properly, which probably means it's related
to the particular font on your system.

Comment 5

[Ilya Konstantinov]

You're right. If I specify the entire text to be presented in a font which has
Hebrew glyphs, such as Arial, Courier New or Times New Roman, the Hebrew text
displays properly. However, if I specify it to be presented in a font without
Hebrew glyphs (either explicitly via font-family or implicitly by having it as
my default font in the Preferences), it fallbacks to a Hebrew-supporting font
(in a totally unexpected fashion, btw; it prefers Times New Roman for 'Courier'
and Courier New for 'Times') only on the last word of the sequence (as you see
in my screenshots).

Comment 6

[Noah Levitt]

I have the same problem, using mozilla-snapshot in debian sid. This was true of
the Jan 30 build and the Mar 03 build. (I had been waiting impatiently for a moz
update, and when it came and had the same bug, I was very disappointed, so I am
finally taking some action. :-)

I talked to keithp, and he initially suggested it might be an Xft problem, but I
tried a couple things that led us to believe it was not.

Three screenshots to follow, showing how strangely it behaves when I select
different parts of the text.

Comment 7

[Noah Levitt]

Attachment: there should be text in the blank spot

Comment 8

[Noah Levitt]

Attachment: with one part selected, you see a few more glyphs

Comment 9

[Noah Levitt]

Attachment: with another part selected

Comment 10

[Danny Colascione]

Might have something that's related. Pure English text, though. I'm getting
other weird Xft problems, but this is the most pronounced. Like this bug, I see
the additional text if I highlight it.

Comment 11

[Danny Colascione]

Attachment: correct rendering

This was made on a non-Xft build. Same font (used through XFree86 truetype
support). Moz 1.3.0.

Comment 12

[Danny Colascione]

Attachment: incorrect rendering

This is what it looks like on an Xft build.
http://www.mozilla.org/projects/xbl/test2/test.html

Comment 13

[Christopher Blizzard (:blizzard)]

Attachment: small testcase

This is a small testcase that shows the problem.  Make sure that it's encoded
with UTF-8.  For some reason if I download this from a web server it's always
encoded with ISO-8859-1.  You have to force the encoding from the View menu.

Comment 14

[Christopher Blizzard (:blizzard)]

Attachment: patch to xft that might fix this problem

Comment 15

[Noah Levitt]

This patch (actually latest Xft from cvs) fixes the bug for me.

Thank you blizzard and keithp!

Comment 16

[Christopher Blizzard (:blizzard)]

*** Bug 182650 has been marked as a duplicate of this bug. ***

Comment 17

[Christopher Blizzard (:blizzard)]

*** Bug 204390 has been marked as a duplicate of this bug. ***

Comment 18

[Christopher Blizzard (:blizzard)]

*** Bug 198675 has been marked as a duplicate of this bug. ***

Comment 19

[Christopher Blizzard (:blizzard)]

*** Bug 204502 has been marked as a duplicate of this bug. ***

Comment 20

[Christopher Blizzard (:blizzard)]

*** Bug 205706 has been marked as a duplicate of this bug. ***

Comment 21

[Christopher Blizzard (:blizzard)]

*** Bug 207018 has been marked as a duplicate of this bug. ***

Comment 22

[Moses Lei]

Could we update the summary to reflect the actual nature of the bug instead of
one of the many symptoms? The huge number of dupes is because this summary
doesn't seem to apply in most cases-- not all of us use Hebrew.

What about: "Xft does not show certain UTF-8 glyphs until highlighted"

Comment 23

[Christopher Blizzard (:blizzard)]

*** Bug 209150 has been marked as a duplicate of this bug. ***

Comment 24

[Christopher Blizzard (:blizzard)]

*** Bug 209432 has been marked as a duplicate of this bug. ***

Comment 25

[Trever Adams]

This doesn't seem to be just a UTF-8 bug.  This seems to happen if you have a
"paragraph" (not sure if this is a <P> set or not) that starts with several
capital letters.  See usatoday.com under news briefs and under that on nation
(either one does the trick).  You will see this same bug completely in action. 
The first line is invisible.

Comment 26

[Christopher Blizzard (:blizzard)]

*** Bug 204861 has been marked as a duplicate of this bug. ***

Comment 27

[Cyd]

The problem was not fixed when I applied Blizzard's patch 121540 to libxft2
2.1.1 under XFree86 4.2 (the latest package versions in Debian unstable.)
However, updating to the CVS version of libxft2 (which already includes the
patch) and XFree86 4.3 (which is required to build CVS libxft2) fixed the problem.

Comment 28

[Christopher Blizzard (:blizzard)]

*** Bug 213797 has been marked as a duplicate of this bug. ***

Comment 29

[Tom Mraz]

Adjusting a summary a little bit to avoid dupes.

Comment 30

[Tom Mraz]

*** Bug 208720 has been marked as a duplicate of this bug. ***

Comment 31

[Tom Mraz]

*** Bug 200486 has been marked as a duplicate of this bug. ***

Comment 32

[Jungshik Shin]

This bug was fixed by blizzard's Xft patch, wasn't it? If it  was, we have to
release-note it. Could someone come up with a brief summary of the problem?
Have RedHat, SuSe and other Linux distribution vendors released a new Xft with
this patch? 

Comment 33

[Tom Mraz]

RedHat (which I use) ships the libxft only as part of XFree86-libs package so
I'll have to download Rawhide sources, see if the patch is there and if it's
there recompile it to test if the patch works. It will take some time.

As it's a pretty nasty bug couldn't it be workarounded by mozilla somehow?

Comment 34

[Tom Mraz]

OK. The XFree86-4.3.0-22.1 from Rawhide (contains a xft2-2.1.2 with the patch)
fixes the bug for me.

Comment 35

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

*** Bug 220072 has been marked as a duplicate of this bug. ***

Comment 36

[Christopher Blizzard (:blizzard)]

*** Bug 224038 has been marked as a duplicate of this bug. ***

Comment 37

[Boris Zbarsky [:bzbarsky]]

*** Bug 224679 has been marked as a duplicate of this bug. ***

Comment 38

[Christopher Blizzard (:blizzard)]

*** Bug 225623 has been marked as a duplicate of this bug. ***

Comment 39

[Christian :Biesinger (don't email me, ping me on IRC)]

*** Bug 205062 has been marked as a duplicate of this bug. ***

Comment 40

[Tejun Heo]

XFree86 4.3.0-0pre1v3 (debian package) + patched libXft2 doesn't make any
difference on mozilla-xft 1.5-3 (debian package). Many Korean texts are still
missing in the rendered page. Turning off "Allow documents to use other fonts"
options remedies the problem to some extent but not completely. If I missed
something, please point out. I used source checked out from xfree86 repository
as of 23th November and applied Blizzard's patch. TIA.

Comment 41

[Jungshik Shin]

Xft is not maintained at XFree86. For the CVS version of Xft, you'd better go to
http://freedesktop.org(go to http://freedesktop.org/Software/xlibs) Anyway, I'm
not sure what your problem is and whether that's the same as this bug. Can you
give a test case (or at least addresses of sites you have the trouble with)? 

Comment 42

[Tejun Heo]

The problem I've had closely matches description of this bug. In MOST pages,
Korean characters are not rendered properly, selecting or hovering over hiddent
texts tend to reveal them. Less frequently, there are pages with disappearing
western characters. The last such page I've seen was a longhorn preview page
I've followed from the Inquirer. If the URL makes any difference, please tell
me. I'll try to find the page again.

Turning on anti-aliasing or turning off "allow documents to use other fonts"
remedies the problem.

I've been seeing this this problem on all mozilla-based browsers w/ xft released
during the last six months. I'm attaching screenshots. (The screenshots are
taken with mozilla-xft 1.5-3 and the CVS snapshots of fontconfig/Xrender/Xft
from freedesktop.org as of 25th November.)

Comment 43

[Tejun Heo]

Attachment: google in Korean, several Korean characters are missing.

Comment 44

[Tejun Heo]

Attachment: The same page. Selecting makes all characters visible.

Comment 45

[Jungshik Shin]

Thanks for the screenshot.

I'm wondering how come I've never seen that (I google every  5 minutes). Anyway,
what fonts do you have specified for Korean and other langGroups? You wrote that
honoring the font specified in documents made a  difference so that your fonts
must have something to do with this.

Comment 46

[Tejun Heo]

I'm currently using "new gulim" taken from Windows XP (ngulim.ttf) for both
Serif and Sans-serif for Korean (12point, 12point min). I didn't modify any
other configuration. Using baekmook ttf fonts doesn't make any difference. IIRC,
I once also removed all windows fonts and installed only baekmook fonts. That
didn't make any different either.

Oh.. one thing I've noticed is that rendering failures usually occur after
abnormally wide white space characters. This can also be seen in the screenshots
I've posted.

Would posting fontdir file (ttf-commercial.scale) help?

Comment 47

[Jungshik Shin]

Thanks a lot for giving me details. Unfortunately, I have no clue what's going
on. For the last several months, I've almost exclusively used Un-series
true/opentype fonts (http://chem.skku.ac.kr/~wkpark ) for Korean text (although
I occasionally tested Baekmuk and Windows fonts). That's one difference, but it
doessn't sound like Un-series fonts are any different from others as far as the
symptom you described is concerned. (well, you may want to install them anyway
because they look much better than Baekmuk and look pretty good at smaller sizes
without embedded bitmaps ).

BTW, fonts.scale has nothing to do with Xft.  Anoter, btw, could graphic adaptor
and/or X server make a difference? Not likely, but there may be something.....

Comment 48

[Tejun Heo]

I have two almost identically configured debian sid boxes. One is P3 / radeon
VE. The other is P4 / radeon VE. I was using the latest 4.2 server until a week
ago and switched to 4.3.0-pre1v3 (deb http://www.schuldei.org/debian/bruby/ ./).
The switch didn't make any difference. :-(

So, is the symptom I'm seeing different from this bug? Should I open a new bug?
Am I the only one seeing this problem? I'm willing to supply as much information
as needed to resolve this one. Just tell me what's needed.

Comment 49

[Jungshik Shin]

What you've been experiencing appears to match the dsecription given by others,
but after blizzard's patch was applied, it seems to have disappeared for others.
However,  you still have a problem. Just in case(although not likely to change
anything), can you try XF86 4.3.99 (or whatever it's gonna be 4.4.  4.3.0pre1?
is rather old). As an added benefit, a nasty bug was fixed late in 4.3.9x cycle.
The bug triggerd Mozilla's crash whenever you visit a site with (a) flash
animation(s) with an IM server turned on.

Comment 50

[Moses Lei]

No activity from latest bug reporter since 2003-11-24. This is fixed in Debian
testing's libxft2 2.1.2-5 package. Recommend resolving MOVED or RESOLVED, since
(AFAICT) this patch has been picked up upstream.

Comment 51

[Prognathous]

Can anyone reproduce this bug with recent builds?

Prog.

Comment 52

[Christopher Blizzard (:blizzard)]

Wow, is this still open?  It's really an upstream bug, marking WORKSFORME since
upstream fixes will make it better.

Comment 53

[Christopher Blizzard (:blizzard)]

*** Bug 244437 has been marked as a duplicate of this bug. ***