252033 - gtk2+xft text clipping problem: only first word of line is rendered, rest of text displays on mouseover

Status: RESOLVED FIXED

(Core :: Web Painting, defect)

Description

[Kurt Erickson]

User-Agent:       Mozilla/5.0 (X11; U; Linux ppc; en-US; rv:1.7) Gecko/20040621 Firefox/0.9
Build Identifier: Mozilla/5.0 (X11; U; Linux ppc; en-US; rv:1.7) Gecko/20040621 Firefox/0.9

In the page at www.theregister.co.uk, most of the text in the headline links is
invisible and isn't rendered, until it is moused over.

Reproducible: Always
Steps to Reproduce:
1. Go to theregister.co.uk
2.
3.

Actual Results:  
Most of the headline text was missing, and would only appear on mouseover.

Expected Results:  
Displayed all text without mouseover.

I'm using the truetype fonts from corefonts.sourceforge.net. Anti-aliasing is
off, and I'm running fedora core 2. I can provide screenshots if you want them.

Comment 1

[Boris Zbarsky [:bzbarsky]]

Looks GTK2-specific....

Comment 2

[Christopher Blizzard (:blizzard)]

Might be one of the usual suspect xft bugs.

Comment 3

[Kevin Buhr]

I've been able to duplicate this on PC Linux, so changing hardware to All. 
Also, The Register seems to have changed their layout, so I've changed the URL
from http://www.theregister.co.uk to a different page that exhibits the problem.

This only affects Mozilla or Firefox compiled with gtk2 and xft.  (Non-xft gtk2
builds are not affected.)

Steps to reproduce:
1.  Go to http://www.wicell.org/forresearchers/
2.  Observe that in the red menu on the left, only the first word
    of each line is drawn (e.g., for the fifth item, instead of:
       FAQs - Culturing
       Human ES Cells
    only the text:
       FAQs
       Human
    appears.)
3.  Pass the mouse over each item in turn, and the full text is
    redrawn with an underline.  As the mouse cursor leaves the
    item, the underline disappears but leaves the full text.
4.  Now, give another window focus, and (usually) all but the first
    word of each line disappears again.

Verified with xft-enabled Seamonkey nightly and Firefox aviary build pulled
20041215.  The non-xft Seamonkey nightly (and a non-xft Firefox aviary pulled
last week) does not exhibit the problem.

I'll try to develop a simple testcase.

Comment 4

[Kevin Buhr]

Note that in bug 199564 (which seems to combine several different bugs into one
report) the reporter added a comment with a testcase that seems to illustrate
the present problem:

        http://www.mozilla.org/projects/xbl/test2/test.html

Moving the mouse around in that page with an xft build will periodically cause
paragraphs to be redrawn with only the first word of each line visible.

Comment 5

[Christopher Blizzard (:blizzard)]

I can't reproduce this with an xft build and firefox off the trunk on FC3.

Comment 6

[Kevin Buhr]

I can no longer reproduce exactly the same problem on either a recent (Feb 13)
or the old (Dec 15) build where I first verified this, so a library or other
configuration change has changed something for me.  (Both builds are off the
AVIARY_1_0 branch.)

Anyway, I can still reproduce with the testcase:

     http://www.wicell.org/forresearchers/

by doing a Ctrl-F5 force reload on the page, and the red paragraphs on the left
are cut off.

I took a bit of a hiatus, but I renew my intention to develop a simple testcase
for this to track down if it's a library problem or what.

Comment 7

[Kevin Buhr]

Whoops.  It looks like the www.wicell.org testcase depends on a subtle timing
issue with image loading, so it won't reliably reproduce for people, and I can't
reproduce it with a local copy.

The good news is that I *am* able to reproduce the problem with the testcase:

     http://www.mozilla.org/projects/xbl/test2/test.html

with a *lot* of mouse movement, but it is much harder to reproduce than before.

I'll keep looking.

Comment 8

[Kevin Buhr]

I finally found a page where I could reliably reproduce this problem and boiled
it down to the following testcase:

     http://www.asaurus.net/~buhr/mozilla-testcases/252033-1.html

which only displays the first word of each line/link on a gtk2+xft build off the
current aviary branch.

It appears that this is actually an XFree86 server bug in the XRender extension:
 In mozilla gfx/src/gtk/nsFontMetricsXft.cpp, we ultimately send the glyphs out
to the X server in nsAutoDrawSpecBuffer::Flush() via the XftDrawGlyphFontSpec()
function.  The Xft library packs the glyphs into "elts" (GLYPHELTs in the
XRender specification), runs of glyphs rendered at their natural spacing, and
sends them via an XRenderCompositeString8 request to the X server.  However, it
appears that if an elt consists only of a zero-area glyph (like a space), the
request's remaining elts won't be rendered.  This normally isn't a problem when
glyphs are being rendered at natural spacing, since the zero-area glyphs will be
part of a multiglyph elt.  But, if non-standard letter or word spacing is in
effect, a space might be the only thing in an elt, and the rest of the line (or
link or whatever) will be dropped.

I'm still trying to track down exactly where the bug is, and then I'll send it
upstream.  However, it may be quite some time before the fix percolates through
our userbase (though this is mitigated by the fact that xft builds aren't yet
particularly standard).  Perhaps we want to patch around the buggy behaviour
ourselves?

Anyway, I'll attach the patch I developed during testing.

Comment 9

[Kevin Buhr]

Attachment: patch to work around XRender bug

Here's a patch that ships out runs of glyphs with area while skipping glyphs
with no area.  This isn't ideal performancewise---we can identify these
zero-area glyphs earlier, drop them from our nsAutoDrawSpecBuffer entirely, and
ship the whole line out with a single XftDrawGlyphFontSpec call---but I'll need
a little more time to work on a better patch, assuming we want to work around
this XRender bug in the first place.

Comment 10

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Comment on attachment 193199 [details] [diff] [review]
patch to work around XRender bug

I think we should take a workaround for the XRENDER bug.

Comment 11

[Kevin Buhr]

By the way, for those having trouble duplicating this bug, antialiasing may need
to be shut off for the font in question.  (I had shut it off for small fonts.).
 In particular, the following "$HOME/.fonts.conf" file might help trigger the
buggy behaviour:

<?xml version="1.0"?>
<!DOCTYPE fontconfig SYSTEM "fonts.dtd">
<fontconfig>
  <match target="font">
    <edit name="antialias">
      <bool>false</bool>
    </edit>
  </match>
</fontconfig>

Comment 12

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Do spaces count as "glyphs with no area"?

Comment 13

[Kevin Buhr]

Yes, more or less.

The area referred to here is the width times height of the drawable content of
the glyph.

A glyph with zero area (the kind causing the problem) has no drawable content
and so must be *some* sort of space.  On the other hand, a space in a font might
have non-zero area but with associated drawable content that happens to be
empty.  For example, when antialiasing is turned on, it appears that the derived
font will have a space glyph that has area.  However, these kinds of
spaces-with-area don't tickle the bug and so don't need to be worked around.

Does that answer your question?

Comment 14

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Yes.

Then this patch will cause us to do *a lot* more XRender calls, right?

Comment 15

[Kevin Buhr]

You are correct, there will be an XRender call for every word instead of every line.

As mentioned in comment #9, another approach is to identify the zero-area glyphs
earlier (the existing code is already fetching the width and height of every
glyph to get spacing right) and simply drop them from the autodraw buffer.  This
would reduce the number of XRender calls back to pre-patch levels.  The ultimate
impact on performance is still not clear---the patch will still result in the
XRender request being "packaged" in a different manner (one "elt" per word
instead of one "elt" per line).

It may be a couple of weeks before I can come up with such an alternate patch,
though.

Comment 16

[Robert O'Callahan (:roc) (email my personal email if necessary)]

I don't believe this is a regression, and I'm afraid of the performance impact
of issuing so many more XRender calls, so I think we'll have to not take this
for branch. And this problem will become someone else's when we move to cairo, I
guess. (Does cairo have this workaround? It probably needs it!)

Comment 17

[uamjet602]

FYI, this problem resolved itself in my case when I updated my X server (xorg)
to the latest CVS version today.

Comment 18

[Felix Wiemann]

FYI, this problem has gone away for me when upgrading from gtk+-2.6.x to gtk+-2.8.8.

Comment 19

[Felix Wiemann]

Sorry -- turns out it hasn't completely gone away with gtk+-2.8.  However, I see the problem less often now.

Comment 20

[Ilja Sekler]

(In reply to comment #16)
> I'm afraid of the performance impact of issuing so many more XRender calls, so
> I think we'll have to not take this for branch.

I've built Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1b2) Gecko/20060901 Firefox/2.0b2 once with this patch and once without and there is absolutely no perceptible difference in performance on a 1,7GHz Ahlon machine (both builds are *very* fast) running Fedora Core 5, if I've got the point. And of course - the patch fixes an utterly annoying bug.

> (Does cairo have this workaround? It probably needs it!)

https://bugzilla.mozilla.org/show_bug.cgi?id=350111#c4

Comment 21

[Jon Hurst]

(In reply to comment #8)
> I'm still trying to track down exactly where the bug is, and then I'll send it
> upstream.

Do you have an upstream bug number?

Comment 22

[Oleg Romashin (:romaxa)]

Ok, Probably we will not use gfx/src/gtk anymore....

Could you chekc the same problem with cairo builds?

Comment 23

[Robert O'Callahan (:roc) (email my personal email if necessary)]

This bug is obsolete now, all this code is completely gone/changed.

Comment 24

[Ilja Sekler]

(In reply to comment #23)
> This bug is obsolete now, all this code is completely gone/changed.
> 

But the problem as such persists. I don't have a patched branch build at the moment, but I remember, that now the patch doesn't work at all. With the actual Xorg releases used in Fedora 7 and in Ubuntu 7.04, the text clipping depends on Xorg driver used.

nv: text is clipped

radeon: text is clipped

nvidia: text is *not* clipped, even without the patch

i810: text is *not* clipped, even without the patch

The new thing is, that the current trunk is also affected (tested just now with Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a8pre) Gecko/2007091404 Minefield/3.0a8pre). Shall I file a new bug about the same symptoms?

Comment 25

[Kevin Buhr]

No, don't file a new bug.  Are you able to reproduce the bug using the page in comment #8 with the "nv" and "radeon" drivers?  And is the symptom exactly the same, namely that only the first word of each list item is shown?

Comment 26

[Ilja Sekler]

(In reply to comment #25)
> Are you able to reproduce the bug using the page in
> comment #8 with the "nv" and "radeon" drivers?

A bulletproof YES using "nv", just tested on Ubuntu 7.04 with a fresh custom branch build with the patch applied, a two days older one without the patch as well with the newest trunk nightly.

For "radeon" and "i810" I have to rely on testing by a colleague of mine. It dates from 2007-06-24 only with the Firefox version shipped with Ubuntu 7.04 (AFAIK it doesn't include the patch).

> And is the symptom exactly the same, namely that only the
> first word of each list item is shown? 

Yes, it is.

Comment 27

[Kevin Buhr]

It's been a long time since I've seen this bug, and now I can't seem to reproduce it using the "nv" driver under Debian Etch.  That could be for a whole host of reasons, of course.  If your bug 350111 reproduction notes are relevant, it may be simply that FreeType on Debian ships with the bytecode interpreter disabled.

What .mozconfig (or options to "configure") are you using for your build?  There's an essentially identical block of code in "gfx/src/cairo/nsFontMetricsXft.cpp" starting around line 2152.  If you're building a "cairo-gtk2" build (or, more directly, if that file is being compiled in your build directory), that may be the version that's being used, so that file would need to be patched, too.

Comment 28

[Ilja Sekler]

(In reply to comment #27)
> It's been a long time since I've seen this bug, and now I can't seem to
> reproduce it using the "nv" driver under Debian Etch.  That could be for a
> whole host of reasons, of course.  If your bug 350111 reproduction notes are
> relevant, it may be simply that FreeType on Debian ships with the bytecode
> interpreter disabled.

Yes, they are still valid and reflect the setup of font environment I prefer since years on Linux, but there is a new aspect with Xorg 7.2 (and possibly already with 7.1) that the bug strikes Gecko with some video drivers and with some ("nvidia", "i810") not, independently from the patch. Etch ships Xorg 7.0 AFAIK, it may be too old to be able reproduce the bug that affects users of more "bleeding edge" mainstream desktop distributions.

It is also new that the current trunk is affected (but not in the way it was in bug 350111: only the displayed content is affected, not the GUI) and that the patch in its present shape doesn't help.
 
> What .mozconfig (or options to "configure") are you using for your build? 

For MOZILLA_1_8_BRANCH:

. $topsrcdir/browser/config/mozconfig

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox

ac_add_options --disable-tests
ac_add_options --disable-debug
ac_add_options --enable-optimize=-Os
ac_add_options --enable-default-toolkit=gtk2
ac_add_options --enable-xft
ac_add_options --disable-freetype2
ac_add_options --enable-svg
ac_add_options --enable-canvas
ac_add_options --enable-pango
ac_add_options --enable-official-branding

(the last option for my personal pleasure as I don't redistribute the builds, it makes it also easier to distinguish between simultaneously running branch and trunk builds).

For the trunk:

. $topsrcdir/browser/config/mozconfig

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/firefox

ac_add_options --disable-debug
ac_add_options --enable-static
ac_add_options --disable-shared
ac_add_options --disable-tests
ac_add_options --disable-libxul
ac_add_options --enable-optimize
ac_add_options --enable-default-toolkit=cairo-gtk2
ac_add_options --enable-svg

For the actual testing I used an official trunk nightly, not a homebrew build.

> There's an essentially identical block of code in
> "gfx/src/cairo/nsFontMetricsXft.cpp" starting around line 2152.  If you're
> building a "cairo-gtk2" build (or, more directly, if that file is being
> compiled in your build directory), that may be the version that's being used,
> so that file would need to be patched, too.

I am building traditional "gtk2" builds from the branch, but I'll patch "gfx/src/cairo/nsFontMetricsXft.cpp" in the same way, test and report.

Comment 29

[Ilja Sekler]

Attachment: screenshot of a patched branch gtk2 build on Ubuntu Gutsy (from a live CD, driver "nv")

Firefox 2.0.0.7pre (gtk2) with patch applied additionally to "gfx/src/cairo/nsFontMetricsXft.cpp" (of course, it didn't have any effect on a gtk2 build) on Ubuntu Gutsy daily build http://cdimage.ubuntu.com/daily-live/current/ 14-Sep-2007 09:13 (from a live CD). Xorg 7.2, driver "nv" -> text is clipped.

Ubuntu ships libfreetype6 with bytecode interpreter enabled. The font rendering is set to "Monochrome" in gnome-appearance-properties, Arial, Times New Roman and Courier New are installed locally for a prettier look.

Comment 30

[Ilja Sekler]

Attachment: screenshot of minefield on Ubuntu Gutsy (from a liveCD)

The same with the official Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a8pre) Gecko/2007091404 Minefield/3.0a8pre -> text is clipped.

Comment 31

[Kevin Buhr]

Okay, I've finally been able to reproduce this bug on Feisty, both using the stock Firefox 2.0.0.3+1-0ubuntu2 (no patch) and using a fresh branch build (with and without the patch applied to both copies of nsFontMetricsXft.cpp).

I'll work on it.

Comment 32

[Robert O'Callahan (:roc) (email my personal email if necessary)]

Is this bug present on trunk or not? I can't tell from the comments.

Comment 33

[Ilja Sekler]

(In reply to comment #32)
> Is this bug present on trunk or not?

Yes, it is. The screenshot in comment #30 illustrates it unmistakably.

Comment 34

[Kevin Buhr]

On the branch, the issue is that "--enable-pango" on a "gtk2" build switches from using nsFontMetricsXft.cpp to nsFontMetricsPango.cpp (both in gfx/src/gtk) so the patch has no effect.  The problem disappears for a *patched* branch build if Pango is disabled at compile time or runtime (by setting MOZ_DISABLE_PANGO=1).

The runtime setting isn't a useful workaround:  in stock Ubuntu, Firefox doesn't have the patch, so even disabling Pango won't fix the problem; and even for builds where it does, disabling Pango can have undesirable effects on font rendering.

A Pango workaround is probably possible, but seems messy.  Robert, are you interested in a branch workaround at this point?

On a trunk "cairo-gtk2" build, I suspect that the patch, if applied to "gfx/src/cairo/nsFontMetricsXft.cpp", would be effective (I don't think Ilja tested this particular case), but I need to build a trunk to find out.

I'm really out of touch with font stuff:  is there a roadmap somewhere showing where we're moving with all this GTK2/Pango/Cairo/Thebes/etc. and how fast we're trying to get there?

Comment 35

[Kevin Buhr]

Never mind what I said about the trunk.  I see now what Robert meant about all the code being gone/changed.  However, I think a trunk workaround will probably be relatively easy (and computationally inexpensive), if I understand what I'm seeing correctly.  Unfortunately, I can't work on this anymore today, and it's quite possible I won't get back to it for a couple of weeks.

Robert, I guess this bug should be restricted to the workaround in the above patch, and if I'm correct that both "gfx/src/gtk/nsFontMetricsXft.cpp" and "gfx/src/cairo/nsFontMetricsXft.cpp" are dead/dying code paths, then this bug should be resolved as WONTFIX.

I assume that a workaround for the trunk is desirable, and I'll open a new bug for that, assign it to me, and try to collect together all the certain duplicates.

Do you think a workaround for Pango-enabled branch builds is worthwhile?  If so, I'd propose opening a separate bug for that.

Finally, I'd propose opening a tracking bug for all bugs that appear to involve the same underlying XRender bug.  There are open bugs and testcases all over the place and multiple code paths that tickle it all requiring separate workarounds.  It would be nice to have a reference all in one place.

Sound okay?

Comment 36

[Robert O'Callahan (:roc) (email my personal email if necessary)]

That all sounds good.

The trunk code that needs to be patched is probably in cairo-xlib-surface.c --- i.e., it's really a cairo issue. Look at the code related to show_glyphs.

Comment 37

[Kevin Buhr]

Well, a couple of weeks turned into 7 months.  Sorry about that.

I am not able to duplicate the problem on the latest stable Ubuntu (8.04 AKA Hardy Heron) with the "nv" driver and antialiasing shut off, on any of:

1) the stock Ubuntu Firefox 3 package (firefox-3.0 3.0+nobinonly-0ubuntu0.8.04.1)
2) the old branch and trunk builds I was able to duplicate the problem on, as referenced in comment #31
3) a recent Seamonkey build.

The X server bug may have been fixed, at least in "nv", or else a Cairo workaround (as documented in bug 350111) may be fixing the problem.

Ilja, are you able to duplicate this behavior on the current trunk on a distribution of recent vintage?

Comment 39

[Ilja Sekler]

(In reply to comment #37)

> Ilja, are you able to duplicate this behavior on the current trunk on a
> distribution of recent vintage?

The issue has gone. I could not test with the "radeon" driver yet, but I can confirm your results not only for Ubuntu 8.04 but also for Fedora 9 Live CD with "nv" using both stock Firefox packages and the latest official trunk and 1.8 branch builds.

I'll have access to a computer with a Radeon graphics card and Ubuntu 8.04 installed probably later this week and report, if I manage to reproduce the bug there.

Comment 40

[Ilja Sekler]

(In reply to comment #39)
 
> I'll have access to a computer with a Radeon graphics card and Ubuntu 8.04
> installed probably later this week and report, if I manage to reproduce the 
> bug there.

Just for the sake of completeness: no text clipping with the "radeon" driver either.

Comment 41

[Zack Weinberg (:zwol)]

This bug hasn't been touched for three years, the font code has changed substantially meanwhile, and all the test cases seem to have been taken down.  Can anyone still reproduce?  If so, please *attach* a test case to this bug so we don't lose it in the future.

Comment 42

[Jeff Muizelaar [:jrmuizel]]

Seems fixed.