166072 - UTF-8 pages load slowly when many fonts are installed on xfs

Status: RESOLVED WONTFIX

(Core :: Internationalization, defect)

Description

[S. A. K.]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.1) Gecko/20020827

I just installed a bunch of ttf fonts on my Redhat 7.3 machine (xlsfonts | wc -l
--> 5731).  Now, when a UTF-8 encoded page is loaded in mozilla, it takes a long
time (about a minute) to load.  Before I load a UTF-8 page, gtop reports that
mozilla-bin is using approx 136,000k of memory.  After the page is loaded, just
under 1,000,000k of memory.  Most of the time the pages eventually load. 
Sometimes, everything freezes up and a hard reboot is needed.  Removing the
fonts fixes the problem.  


Reproducible: Always

Steps to Reproduce:
1.  Install many ttf fonts to xfs.
2.  Load a UTF-8 encoded page

Actual Results:  
Page is loaded slowly, mozilla eats up memory.

Expected Results:  
Load page in a reasonable amount of time and use a reasonable amount of memory.
 I'm not sure if I "shouldn't" have so many fonts installed, but all of my other
applications seem to be able to cope with the extra installed fonts.

On a related note, when I go to
http://www.google.com/search?q=trademark&ie=UTF-8&oe=UTF-8 mozilla picks a wrong
font for the (tm) symbol (it shows something like a wingdings symbol). That page
loads slowly.  If I go to  http://www.google.com/search?q=trademark, the page
renders correctly and loads normally.

Comment 1

[Niklas Dougherty]

I believe Mozilla must search for fonts with usable code points to render such a
page as a UTF-8 test page. The fonts are usually defined in other apps.

The real bug would be if Mozilla makes a full search for every new instance, or
if it somehow makes a hash list of collected results.

Comment 2

[S. A. K.]

I have tried a few more things to help pin down this problem.

The smoke test unicode pages always seem to load in under 5 seconds with many
fonts installed:
http://www.unicode.org/iuc/iuc10/x-utf8.html
http://www.unicode.org/iuc/iuc10/x-ncr.html

I started up mozilla and visited the following pages in following order and took
note of their loading times (with many fonts):
http://www.columbia.edu/kermit/utf8.html -- just under 2min
http://www.google.com/search?q=trademark&ie=UTF-8&oe=UTF-8 -- just over 1min
http://www.ltg.ed.ac.uk/~richard/unicode-sample.html -- under 10 seconds

After restarting mozilla, I visited the three pages in reverse:
http://www.ltg.ed.ac.uk/~richard/unicode-sample.html -- just under 2min
http://www.google.com/search?q=trademark&ie=UTF-8&oe=UTF-8 -- just over 1min
http://www.columbia.edu/kermit/utf8.htm -- under 10 seconds

Another restart of mozilla:
http://www.google.com/search?q=trademark&ie=UTF-8&oe=UTF-8 -- 20 seconds
http://www.ltg.ed.ac.uk/~richard/unicode-sample.html -- 2.5 minutes
http://www.columbia.edu/kermit/utf8.html -- under 10 seconds

- Going back to a page that I have already been to always loads under 5-10 seconds.
- During all three of these runs, after a UTF-8 page was visited gtop was
reporting that mozilla-bin was using approx 900,000k of memory (although it
makes swings of about 200,000k above and below that).  Mozilla holds on to that
memory until I exit.
- I've tried these three "runs" of pages two times.  The second time I tried
each run, the I could predict when the page would be loaded within about 10
seconds---so this is easily reproducible on my machine.
- Saving a copy of the page and loading it off my own hard drive did not seem to
affect loading times.
- After removing many of the new fonts I installed (fewer fonts: xlsfonts | wc
-l --> 2693), all of the pages mentioned so far load under 10 seconds, many
under 5.  Memory usage increases but is at more reasonable levels, about 350,000k.

Also, the www.ltg.ed.ac.uk page above acts strangely.  I can make any line of
text change fonts by scrolling so it goes off the screen and scroll back to it.
 Even the plain line that says "This page contains characters from each of the
Unicode character blocks" changes fonts.  I did a "view source" on the page
(that took about 2 minutes to pull up even with the page already open).  The
special characters are also rendered in that window.  I can get the fonts to
change by scrolling in that window too.  This problem doesn't seem to be
dependent on the number of fonts installed---it still happens when I removed
many of the installed ttf fonts.  I haven't seen this happen on any other page.
 I'm somewhat new to this---should I open another bug for this "fonts change
when scrolling on some utf8 documents" problem?

Comment 3

[S. A. K.]

component to internationalization---made a mistake on the initial bug report.

Perhaps this bug is related to bug 118011?

Comment 4

[Boris Zbarsky [:bzbarsky]]

gtop's memory reporting is utterly wrong, since it adds the memory of multiple
threads together.. for accurate memory figures, please use regular 'top' and
look at the usage for _one_ of the mozilla-bin lines (don't add them together).

I suspect we're still balooning in size, but it'd be good to eliminate the
ambiguity here (did we just create more threads or did we really allocate more
memory?)

Comment 5

[S. A. K.]

Using top and http://www.columbia.edu/kermit/utf8.html

mozilla-bin on startup: ~21,000k
after loading page with many fonts installed: 165M
same page, fewer ttf fonts installed: ~38,000k

Comment 6

[Roy Yokoyama]

assign to shanjian

Comment 7

[Mike Young]

Attachment: testcase

I recently transferred my Win ttf fonts in to linux, and surprise surprise, but
when I did a google search (sent as UTF-8) mozilla sucked up 137MB of VM, 83MB
of RSS, and stalled for about a minute and a half while it churned away.

Attached is the single character that caused this entire problem. I broke down
the results page until I found this little bugger. From what I can determine
(just analyzing top and /proc/procID/status) the number of characters doesn't
matter. Just one upper level character causes this bug.

Comment 8

[kill this account]

The problem is that to find out what is in a iso10646 (unicode) core X font an app 
has to XLoadQueryFont the font which uses a lot of memory. (We could save some 
server memory by using XQueryFont). 

So when moz is searching for a font that can display that char it loads the font
(metrics) for iso10646 fonts so that it has a list of chars in the fonts.

If there is no font that supports the particular character then moz will eventually
load every font.

Comment 9

[kill this account]

you might consider using xft or the moz direct freetype code 
(http://www.mozilla.org/projects/fonts/unix/enabling_truetype.html) to access the 
truetype fonts 

Comment 10

[Mike Young]

I'm not exactly sure how the calls to X work, but shouldn't Moz release the
fonts when it no longer needs them (or finds that they don't fit) ?

Comment 11

[kill this account]

yes, it is possible for the char list to be stored in a ccmap and the font 
to be released.

As far as I know people who are working on Truetype (outline) font support
these days are working on client side (eg: Xft) not server side (eg: Xfs).

Comment 12

[Frank Tang]

shanjian is no longer working on mozilla for 2 years and these bugs are still
here. Mark them won't fix. If you want to reopen it, find a good owner first. 

Comment 13

[Travis Chase]

Mass Reassign Please excuse the spam

Comment 14

[Travis Chase]

Mass Re-opening Bugs Frank Tang Closed on Wensday March 02 for no reason, all
the spam is his fault feel free to tar and feather him

Comment 15

[Travis Chase]

Reassigning Franks old bugs to Jungshik Shin for triage - Sorry for spam

Comment 16

[Jungshik Shin]

Gfx-X11 only bug. we're not interested in doing further work