701661 - Firefox maintain an open file descriptor for every single font...

Status: NEW

(Core :: Graphics, defect)

Description

[Ollivier Robert]

Attachment: FF8-fonts : output of lsof -p |grep Font

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.4 Safari/535.7

Steps to reproduce:

MacOS X 10.6.8, Firefox 8.0, beta channel, version from two days ago (9/11/2011)

Just ran lsof -p <firefox pid> on my machine to see how many network connections were open.


Actual results:

I just saw that FF keeps an open file descriptor for every single font in my system, awsting a *lot* of resources, it has currently 543 open fd for the fonts...


Expected results:

I do not expect any font to remain opened during the lifetime of the process.

Comment 1

[John Daggett (:jtd)]

> User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7
> (KHTML, like Gecko) Chrome/16.0.912.4 Safari/535.7

???

Comment 2

[Boris Zbarsky [:bzbarsky]]

> awsting a *lot* of resources

What sort?

> I do not expect any font to remain opened during the lifetime of the process.

Do you expect us to cache all the font data for all the fonts in memory instead?  That seems strictly worse, no?

Comment 3

[Ollivier Robert]

(In reply to John Daggett (:jtd) from comment #1)
> > User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7
> > (KHTML, like Gecko) Chrome/16.0.912.4 Safari/535.7
> 
> ???

The bug report was submitted with Chrome yes.

Comment 4

[Ollivier Robert]

(In reply to Boris Zbarsky (:bz) from comment #2)
> > awsting a *lot* of resources
> 
> What sort?

I wrote it, open file descriptors.
 
> > I do not expect any font to remain opened during the lifetime of the process.
> 
> Do you expect us to cache all the font data for all the fonts in memory
> instead?  That seems strictly worse, no?

Uh, do you really need to have a *open* handle on every single present font in a given system whether you use it or not?  You know that you can open, read and close a file during the lifetime of a process, right?  That you can also only cache fonts you need, right?

If you have a really good reason to keep all of them open, pray do tell, I am very interested.

Comment 5

[Boris Zbarsky [:bzbarsky]]

> I wrote it, open file descriptors.

Last I checked the limit on the number of open file descriptors on OSX was pretty high; 500 fds is not a big deal as far as I can tell.

> whether you use it or not

Given that CSS specifies per-character fallback, whether we use a font depends on the exact characters on the page...

> You know that you can open, read and close a file during the lifetime of a process,
> right?

Yes, and that takes time.

That said, this is worth looking into, which is why the bug is still open.  The first question is whether we're holding the fds or whether system libraries are...

Comment 6

[Jonathan Kew [:jfkthame]]

We don't open the font files directly, we access font data via the Core Graphics, Core Text and Cocoa APIs. In particular, we create a CGFontRef for each font in the system in order to read data such as character maps, and it seems likely that the OS keeps the file open as long as that font reference exists.

It would be possible to dispose of these font references and re-create them on demand whenever we need to use a given font, but this would add some overhead (I'm not sure how much, but we could measure this); it's also not clear to me that keeping the font references alive is actually a problem.

Comment 7

[Ollivier Robert]

(In reply to Jonathan Kew (:jfkthame) from comment #6)
> We don't open the font files directly, we access font data via the Core
> Graphics, Core Text and Cocoa APIs. In particular, we create a CGFontRef for
> each font in the system in order to read data such as character maps, and it
> seems likely that the OS keeps the file open as long as that font reference
> exists.

I just looked at Chrome and processes are also having open file descriptors for fonts but each one has between 2 and 10 more or less, not all of them.

> It would be possible to dispose of these font references and re-create them
> on demand whenever we need to use a given font, but this would add some
> overhead (I'm not sure how much, but we could measure this); it's also not
> clear to me that keeping the font references alive is actually a problem.

Firefox is often seen as wasting resources and memory (with good reasons), I'd say that anything that can change that is good, especially that you have taken measures to keep memory consumption down in 8 and later...

Comment 8

[Boris Zbarsky [:bzbarsky]]

I'm not sure I buy the argument that reducing open file descriptor count at the cost of more memory and/or CPU usage is a good move, even purely from the perception-management angle (e.g people are more likely to look at memory or CPU usage than they are to look at lsof).

If the open fd count can be reduced without any other costs, then sure.

Comment 9

[Wayne Mery (:wsmwk)]

is this more a footprint issue, than perf?

Comment 10

[Pascal Chevrel:pascalc]

Boris, should this bug be considered or should we close it? (for background, I was pinged about the status of this bug on Twitter ;) )

Comment 11

[Boris Zbarsky [:bzbarsky]]

Up to the people actually familiar with our font subsystem.  See comment 6.

Comment 12

[Pascal Chevrel:pascalc]

Thanks Boris, Jonathan any opinion about this bug?

Comment 13

[Ollivier Robert]

I don't really see the point and/or optimization leading to have 500+ fonts opened all the time, knowing that you will use maybe 10 or them.  It is a waste of resources.

Comment 14

[Jonathan Kew [:jfkthame]]

I did a bit of tracing here. This appears to be caused by behavior that is internal to the macOS font system, and not something we can readily control.

In order to fully support localized font family names (e.g. the family named "Heiti TC" in English also has several other names, "黑體-繁", "黒体-繁", "Heiti-번체", "黑体-繁"), as required by the CSS Fonts spec[1], we do not simply recognize the single family name returned by an API like CTFontManagerCopyAvailableFontFamilyNames(), we iterate over the available fonts (in the background, shortly after startup), loading the 'name' table of each font and checking for the presence of localized names. It seems that when we use CTFontCopyTable() to load the 'name' table, the Core Text font system (or maybe underlying Core Graphics code) opens the font file. And even though we subsequently CFRelease the font object we created, the file is not closed.

What I'm guessing happens is that just calling CTFontManagerCopyAvailableFontFamilyNames doesn't need to open font files, because the system has already cached the list; but once we explicitly instantiate a given font (in order to access its 'name' table), the system opens the file, and then doesn't close it (even though we release the font) until the process terminates.

Subsequently, we also cache CGFont references in the MacOSFontEntry for each face (I think this happens on first use of the face), which might also serve to hold font files open. However, AFAICS it looks like even if we were to avoid caching these (which is what I was thinking of in comment 6), this won't actually result in the files being closed.


[1] "Some font formats allow fonts to carry multiple localizations of the family name. User agents must recognize and correctly match all of these names independent of the underlying platform localization, system API used or document encoding." https://drafts.csswg.org/css-fonts-3/#font-family-prop

Comment 15

[Jonathan Kew [:jfkthame]]

IMO, unless there is evidence that the open font files are in fact causing problems (e.g. causing something to fail or perform poorly because some kind of resource is exhausted), there's nothing to do here.

I suppose the "problem" would largely go away if, for example, we were to save all the info about the available fonts -- including localized names, etc. -- in the startupCache instead of querying the fonts themselves; we've talked at times about doing that kind of thing. It could be tricky to get reliably right, though, given the various ways in which the collection of available fonts may change (both through manual installation/removal of files and via system APIs that activate/deactivate font resources).

Comment 16

[Boris Zbarsky [:bzbarsky]]

Jonathan, do you know offhand whether Chrome supports localized family names?

Comment 17

[Jonathan Kew [:jfkthame]]

(In reply to Boris Zbarsky [:bz] (still a bit busy) from comment #16)
> Jonathan, do you know offhand whether Chrome supports localized family names?

It doesn't appear to, at least on macOS:

data:text/html;charset=utf-8,<div style="font:36px %E9%BB%91%E9%AB%94-%E7%B9%81, monospace">foo bar

displays "foo bar" in the default monospaced font using Chrome, whereas it correctly uses the Heiti TC font (which has distinctive Avant Garde-like Latin letters) in Firefox.

Comment 18

[Boris Zbarsky [:bzbarsky]]

OK, well, that explains why Chrome has fewer open file descriptors, in spite of the CoreText behavior.  ;)

Comment 19

[Jonathan Kew [:jfkthame]]

Safari, on the other hand, does seem to recognize some localized names, though not all of them. E.g. for Heiti TC (a font targeted at Traditional Chinese), it accepts the Trad Chinese name, but not the Japanese or Korean ones; for Songti SC (targeted at Simplified Chinese), it accepts the Simplified Chinese name but not the Traditional variant.

I'm guessing this may be based on Cocoa behavior, where NSFontManager (or something) may have loaded localized names for certain fonts/locales.

(I believe IE accepted localized font names on Windows, too -- possibly depending on the OS locale setting -- but I'm not on my Windows machine just now to confirm that, and I don't know if Edge behaves the same as old IE versions in this regard. But my recollection is that certain Windows CJK fonts were commonly referred to using localized names in CSS found in the wild.)

Comment 23

[BugBot [:suhaib / :marco/ :calixte]]

The severity field for this bug is relatively low, S3. However, the bug has 3 duplicates.
:bhood, could you consider increasing the bug severity?

For more information, please visit auto_nag documentation.

Comment 24

[BugBot (nomail) [:suhaib / :marco/ :calixte]]

The last needinfo from me was triggered in error by recent activity on the bug. I'm clearing the needinfo since this is a very old bug and I don't know if it's still relevant.