82076 - add support for anti-aliased fonts (Render)

Status: VERIFIED DUPLICATE of bug 100570

(Core :: Layout, defect)

Description

[kill this account]

we should support the new anti-aliased fonts in Render

Comment 1

[kill this account]

*** Bug 82060 has been marked as a duplicate of this bug. ***

Comment 2

[kill this account]

I discussed this with Erik.

We could add a new concrete font class

 class nsFontGTKRender : public nsFontGTK

that uses the Render extension; see:

  http://www.xfree86.org/~keithp/render/

same interface as nsFontGTKNormal; see:

  http://lxr.mozilla.org/seamonkey/source/gfx/src/gtk/nsFontMetricsGTK.cpp#1887

This should be a relatively straight forward bit of work.

Once we had working prototype we would put hooks in the
TryNode(), TryFamily(), etc. to check for a Render font first.

Comment 3

[Christopher Blizzard (:blizzard)]

*** Bug 68314 has been marked as a duplicate of this bug. ***

Comment 4

[Stuart Ballard]

I don't think bug 68314 is a dup of this - I think 68314 is more general.

Render, in general, can do a lot more than just antialiased text. I'm not sure
how far along support is for these features right now, but when it's completed
it will be able to do at least all of the following:
- Basic alpha blending (CSS opacity and -moz-opacity properties)
- Per-pixel alpha blending (PNG, MNG)
- Antialiased line, polygon, curve etc drawing (SVG, MathML?)
- Antialiased image resizing (img width= height= and CSS equivalent)

I'm pretty sure that the first of these is supported by Render right now
(although it might not be in an XFree release yet) and I wouldn't be surprised
if the second is too. I'm pretty sure that the third and fourth aren't
implemented yet.

However, bug 68314 had the summary "X-Render support for GTK" which implicitly
includes ALL of these areas - not just antialiased fonts like this one does.

Can we reopen bug 68314 as a tracking bug for FULL use of Render, and mark it
FUTURE for now since most of the things we need out of Render aren't even
available yet? Then make this a dependency and potentially add other
dependencies as appropriate...

Comment 5

[kill this account]

Can someone give me a *small* (striped to the basics please) sample program 
using Render that:

 1) Get the list of fonts
 2) inspects the font properties (from the list info)
 3) opens a font
 4) inspects the font properties (from the font info)
 5) gets the character map(s) (list of available encodings/glyphs)
 6) measures the size of some text
 7) draws some text

Should keithp be cc:'d on this bug?

Comment 6

[Christopher Blizzard (:blizzard)]

Here are two urls that I know about that have information on them:

http://keithp.com/~keithp/render/Xft.tutorial

HTTP://www.eax.com/render/

Comment 7

[Jeffrey Baker]

timecop on #mozilla has added Render support to Xlib Mozilla.  You might ask to
see his diffs.

Comment 8

[Matthew Cline]

Looks like a dub of bug 31296.

Comment 9

[Teruko Kobayashi]

Changed QA contact to [e-mail] because this is related to bug 31296.

Comment 10

[kill this account]

> Looks like a dup of bug 31296.

Sort of.

There are at least two ways to do anti-aliased text:

 1) easiest/least feature: use Xft / Render
    requires user have the Render extension (ie: XFree86 4.02+)
    (of course we could always try to add feature to Xft)

 2) medium difficulty/full features: use FreeType and Render
    requires user have the Render extension (ie: XFree86 4.02+)
    we can add (complex text language) CTL support

 3) very difficult: do the anti-alias our selves
    would work on all systems; ie: does not require any extension
  

Comment 11

[Jeffrey Baker]

That's three ways, not two, and I have a fourth.  Wait for GTK+ 2.0 and Pango,
use them for all you complicated text layout and rendering needs.

Comment 12

[Stuart Ballard]

I suggest Way #2. My reasons against each of the other methods are...

1) Lack of features. This would be my second choice, though.
3) Doing anti-alias by hand would be SLOW, given the way that X works. If this
were practical, everyone would have been doing it long before Render came along
:) This fact was the whole rationale for creating Render in the first place.
4) It seems to me that Mozilla actually is almost independent of GTK already (it
certainly doesn't use it for any user-visible widgets). This suggests we should
be trying to remove/reduce our GTK dependencies, rather than adding more of
them.

Comment 13

[kill this account]

Jeffery: we have looked at Pango and their font selection code has zero
support for CSS type font specification (or even run time changeable
font settings). Until that changes, using Pango would be a big, big step
backwards.

Stuart: yes, I also am leaning toward #2 (especially to get correct support 
for Arabic and Thai). 

Doing #1 would be much faster though so I'm trying to understand exactly what 
the visual difference between #1 and #2 would actually be. Another reason to 
consider #1 is that then the whole Linux community would benefit.

The main benefit of #3 is that it would work on Redhat 6.2 (and equivalent
systems). No one did this in the past because in the past there was not a
labrary like FreeType2 available.

Comment 14

[kill this account]

http://bugzilla.mozilla.org/show_bug.cgi?id=78643 also has a bit of info

Comment 15

[kill this account]

How will printing work? XPrint? GhostScript?

Comment 16

[kill this account]

Here is what I believe to be the rough list of things that need to happen:

1st Order
----------------------------------------------
1) get a list of the fonts on the system
2) Open a font and determine which Unicode characters are available in each 
   font.
3) Measure the width/height of a string of Unicode characters.
4) Render the glyphs and draw to screen (non anti-aliased).
5) Render the glyphs and draw to screen (anti-aliased).
6) Render and draw to printer.

2nd Order
----------------------------------------------
1) Cache the list of available fonts.
2) Cache the rendered glyphs for each font.
3) Prune "no longer needed" glyphs.

3rd Order
----------------------------------------------
1) Glyph substitution (Arabic).
2) Glyph positioning (Arabic).

Interesting things that could come later:
----------------------------------------------
dynamic download of fonts from the web

For the record, here are some resources:
http://www.xfree86.org/~keithp/render/
http://www.eax.com/render/
http://freetype.sourceforge.net/freetype2/

Comment 17

[Brice Ruth]

Hey there.  I've been looking at FreeType and such and thinking a lot about the
fonts.  Personally, I would like to see Mozilla render fonts as nicely in Linux
as it does in MacOS.  I think that's a lofty goal that we should set for
ourselves, I'm open to arguments, but I think that the MacOS (9.1/X) has by far
the prettiest display of fonts.

So - I went and grabbed the latest FreeType2 and did some tests w/ the HelloX
application (patched to use Xft).  I checked out the following fonts: Arial,
Times, and Courier - arguably the most important for Mozilla.  If we get these
fonts right (a) we've won 98% of the battle and (b) the remaining 2% will
probably fall right into place.  My analysis so far: FreeType2 still needs some
work.  No offense at ALL to the FreeType guys - the fact that we have
anti-aliased fonts at all and that their engine supports practically every
available font format is amazing.  But, I compared the anti-aliasing under xmag
of a string produced in the HelloX window and one created under MacOS w/
Photoshop 5.5 (same fonts).  I'm attaching two jpgs of my name (Brice) under
magnification rendered using FreeType2 and PS - you'll notice significant
differences in the 'B' - which is very apparent w/o magnification.

This submission is mainly an FYI - I'm sure the FreeType folks are constantly
improving their stuff ;-).

-Brice Ruth

Comment 18

[Brice Ruth]

Attachment: FreeType2 rendering; Arial

Comment 19

[Brice Ruth]

Attachment: Photoshop rendering (smooth); Arial

Comment 20

[Brice Ruth]

Attachment: Italic font in Mozilla (Linux); Arial

Comment 21

[Brice Ruth]

Attachment: Italic font in Mozilla (MacOS); Arial

Comment 22

[Brice Ruth]

The last two attachments were to show the smoothness of fonts in Mozilla in
MacOS vs. Linux - obviously the MacOS clip uses anti-aliasing whereas the Linux
clip does not.  This is mainly to say "this is our goal" ;)

Comment 23

[kill this account]

I'm starting to explore integrating FreeType2 with Linux mozilla.

One of the biggest issues to be resolved early on is what to do about
libfreetype.so not being available on the developer/user system.

Is the FreeType library too large to ship with mozilla?

	libfreetype.a  is 1.3M 
	libfreetype.so is 800K.

I would prefer that it be a standard build option as it is a very
important feature. However, making it a standard build option would 
force all Linux developers to upgrade which is rude. I suppose for 
developers we can have some configure option like MathML uses.

How will we handle the end user not having libfreetype.so?

While newer versions on Linux may well have the FreeType library
there are a lot of older systems that do not.

I'd hate to end up shipping two versions.

Could we layer it behind a XPCOM interface and if it fails to load
then do not use it?

Suggestions and volunteers are welcome.

Comment 24

[Jeffrey Baker]

Statically linking in Freetype might be a solution, but wouldn't Mozilla also be
relying on Xft?  Freetype by itself doesn't know where or how to get fonts, you
have to point it to a file.  Xft manages font selection and location on disk. 
So, you would already be requiring XFree86 4, and by implication Freetype 2.

Freetype isn't exactly bleeding edge.  It has a stable and widely used release,
it is shipped in Mandrake 8 and Slackware 8, GTK+ 2 and Pango both use it, KDE
uses it, and it is required for GNOME 1.4.  I don't think anyone will shout
about this requirement, except people whose machines can't run XFree86 4.

Comment 25

[Jeffrey Baker]

BTW on my Slackware 8.0 system, libfreetype.a is 256K and libfreetype.so.6.0.1
is 223K.

Comment 26

[Stuart Ballard]

Jeffrey, see comments from Brian Stell on 2001-06-28 12:39. It is entirely
possible to use Render without Xft, which is just a convenience library
providing a common API for core fonts and Render based freetype fonts. In fact,
it's possible that Xft won't even provide all the control over fonts that
Mozilla needs (for the complex font-matching mandated by CSS) and will *have* to
roll its own equivalent (talking directly to Freetype and Render when available)
to get full CSS compliance.

Btw, I think the concern about Freetype being too "cutting edge" is more
relevant for proprietary unixen than for Linux, where people are already used to
the requirement of cutting-edge libraries and most of them have distributions
that will auto-update anyway (especially debian users :) ). I heard somewhere
that some of the proprietary unix people even found the Gtk+/Glib requirement to
be something of a problem.

Comment 27

[kill this account]

Mine must be debug builds.

I like the idea of a XPCOM interface. This could hide the availability
non-availability of FreeType and the Render extension.

I have not decided about Xft as that requires the Render extension and
does not address several important issues:

  Font summaries 
  Font/Glyph caches

Mozilla has a large amount of code that does font lookup and glyph fill in
which we want to keep.

It would be too expensive to open every FreeType font on app start so
for performance reasons we will need to create a font summary:

  foundry name
  family name
  weight
  slant
  Language group
  Unicode Character map

We will need to store this font summary somewhere. 

We will also need a font cache and a glyph-image cache.

All this either needs to go into Xft or Mozilla.

The actual rendering calls are not that large and Xft's ability to draw old 
style bitmap fonts is already in Mozilla.

Regardless of Xft or not we still need to manage the avilability of the
FreeType lib.

Comment 28

[kill this account]

Any suggestions on the best forum(s) to hold these discussions?

Comment 29

[Jeffrey Baker]

I think you would get a lot more pushback if Mozilla is going to go around the X
libraries for font selection, because I think that implies additional
configuration of Mozilla's fonts that other applications don't require.  Better
to work with keithp and the XFree folks to get the required features into Xft.

The original question was about statically linking freetype or using the
system's freetype.  If vendor's unixes are a concern, can those vendors be
relied upon to make their own packages of Mozilla?  Alternately, the linux
builds can be made with freetype dynamically linked, and the sunos builds made
with freetype statically linked.  We have an option to statically link gtk+. 
Dunno if that works.

Comment 30

[kill this account]

Since the size/performance group is going to great lengths to remove 10's of
kilobytes, I doubt they will accept a 250K bloat casually. Hence, static 
linking is a problematic option. Dynamic linking is fine as long as the app 
will run in the absence of the library.

I certainly do not see Xft as the official font access method. Here are
Keith's own word on the subject:

http://www.xfree86.org/pipermail/render/2000-November/000493.html

> I'm trying to transition applications from core text rendering to Render 
> extension based rendering so that I can anti alias even more text on my 
> screen.
> 
> Instead of whacking a million applications, I'm writing a new library that 
> will fall back to core code where necessary.  The hard part of this 
> library is font matching; while we've punted this out of the protocol, it 
> hasn't gone away.
> 
> Instead of trying to solve all of the worlds font matching problems, I'm 
> trying to make simple applications relatively simple to write.  Along the 
> way, we may discover more general mechanisms for font selection, but for 
> now I'll be satisfied if my Xt/Xaw based apps can display things using the 
> Render extension.

Moz's font selection is geared to CSS and glyph-fill-in. This seems quite a 
bit different than what simple apps want. If there is a working library that 
addresses these two issues (or even a working proposal) would you kindly add
the link to this bug report? I will suggested leads.

I am not looking (yet) at a "transition" helper. I am willing to learn how to
use FreeType directly. Learning the FreeType interface is a non-trival thing 
and I laud Keith for making a helper library. Most apps do not need that much 
complex a font subsystem. Mozilla is not a "simple application" and moz's font 
selection mechanisms are already sophisticated. I do not want to make them any 
less.

Perhaps Xft will end up being sufficient but that certianly is not clear at 
this time and I am willing to "whack" mozilla to use FreeType.

Xft only works if the Render extension is available and the Render extension 
is not universally available so I may to make it one of several options but 
it may not be the only.

Comment 31

[kill this account]

s/I will suggested leads./I will lookat suggested leads./

Comment 32

[kill this account]

Jeffrey, 

I don't mean to stop you from pursuing what you think is the best course of
action. Please do so and let's compare notes and results.

I am doing preliminary work to find out what I think is the best course of 
action.

Comment 33

[Christopher Blizzard (:blizzard)]

If we decide to use Xft to do our rendering then we should probably link it
statically.  Even though XFree released it doesn't mean that it probably won't
change underneath you since it isn't done and Keith is still changing things.

Also, we're going to have to do some design up front.  We need to be able to
compile without Xft, and run without it if it's not available as a server
extension.  We don't want to end up with an ifdef mess that makes it
unmaintainable either.

On that note my first instinct is to just create a new file that contains the
Xft render version of the rendering and then when the first factory call comes
in check if the extension is available and if it isn't, load the gtk version. 
You only need ifdefs in one place ( the factory ) and it's easy to choose at
build time if you need it.

Comment 34

[kill this account]

Attachment: html specificying a algerian font

Comment 35

[kill this account]

I am leaning more and more toward putting the code behind a XPCOM interface.

At build time a configure.in flag would test if the build system has the 
necessary parts or not.  If not available on the build system the interface
would just become a stub.

At run time the interface could (gracefully I hope) fail to load if the
libraries are not there.

Comment 36

[Christopher Blizzard (:blizzard)]

You don't need to hide it behind an XPCOM interface.  You just need to be able
to switch at runtime depending on whether or not the extension is on the server.
 We already have an XPCOM interface.  That's the current font api.

Comment 37

[kill this account]

> switch at runtime depending on whether or not the extension is on the server

Am I correct in infering you mean whether the Render extension is there?

I was thinking of a XPCOM to manage whether the FreeType library,
libfreetype.so, was available to the mozilla (the client) not whether the 
Render extension as available on the X server.

Am I missing something here?

Comment 38

[Christopher Blizzard (:blizzard)]

I don't think that you can try to work around a missing shared library.  If you
have to have it you have to have it.  If you're building it on a particular
system with support for render then freetype will be there and you will be
passing on that dependency to people who consume your software - there's no way
to get around that.

Comment 39

[Dan Mosedale (:dmosedale, :dmose)]

blizzard: couldn't one attempt to have the XPCOM component not be linked against
the shared library with ld.so, but attempt to use PR_LoadLibrary to do the
dlopen() itself? Or, if that's considered too gross, just take all the objects
that make up the shared library and link them into an XPCOM component.

Comment 40

[kill this account]

Newer systems will very probably have both libfreetype.so and the Render 
extension and this should be our main target.

I would prefer, if possible, to ship one Linux client that can at least run 
in the absence of either of these.

It will be a while before all the older Linux versions go away and even 
without the Render extension they could definitely benefit from using the 
larger set of outline scaled fonts (non-anti-aliased) that are available.

I will checkout PR_LoadLibrary.

Comment 41

[Christopher Blizzard (:blizzard)]

The problem with dynamically loading the shared library is that you have to call
into the library through looked up addresses.  You probably have to end up
making all your own vtables.  It's just not worth it.

The fact is that Render and freetype are new to the unix world and we're going
to have to support the old way of doing things for a long time.  Red Hat 7.1 was
one of the first distros that supported this.  There are a couple of others, too.

If you're thinking ahead to distribution then you will probably have to do
builds for newer systems and older ones.  It's a PITA from a QA standpoint but
it's almost unavoidable.

Next week I'm starting my gtk2 port.  gtk2 uses Render but I don't know if the
interfaces in gtk2 will be enough or if we're going to have to dig down to the
lower levels.  This might be what you need to be able to get your portability, too.

Comment 42

[Dan Mosedale (:dmosedale, :dmose)]

Or we ship our own version of the shared lib, like we do with (eg) jpeg, gif,
etc.  Or link the objects directly into an XPCOM shared component.  (Not that
these don't have their own issues; but they are options).

Comment 43

[Christopher Blizzard (:blizzard)]

And start worrying about maintenance and upgrades and future compatibility and
clashing symbols with embeddors?  No thanks.

Comment 44

[kill this account]

Attachment: html spec'ing dnahand public font

Comment 45

[kill this account]

where to find dnahand font:
http://fonts.tom7.com/fonts98.html
http://fonts.tom7.com/f/dnahand.ttf

Comment 46

[kill this account]

Attachment: patch; my current horrible hack integrating FreeType and Mozilla; this needs alot of work

Comment 47

[kill this account]

This patch, attachment 42321 [details] [diff] [review], is my current code that displays a FreeType font
in Mozilla. It only works with the html in attachment 42320 [details] at present.

I have tried to comment what is okay and what needs work (most of it).

I created a new nsFontGTK subclass, nsFontFreeType that does FreeType rendering.

Currently, it does not use the Render extension and it requires a 24/32 bit
frame buffer.

This is very crude but I thought people might want a peek at it.

If you see that something "kinda works but for the wrong reason", well that's
the way it is right now.

Hopefully, I will get a branch going soon so others can help.

Comment 48

[kill this account]

Attachment: image: mozilla home page using Cyberbit font (anti-aliased but not that good looking yet)

Comment 49

[kill this account]

Attachment: patch: added nsFreeType.cpp/h; still needs FreeType font cataloging

Comment 50

[kill this account]

Attachment: status of the code in attachment 43377

Comment 51

[kill this account]

I believe that the general sturcture of the code in attachment 43377 [details] [diff] [review] is 
reasonable.

The code is displaying FreeType using get/put XImage so it works on any X
system. (Would someone like to add support for the Render extension?).

FreeType font-face and glyph caching (FTC) is in.

Still missing the font cataloging code.

Comment 52

[kill this account]

As a point of reference:

Opening ~220 truetype fonts on my 950 Mhz (64 bit ram bus) PC takes 
  1st time: 14 sec
  2nd time: 11 sec

This is done with a standalone test program that:

  scans the dir specified on the command line
  for each file {
    Opens the font
    gets the family name
    gets the style name
    gets the face flags
    gets the style flags
    get the list of supported characters
      ie: scans the complete Unicode range (0-FFFF) for glyphs
    puts all these summary items in a (malloc'd) list
    closes the font
  }
  releases all memory and exits

Comment 53

[Brice Ruth]

A couple things to keep in mind with the fonts - especially after the times
posted recently: (1) most basic systems don't have many fonts, of those fonts,
the ones used 98% of the time in web pages are even fewer
(Verdana,Times,Arial,Helvetica) (2) One way to get around needing to know which
fonts are on a system when you start up is simply looking for the most commonly
used fonts (monospace, serif, sans-serif) - this takes care of you *most* of the
time - when a different font is requested, take a local performance hit and
search for the font at the time it is needed.  This is how Adobe's ATM
application works - it doesn't even let you have all your fonts activated at a
time (if you have a lot) - but if you enable auto-activation, it intercepts the
calls for fonts from applications when the application needs them and then
activates the font on the fly - the font remains active until the computer is
shut down, if I recall correctly.

Since I don't know the mozilla font sub-system all that well, I don't know how
well this can be handled, but its something that should be considered.

Comment 54

[kill this account]

Attachment: sample code to generate/read a font catalog; tar/gz

Comment 55

[kill this account]

the sample prog in attachment 43705 [details] is completely functional except it is
missing the code to determine language groups for a font

Comment 56

[kill this account]

(groan)
Frank and Erik explained to me that TrueType cmaps sometimes map non-space 
characters to the space char; ie: a blank glyph

To find out if a char has a glyph I now need to convert the char to a glyph
index and load the glyph (but not render it).

1st time catalog 200+ TrueType font files: 86 sec
2nd time catalog 200+ TrueType font files: 73 sec

load 200+ font summaries from the font catalog: ~1 sec
(include character maps)

Comment 57

[kill this account]

Attachment: sample font catalog prog; with language group and blank glyph detection; tar/gz

Comment 58

[kill this account]

The one second time to read the font catalog is fine. The time to generate the
font summary is quite another issue.

If it takes 90 sec on my high end system to generate the font summaries then 
on a slower system might take 2 to 5 minutes which is "way too long" to 
tolerate. Users that do not know what is happening (and that it is a one 
time only event) will complain that moz is broken.

Clearly we cannot just silently generate the font catalog from inside a 
normal mozilla.

Options: (your suggestions here are welcome)

 1) Popup a dialog explaining the TrueType fonts have changed
    and the font summary is being regenerated (ie: please wait
    and wait and wait and wait and ...).

 2) Have a web page explaining how to generate the font
    catalog. Will users find this?

 3) Generate the font catalog during installation. What happens
    when new files are loaded?

 4) In font prefs have a TrueType panel where the user could
    enter the dirs and a button to generate the font catalog
    with a warning that it could take a long time.

 5) Popup a dialog telling the user how to generate the font
    catalog.

 6) Fork/exec a separate a font catalog program to run in the
    background. Of course during that time the fonts will not
    be available.

Comment 59

[Brice Ruth]

Always the troublemaker, I'd suggest a combination of your options.  Generating
the font catalog at install is definitely a good idea.  That will take care of
the typical 95% so oft quoted in statistics.  

Now - that doesn't help if new fonts are added to the system, right?  Well - to
my (albight limited) knowledge, if I install a plugin, I have to run as root for
Mozilla to recognize it, if Mozilla is installed as the root user - which it
will be on most systems as we go forward, as it will most probably be provided
as an RPM or apt package (or something in that vein).  

So - when Mozilla is run as root, fork a process to regenerate the font catalog
- while it is running, use the old one.  When it has completed, inform Mozilla
to begin using the new one.  Having some kind of dialog box associated will keep
the user from wondering what is eating up their processor cycles.

And - also have a button in the font preferences to force Mozilla to regenerate
its catalog - this should the trigger the same mechanism as above, fork, dialog
box, when complete - inform Mozilla.

My $0.02.
Brice

Comment 60

[Stuart Ballard]

Is it possible to lazily generate the catalog? That is, generate the catalog for
each individual font as it's used?

So on program startup, first time, the catalog is generated for each font used
in the UI; later, when a web page requests a font, incrementally generate the
catalog for that font. Also, when a font is requested, check whether the font
has changed and if so regenerate (only that font's) catalog.

Presumably generating a catalog for a *single* font won't take 5 minutes, right?

Comment 61

[kill this account]

> Is it possible to lazily generate the catalog?

Sadly, not. The big requirement for the font info comes when moz does fill-in 
glyph subsitution. This is the time when moz needs info on all the fonts.

Its the "which glyphs" does this font have that takes all the time. Loading
all the fonts but excluding that info only takes a few seconds.

It would be a "real bummer" to load a page and take a 5 minute hit looking for
a fill in glyph (especially if no font has the glyph and we end up drawing a
'?' anyway!).

My current thinking is to build a font catalog on install. Then, on moz 
startup, check for new (or deleted) fonts and only do font glyph testing
on the new font and merge the old and new font info into a new catalog.
I think this might limit it to a 10-20 sec which I would consider okay.

Comment 62

[kill this account]

Here are my current thoughts on using Xft:

Issues regarding using Xft to get TrueType support in Mozilla:

1) Requiring the Render Extension:
====================================
The Render extension is a good thing. However, currently it is not a 
standard item on most Linux / Unix systems. Choosing to tie Mozilla's 
use of anti-aliased fonts to Render would severely limit its 
application for a long time to come.

Also, Render is not fully stable (the APIs may still change) so using 
it in moz may lead to configuration problems in the future. Once 
Render is declared stable it would be appropriate to make it one of 
moz's options (hopefully, the 1st/primary option).


2) Detecting Blank Glyphs
========================================
All versions of Mozilla (Windows/Mac/Linux) have a large amount of
code to find "fill in" glyphs if the primary font does not support a
needed character. This is a critical user requirement: "if moz cannot 
make the text perfect then at least make the text readable". On Linux 
the fonts mostly have all the glyphs for the characters of the 
encoding of a particular font. (There is some messy code that tries to
resolve these incomplete Linux fonts.)

TrueType fonts, however, often will return a valid glyph index for a 
given character which when drawn will draw a blank (or space character). 
Obviously, these blank characters are not readable. Thus there needs 
to be a way to determine the true character map. With direct access to 
the TrueType font via FreeType2 moz can look at the number of outline 
contours for each glyph and thus determine if the glyph is blank when 
it is not a blank character such as space, nbsp, etc. While it might 
be technically possible, I do not think it desirable to render every 
glyph and actually look if any pixels are set.

A new X API would be necessary for this.

3) Printing
===========================================
X has never had a printing solution (it is amazing that X survived 
this shortcoming). Since printing is critical and moz has always done
"something" on Linux to support printing it seems unwise to give up
access to the TrueType font info until there is official X support for
printing.

Several new X APIs are needed for this.

4) Opening "All versus Some" Glyphs
=================================
The X APIs provide no way to open/render only a subset of glyphs.
Even if moz only needs a glyph or two all the glyphs must be rendered
which can be a big time penalty. Also, web pages that want one 
"really big" character will cause the X server to render the entire 
set of glyphs. For TrueType fonts with large character sets like 
CyberBit this can actually exhaust the X server's virtual memory. 

A new X API would be necessary for this.

5) To Anti-Alias or Not
===========================================
Anti-aliasing does not produce the best results for all sizes of all
fonts. (I am not an expert on this particular point and I am trying to 
accurately parrot the text in the TrueType specification). This is an 
important control where the font designer's input should be 
considered. 

A new X API would be necessary for this.

6) Access to TrueType info/controls
===================================
The X API currently does not provide for access or control of 
interesting TrueType feature like enable/disable hinting, sub-
pixel placement, glyph substitution info in the font, glyph 
positioning info in the font (some Arabic fonts should vertically
position each glyph in a word to look correct), vertical layout
info.

Several new X APIs would be necessary for this.


7) XLFD of TrueType font info:
==============================
I have never been a big fan of string encoding a data struct to pass 
it between applications and libraries. XLFD is such a string'ified 
item. I will admit that it does work but ...

Xft directly opens the font so why should we continue the silliness 
of string'ifying the info to XLFD only to de-string'ifying this data 
so we can use it in mozilla?

While this API is still usable it needs an upgrade. I have very little 
interest in carrying forward that 1980's baggage into 2001+.

A new X API would be good here.

Comment 63

[kill this account]

Here is my current thinking of the font catalog creation ("takes a long time") 
issue:

In the notes on how to get Mozilla to use TrueType fonts we will tell the user 
that after the dir path is initialized (or changed) the mozilla will 
automatically regenerate a new font catalog (which will take a 1-5 minutes) so 
please be patient for this "create a font catalog from scratch" event. IE: we
handle it in documentation.

Also, when new fonts are added mozilla will *merge* the new information about
the new fonts into the font catalog.  This may take 10-30 sec per new font so 
if the user only added a one or two fonts this one-time-hit should only be
a few ten's of seconds.

I have modified the font catalog code to automatically do this merging.

Comment 64

[Dan Mosedale (:dmosedale, :dmose)]

bstell: instead of just putting it in the notes, how about throwing up a dialog
telling users what's going on, similar to the way Windows does when it's
indexing a help file for the first time?

Comment 65

[kill this account]

do I hear someone volunteering? (any help is welcomed!)

Comment 66

[kill this account]

Attachment: patch; has font cataloge which merges; charset info needs work

Comment 67

[kill this account]

The latest patch is working but the charset info is sketchy (needs work).

To use the latest patch modify the unix.js file to indicate where your TrueType
font are.

Comments / code review is welcome but this is still a work in progress.

The TrueType fonts are in a separate node list from the X server fonts
and are always checked first.

People interested in the Render extension may want to take a look at 
nsFreeType.h and nsFreeType.cpp to see where to add Render support.

The current code assumes FreeType2 is available. It does not use configure.in 
to turn off the FreeType code when FreeType is not available on the developement
system. It does not handle the case where FreeType2 is not available on the
runtime system.

File handling is basic Linux/Unix style and needs to be more mozilla like.

Comment 68

[kill this account]

current stats:

nsFontMetricsGTK  ~300 lines, mostly moving structs to headers

nsFreeType        ~1200 lines of code to interact with the FreeType2 lib and 
                  draw to the X server using XGetImage and XPutImage

nsFontCatalog     ~2300 lines to manage reading / merging the font summaries
                  (performance related issues)

Comment 69

[Jeffrey Baker]

Most XFree86 users are going to have their TrueType fonts, if any, in the X
server's font path.  If the catalog searched for fonts by default in the X font
path, that would take care of 90% of cases without any further configuration.

Also I read the interestingly-named functions in your patch, and I didn't see an
aweful lot of stuff that is TT-specific.  Does the patch also use FreeType to
render Type1 and other formats?

Comment 70

[kill this account]

> If the catalog searched for fonts by default in the X font path, that would 
> take care of 90% of cases without any further configuration.

Are the fonts in a standard location -OR- how does one get the X font path?

Do we expect our typical user to understand how to configure the X font path?

> I read the interestingly-named functions in your patch

I tried to name the functions in nsFreeType.cpp nsFreeTypeXYZ
I tried to name the functions in nsFontCatalog.cpp nsFontCatalogXYZ
this way there should not be name collision in the debugger.
I expect that the super-reviewer (probably blizzard) will make me make these 
classes in which case the names will become nsFreeType::XYZ and 
nsFontCatalog::XYZ

Any constructive suggestions? (please be gentle and positive)

> I didn't see an aweful lot of stuff that is TT-specific.  Does the patch also 
> use FreeType to render Type1 and other formats?

If the font does not have a (TrueType) OS/2 table I currently ignore it.

I get the (critical) language group info about the font out of the TT OS/2 
tables. Here is the prototype:

+  if ((fce->mCodePageRange1 & TT_OS2_CPR1_LATIN1) 
+      || (fce->mCodePageRange1 & TT_OS2_CPR1_MAC_ROMAN)
+      || (fce->mCodePageRange2 & TT_OS2_CPR2_WE_LATIN1)
+      || (fce->mCodePageRange2 & TT_OS2_CPR2_US))
+    printf("latin1 (iso8859-1), ");

I do not know how to determine lang group for other font types.

(Lang group is to help select alternate fonts for glyph fill in and to tell 
which lang group the font should be in the font preferences.)

Later I expect to get more font metrics like subscript offset, superscript
offset, etc. from the TT tables.

Comment 71

[kill this account]

I also get weight (light/regular/bold), width (narrow/medium/wide), and 
vendorID (foundry) from the OS/2 table.

Comment 72

[Jeffrey Baker]

The xset program is able to query the font path, so there must be a programmatic
way to probe it.  I'll look at the xset source.  It seems unlikely that an
XFree86 user would have TrueType fonts *not* in the X font path.

Comment 73

[Jeffrey Baker]

All hail xlib.  man XGetFontPath

SYNTAX
       XSetFontPath(display, directories, ndirs)
             Display *display;
             char **directories;
             int ndirs;

       char **XGetFontPath(display, npaths_return)
             Display *display;
             int *npaths_return;

       XFreeFontPath(list)
             char **list;

Comment 74

[kill this account]

Attachment: patch; uses correct character map

Comment 75

[kill this account]

Attachment: status of patch (attachment 44815)

Comment 76

[kill this account]

I feel that the code in attachment 44815 [details] [diff] [review] has all the important features
and I consider it to be demo grade: not fully done but good enough to show
people.

Comment 77

[Stuart Ballard]

I just remembered hearing that as of Xfree 4.1, the Xft library has some kind of
caching code which sounds like it just might be equivalent to the font catalog
that takes so long for mozilla to build. I really don't understand what Xft's
cache is for OR exactly what mozilla's catalog is for, but IF they turn out to
be the same thing, would it be possible for Mozilla to take advantage of Xft's
cache if Xft is installed and in use on the same machine / user-home-dir? This
might make it possible to avoid the one-time "building catalogs" process if the
user already has Xft installed and working.

Or I could be completely wrong...

Comment 78

[keithp]

Xft provides font support at many levels, at the most basic level, it abstracts away the difference between Render text and core text so that applications can be written that will run on servers withouth the render extension and still funciton.It provides a new matching system designed to be relatively compatible with CSS, but where that system is inadequate, it can easily be replaced with other code.Currently, the TrueType support can either provide Unicode glyph encodings or direct raw access to the glyph indicies within the TrueType/Type1 font file.  As the Xft font objects are transparent, the knowledgable application candirectly access all FreeType APIs to extract additional font file information such as the character mapping tables and OpenType layout information.An outstanding issue is that when using core fonts, Xft doesn't provide a Unicode mapping for glyphs, but I expect to fix that in the near future.I would like to see Mozilla use Xft for font access for a couple of reasons:1)	Unified font configuration.  Xft provides a single configuration	mechanism to locate and name fonts as well as a unified set of	X resources to tune font access2)	Render extension glyphset generation.  Xft does all of the work	to deal with incremental glyph rasterization, this includes LCD	text optimizations.I'm writing a paper on Xft for the XFree86 technical conference; thatshould illuminate some of the mysteries while also help explore areasthat are still horribly broken.I think Xft is moving in a good direction, it was improved greatly byusing it for Qt's AA text support, I suspect basing Mozilla on Xft willserve to refine it even further.-keith

Comment 79

[Stuart Ballard]

(reposting Keith's comments from a browser that supports the proprietary
wordwrapping feature that bugzilla requires - oh, the irony. Hope I got the
paragraph breaks right...)

Keith wrote:
Xft provides font support at many levels, at the most basic level, it abstracts
away the difference between Render text and core text so that applications can
be written that will run on servers withouth the render extension and still
funciton.

It provides a new matching system designed to be relatively compatible with CSS,
but where that system is inadequate, it can easily be replaced with other code.

Currently, the TrueType support can either provide Unicode glyph encodings or
direct raw access to the glyph indicies within the TrueType/Type1 font file.  As
the Xft font objects are transparent, the knowledgable application candirectly
access all FreeType APIs to extract additional font file information such as the
character mapping tables and OpenType layout information.

An outstanding issue is that when using core fonts, Xft doesn't provide a
Unicode mapping for glyphs, but I expect to fix that in the near future.

I would like to see Mozilla use Xft for font access for a couple of reasons:
1)   Unified font configuration.  Xft provides a single configuration       
mechanism to locate and name fonts as well as a unified set of  X resources to
tune font access
2)       Render extension glyphset generation.  Xft does all of the work to deal
with incremental glyph rasterization, this includes LCD text optimizations.

I'm writing a paper on Xft for the XFree86 technical conference; thatshould
illuminate some of the mysteries while also help explore areasthat are still
horribly broken.

I think Xft is moving in a good direction, it was improved greatly byusing it
for Qt's AA text support, I suspect basing Mozilla on Xft willserve to refine it
even further.
-keith

Comment 80

[kill this account]

Overview
----------------------------------------------------------------
Let me start by again repeating that I think Xft is a good thing.

Simple applications will definitely benefit from it.

Applications that need more than the X APIs provide will end
up interacting with FreeType directly. Any library that proposes
to provide a layer between such an application and FreeType 
would need to provide some interesting feature or it would not
be justified. Additional layers can introduce bugs and 
compatibility problems and delays in getting new features out.

Currently I have not found any significant feature for Mozilla
in Xft and there is a large negative feature: it requires the 
Render extension to use TrueType fonts.

Xft is also missing a major feature: an accurate character map.
TrueType fonts often have non-zero glyph indexes that end up 
drawing a blank. No one wants an app where bits of the text
are blank (eg: unreadable). FreeType does provide a way to tell
if a glyph is blank but it takes a lot of CPU time. Because it
takes so long it can not be done on the fly. To manage this
problem I ended up writing 2.4k lines of font cataloging code. 
Render and/or XFree86 may want to look at the this code if they
are interested in producing an accurate (read useful) character 
map promptly.


Point-By-Point
----------------------------------------------------------------

> it abstracts away the difference between Render text and core text 
> so that applications can be written that will run on servers without 
> the render extension and still function.

As I understand this it means no TrueType support without the Render
extension; ie: just core fonts. This would make TrueType unavailable
to a large part of the Linux community. 

I thought of suggesting to Keith that he add anti-alias support
for systems without the Render extension to Xft but this seemed a
bit counter-productive for his work (although it would benefit the
Linux community as a whole).

> It provides a new matching system designed to be relatively  
> compatible with CSS, but where that system is inadequate, it can  
> easily be replaced with other code.

Matching the family name is trivial and would not justify linking in
or requiring a new library for that.

Finding out which lang group a font supports is much more interesting.
Last time I looked Xft did not have this.

> Currently, the TrueType support can either provide Unicode glyph 
> encodings or direct raw access to the glyph indicies within the 
> TrueType/Type1 font file.  As the Xft font objects are transparent, 
> the knowledgeable application can directly access all FreeType APIs to 
> extract additional font file information such as the character 
> mapping tables and OpenType layout information.

If an application is going to interact with FreeType then what value
does Xft add? Adding another layer is only justified if there is good 
value.

> An outstanding issue is that when using core fonts, Xft doesn't 
> provide a Unicode mapping for glyphs, but I expect to fix that in 
> the near future.

Mozilla already handles this.

> I would like to see Mozilla use Xft for font access for a couple 
> of reasons:
> 1) Unified font configuration.  Xft provides a single 
> configuration mechanism to locate and name fonts as well as a 
> unified set of X resources to tune font access

There is no reason Mozilla cannot query the font path and use the
information there. 

Could you describe the "unified set of X resources to tune font 
access"?

> 2) Render extension glyphset generation.  Xft does all of the work 
> to deal with incremental glyph rasterization, this includes LCD 
> text optimizations.

LCD text optimizations is a fine thing to add but I certainly would
not take this feature if it means losing the Redhat 6.2 user community.
I would want to look at what the actual code does before making a
decision to incorporate it.

If LCD text optimization is important then please open a bug 
requesting this feature.

> [Xft] was improved greatly by using it for Qt's AA text support, I 
> suspect basing Mozilla on Xft will serve to refine it even further.

Mozilla helping the Xft be better is only a good thing from Mozilla's
prospective if doing so does not mean degrading Mozilla. Adding an
extra layer without justification is not a good thing.

Comment 81

[Jeffrey Baker]

bstell, I've noticed that a few times you made a distinction between core fonts
and TrueType fonts.  I don't understand why you do this.  A TrueType font can be
accessed through the regular X APIs under XFree86.  Today, I use only TrueType
fonts in Mozilla/X11 (and other apps like xterm), even though Mozilla doesn't
explicitly support FreeType.  You don't get the full features of FreeType, but
the fonts are there.  And, these fonts work perfectly well with the Render
extension loaded or without.

Just saw your message over on [e-mail].  I hope you didn't mind too
much that I invited the Render folks over to this bug.

Comment 82

[keithp]

> Let me start by again repeating that I think Xft is a good thing.
>
> Simple applications will definitely benefit from it.

As I said, Xft provides multiple levels for applications to interact at, in
particular it does not hide the underlying FreeType API at all allowing
applications to take full advantage of the information present there while
still using Xft for font configuration and Render extension interaction.

> Currently I have not found any significant feature for Mozilla
> in Xft and there is a large negative feature: it requires the
> Render extension to use TrueType fonts.

Xft provides font naming missing in FreeType.  FreeType has no mechanism to
extract font names from TrueType files nor any mechanism for cataloging
fonts and matching using patterns.  Nor does it require the Render extension
to use TrueType fonts -- just to use them with anti-aliasing.  It will
happily use core X fonts (including TrueType) while providing font
matching compatible with local TrueType fonts.

> Xft is also missing a major feature: an accurate character map.
> TrueType fonts often have non-zero glyph indexes that end up 
> drawing a blank.

Xft uses the ISO 10646 character map present in the TrueType file.
Unfortunately, many fonts have a broken 10646 map.  This can be easily fixed
by taking advantage of the other maps present to fill out the missing 10646
encodings.  Not a lot of work, but I'm waiting for the remaining i18n dust
to settle before attacking this particular problem.

My intention is also to cache the list of available glyphs when the font
files are queried -- Xft already caches a lot of information about fonts
to avoid opening every font in every application, the available 10646 ranges
would simply be additional information in that cache.

> As I understand this it means no TrueType support without the Render
> extension; ie: just core fonts. This would make TrueType unavailable
> to a large part of the Linux community. 

Anyone running XFree86 4 will have the Render extension.  The next release
of every commercial Linux packaging will include XFree86 4.1 which supports
Render in essentially every driver.

I doubt there is a significant audience interested in running Mozilla, able
to run Xft and yet unable to run a modern X server.

> Matching the family name is trivial and would not justify linking in
> or requiring a new library for that.

No, even family name matching is non-trivial.  Consider the many varients of
Courier -- which do you pick when the style sheet requests that face?  How
about 'arial' or 'verdana'?  What do you do when the application requests
courier oblique and all you've got is courier new italic?

Finally, FreeType has no standard font directory listing; fonts could be
installed essentially anywhere on the system making sharing fonts among
applications difficult.  Xft centralizes the system configuration for font
storage allowing all applications to use a common set of fonts with common
naming mechanisms.

> There is no reason Mozilla cannot query the font path and use the
> information there.

FreeType doesn't use the X font path -- you pass file names to the API and
it opens the font files.  Xft imposes a configuration mechanism on top of
this simple file API so that applications can request fonts by name rather
than by filename.

> Could you describe the "unified set of X resources to tune font
> access"?

Xft provides a small selection of X resources that control font selection
and rasterization depending on the display characteristics:

        Xft.core:   <bool>              Whether to use core fonts
        Xft.render: <bool>              Whether to use Render fonts
        Xft.scale:  <double>            Font size scale factor
        Xft.dpi:    <double>            DPI to use in point->pixel calculations
        Xft.rgba:   {rgb,bgr,vrgb,vbgr} Physical sub-pixel arrangement
        Xft.antialias: <bool>           Whether to use AA glyphs
        Xft.minspace:  <bool>           Whether to override normal TrueType
                                                linespacing value

These are designed to be the parameters which depend on the display
characteristics and not user preferences. Using the X resource mechanism
allows customization to occur when xrdb is run to select the appropriate
values.

> LCD text optimizations is a fine thing to add but I certainly would
> not take this feature if it means losing the Redhat 6.2 user community.

I'm not sure how the Redhat 6.2 community figures into this equation -- are
you saying that you're interested in supporting AA text on ancient X
servers?  Such environments are usually running on slower machines which
means that a full software implementation is likely to be even slower than
it would be on a modern machine.  

I believe that we should support such environments with reasonable
performance, and that means core X text support, not a software AA
implementation that only runs well on the latest hardware.  Xft can mask the
differences between Render text and core text, the only significant
remaining issue is in mapping encodings so that applications always see a
Unicode API.

> Mozilla helping the Xft be better is only a good thing from Mozilla's
> prospective if doing so does not mean degrading Mozilla. Adding an
> extra layer without justification is not a good thing.

One must place Mozilla in a system context.  The rest of the system will be
using Xft, whether that is through Pango and Gtk+ or Qt and KDE.  In such an
environment, good citizenship strongly encourages the use of compatible
configuration and installation mechanisms.  If the only part of Xft used by
Mozilla is the font naming mechanism, that goes a long ways in making
Mozilla compatible with the other font consumers on the desktop.

Besides, I hope to demonstrate that Xft does provide some utility in font
naming and Render extension support.  Xft doesn't negatively impact
interactions with FreeType while solving some persistant problems with X
font access.  If Xft needs additional capabilities to fully support Mozilla,
those will be incorporated.

(I hope that by posting this with Mozilla instead of another browser, it will be
formatted correctly this time)

Comment 83

[kill this account]

> A TrueType font can be accessed through the regular X APIs under 
> XFree86.

Not on a stock Redhat 6.2 system.

I do not consider asking an average Mozilla user to upgrade their 
X server a reasonable request. If someone thinks this is a 
reasonable request then please report back when you have gotten
95% of all Linux/Unix/Mozilla users on all platforms world-wide 
to do so. I will strongly consider going back to my original plan
to use Xft in Mozilla.

I do however think that asking them to download a font is reasonable.

> Just saw your message over on [e-mail].  I hope you 
> didn't mind too much that I invited the Render folks over to this 
> bug.

Discussion is good.

> Xft provides multiple levels for applications to interact at

This is a good thing. But an additional library needs something to
justify the problems that always come adding an additional layer.

> FreeType has no mechanism to extract font names from TrueType files

What about getting the face and looking at the fields? My code
does this now. It was easy.

> Nor does it require the Render extension to use TrueType fonts -- 
> just to use them with anti-aliasing.  It will happily use core X 
> fonts (including TrueType) 

Not on a stock Redhat 6.2 system with a stock XFree86 3.x server.

Again, I do not consider asking the typical Mozilla user to upgrade 
their X server a reasonable request and I do not plan to exclude
these users. I will not budge on this point.

> Xft uses the ISO 10646 character map present in the TrueType file.
> Unfortunately, many fonts have a broken 10646 map.  This can be 
> easily fixed by taking advantage of the other maps present to fill 
> out the missing 10646 encodings.

An accurate character map is too expensive to do when an app is 
running. A pre-calculated font catalog is required. It is a critical 
requirement for the browser that "the text be readable" and blank 
glyphs are not not readable.

> My intention is also to cache the list of available glyphs when the 
> font files are queried

Knowing the available glyph indexes and which actually have contours 
(eg: are not blank) is a good thing. Let me know how things go.
You could look in my patch to see how I do it.

> Anyone running XFree86 4 will have the Render extension

This is true but many Mozilla users do not run XFree86 4.x so this
does not matter.

> I doubt there is a significant audience interested in running Mozilla, 
> able to run Xft and yet unable to run a modern X server.

True, but there is a large audience that *does* run Mozilla and does
not run XFree86 4.x or have the Render extension.

> Consider the many varients of Courier -- which do you pick when the 
> style sheet requests that face?

Does the CSS spec say what to do? Lacking a spec there is no correct
answer. What Xft does is not better or worse than any other system.

> FreeType has no standard font directory listing; fonts could be
> installed essentially anywhere on the system making sharing fonts 
> among applications difficult

I suspect that Mozilla will end up looking in the same dirs as xfs, 
etc. Using font cataloging there is not reason Mozilla cannot get
the X font path and catalog all the fonts there. Passing those names
to FreeType is trival. To support automatic font download, I suspect
that Mozilla will require addition directories. It would be good
if somehow those could be shared with the other X apps. I suspect that
file permissions will be the main stumbling block for sharing downloaded
fonts with the X server.

> Xft imposes a configuration mechanism on top of this simple file 
> API so that applications can request fonts by name rather than by 
> filename.

As I said before simple apps will benefit from Xft. I already needed
a font catalog so knowing the actual file names adds no difficulty.

> Xft provides a small selection of X resources ... allows 
> customization to occur when xrdb is run 

I do not see Mozilla depending on xrdb for preferences.

> I'm not sure how the Redhat 6.2 community figures into this 
> equation -- are you saying that you're interested in supporting AA 
> text on ancient X servers? 

Let me say "yes, loud and clear" that I plan to support our customer 
base. I have no intent to cut off groups of users by only supporting
the latest and greatest systems. It is fine to build systems that
will benefit the newest users> But why cut off old users? There are
lots of Linux/Mozilla users who could benefit from TrueType font 
technology and who could download a font but who could not upgrade 
their X server.

Will FreeType run on older systems? Seems to. 

Will XGetSubImage and XPutImage work reasonably on slower systems? 
Seems to.

The alpha blending at the client does not seem that expensive to me. 

The crude X API that requires the entire set of a fonts metrics to 
be sent at once IS a problem and we already live with that. The way
XListFonts works seems very crude to me and Mozilla goes to a lot of 
effort to avoid asking for the full list. The lack of accurate cmap
information made me "bite the bullet" in the FreeType work and build 
a font catalog with accurate glyph information.  Reading the entire
font catalog seems much faster than XListFonts. I do admit it will
add a slow operation to the installation. Only time will tell if 
all this will work on all Unix systems (HP comes to mind here).

> One must place Mozilla in a system context ... good citizenship 
> strongly encourages the use of compatible configuration and 
> installation mechanisms

Only when there is NO PENALTY to Mozilla. Period. I will not 
sacrifice the Mozilla Redhat 6.2 users in an effort to be a good 
citizen.

> If the only part of Xft used by Mozilla is the font naming 
> mechanism, that goes a long ways in making Mozilla compatible with 
> the other font consumers on the desktop

As a practicle matter, simple apps like xterm and notepad do simple
things with fonts. More sophisticated apps like Mozilla and
MS Word do more complex things.

How do the other "font consumers" plan to manage automatic font 
download?

Where is the common font selection dialog?

I have far more interest in making Linux Mozilla compatable with
Mac Mozilla and Windows Mozilla which are the products of Mozilla
and Netscape.

Do the things that you think will make your apps better. 

I will do the things that I think will make Mozilla better.

In the long run having competing efforts raises the bar for 
everyone. Linux will benefit from the competition.

When I can see that Xft adds good value to Mozilla I will consider 
dealing with the problems of: lack of XFree86 3.x server support, 
potential compatability issues, delayed feature availiblity, 
the potential of additional bugs, more complex debugging, etc.

PS: what does Xft plan to do about printing?

Comment 84

[Brice Ruth]

Hey folks.  I'm seeing a lot of talk about supporting old Linux installations,
old X installations, etc.  It would behoove us all to remember that compromises
must be a part of good software engineering.  You want new features, you need to
make certain things obsolete - not doing so will harm your future and that is
something that won't benefit anyone.

Specifically - insisting on supporting X 3.3.x yet requiring the installation
(and possibly upgrading) of FreeType whilst making a distinction between this
and upgrading to the latest version of X seems somewhat silly.  It is laudable
that the Mozilla team wants to support as many Linux users as possible, but
personally, if the Mozilla code adopts techniques that slow the system down on
newer platforms in order to maintain backwards compatability, that's insane.  It
is acceptable software engineering to penalize older systems when new features
are added to software - it shouldn't be done haphazardly, but it is entirely
acceptable, it keeps things moving in the right direction.  However, penalizing
future users who have every right to expect that the very latest software will
be optimized to the very latest systems is not acceptable.  Keep this in mind.

Another factor to consider is the demographic of Linux systems - the older
systems will often not be desktop systems, but rather server systems since the
admin adage used is 'if its not broken, don't fix it' (as long as security
updates remain available :)).  However, desktop systems are typically upgraded
more frequently, specifically to take advantage of newer hardware and newer
software.  Where will Mozilla be used most frequently?  Certainly the answer
would be desktop systems.

Also - I noticed a mention of the windows and mac versions of Mozilla - keep in
mind that these systems use built-in font capabilities.  I know the fonts on my
Mac system are smoothed amazingly well by ATM - there's no recreation of the
wheel going on here.  I imagine the same can be said for the Win32 version,
though I have far less experience with it.  In this case, wouldn't it behoove
Mozilla/Unix to use system libraries rather than recreating the wheel?  I laud
the efforts of the Mozilla team to incorporate these advanced font capabilities
into Mozilla - but instead of creating a product that is distinct, why not
enhance the abilities of Xft?  If you're worried about 3.3.x compatability,
investigate the possibility of backporting the Xft code.  If you're worried
about the ability to use TrueType fonts in non-Xft systems, figure out a way of
supporting those so that other applications might also take advantage of it. 
Surely, this approach will be more difficult, but in the long run, it will
enhance Linux far more than duplicating another team's effots.

Competition is good in some aspects - Pango vs. KDE, GTK+ vs. Qt.  But in some
instances, I don't believe its necessary.  This could be one of them.

I don't mean for this message to hurt anyone's feelings or anything of the sort,
but let's think about this.  Open Source w/o Open Minds is pretty narrow, isn't it?

One last note - I don't see empowering users to be able to use TT fonts in
Mozilla w/o the ability to use them in other applications as very useful.  And
unless I'm missing something, this is what a 3.3.x system such as RedHat 6.2
would end up being.  Why?  Why would a user who we portray as being unwilling to
upgrade their X system be willing to install TT fonts solely for use in Mozilla?
 That doesn't seem to click in my mind.

p.s. how does OpenOffice handle fonts (and printing?)

Comment 85

[Boris Zbarsky [:bzbarsky]]

> Why would a user who we portray as being unwilling to upgrade their X system
> be willing to install TT fonts solely for use in Mozilla?

If I may interject...  I would say I'm such a user.  I'm using RedHat 6.2.  I
don't want to upgrade because it's a pain.  I don't feel like taking the time to
recompile XFree4.0.x for my system and deal with possible dependency issues.

At the same time, there are only 4 applications I use with any consistency: 
xterm, emacs, xmms, and a web browser (fairly typical usage of Linux systems in
my experience).  Of these, the browser is the only one that really needs to
handle TT fonts.  So if I were to install TT fonts, it would be for use of the
browser only.

And installing TT fonts is trivial... you take the font files off the windows
partition on your dual-boot system (fairly typical setup) and you copy them to
the linux partition.  Or you just make sure the windows partition is mounted and
read the files directly off it.  In either case, this is not nearly as
complicated a process as an OS upgrade or an "X-but-not-OS" upgrade.

Comment 86

[Stuart Ballard]

So it seems like the major "blocker" issues for using Xft in mozilla are
twofold:

1) Support for old / proprietary systems that can't support render or truetype
fonts directly, but could still benefit from truetype support in Mozilla.
2) Support for caching the table of which characters are missing glyphs in which
fonts.

bstell, did I miss any?

Keith, how hard would it be to add support to Xft for "non-antialiased freetype
fonts", and/or "non-RENDER-supported antialiased fonts" for systems that don't
support TrueType in their X server (issue 1)? Did I understand correctly from
your comment that you already have plans to provide support for issue 2 in the
future anyway?

bstell, if these two issues could be resolved in a future version of Xft, would
you then be willing to use it in order to give users the benefits as outlined by
keith (unified configuration mechanism, etc...)?

Comment 87

[Christopher Blizzard (:blizzard)]

>> A TrueType font can be accessed through the regular X APIs under 
>> XFree86.
> 
> Not on a stock Redhat 6.2 system.
> 
> I do not consider asking an average Mozilla user to upgrade their 
> X server a reasonable request. If someone thinks this is a 
> reasonable request then please report back when you have gotten
> 95% of all Linux/Unix/Mozilla users on all platforms world-wide 
> to do so. I will strongly consider going back to my original plan
> to use Xft in Mozilla.
> 

You aren't walking away from those users, Brian.  Those users can still use the
old font system that we have right now.  They just won't have AA fonts, that's
all.  As it says above you can use TT fonts on 6.2 systems - they just aren't
anti-aliased.

>> Xft provides multiple levels for applications to interact at
> 
> This is a good thing. But an additional library needs something to
> justify the problems that always come adding an additional layer.
> 

It does.  It provides system-wide consistency for font names and what fonts will
be used.  This is very important.
> 
> I suspect that Mozilla will end up looking in the same dirs as xfs, 
> etc. Using font cataloging there is not reason Mozilla cannot get
> the X font path and catalog all the fonts there. Passing those names
> to FreeType is trival. To support automatic font download, I suspect
> that Mozilla will require addition directories. It would be good
> if somehow those could be shared with the other X apps. I suspect that
> file permissions will be the main stumbling block for sharing downloaded
> fonts with the X server.
> 
> 

So what are you suggesting?  That we start parsing Xft's configuration files to
get those paths?  When does the reimplementation stop?  Mozilla is _not_ a world
unto itself.  You have to be willing to deal with the fact that on some systems
some services might be unavailable.
>> Xft provides a small selection of X resources ... allows 
>> customization to occur when xrdb is run 
> 
> I do not see Mozilla depending on xrdb for preferences.
> 

Xft does and passes that on to you.  Mozilla doesn't have to directly support
those preferences unless you are planning on reimplementing that, too.

> Will FreeType run on older systems? Seems to. 
> 
> 

Will 6.2 users have to have it installed to build mozilla?  If so, what version?
( 6.2 ships with 1.3x and 7.x ships with 2.x and I don't have any idea what
debian users. )  Are you planning on linking statically?  How will you deal with
symbol conflicts for embedding?  What's the licensing on FreeType?  Is it legal?

> 
> Will XGetSubImage and XPutImage work reasonably on slower systems? 
> Seems to.
> 
> The alpha blending at the client does not seem that expensive to me. 
> 
> 

It's a lot slower than what 6.2 users will have now.  Like, a _lot_ slower since
you have tp XGetImage()/XPutImage() for every draw operation.  That's not
acceptable to me.  That's worse than using old fonts.

> Only when there is NO PENALTY to Mozilla. Period. I will not
> sacrifice the Mozilla Redhat 6.2 users in an effort to be a good 
> citizen.

There is a penalty to Mozilla.  You're just glazing over it.  It means that
Mozilla won't play well with the rest of the system configuration, will suffer
on 6.2 systems wrt performance and will reimplement the wheel.  Again.  It's bad
enough that we don't use native widgets for much of the UI.

Look, sometimes you just have to be willing to say that if you want extra
capabilities you should upgrade.  You aren't walking away from old users you're
just saying that if they want new capabilities they have to be willing to
upgrade.  It's as simple as that.

Comment 88

[kill this account]

[this is a reply to Brice Ruth comments of 2001-08-07 07:41]

> if the Mozilla code adopts techniques that slow the system down on
> newer platforms in order to maintain backwards compatibility, that's
> [a poor choice]

When the Render extension is available the Mozilla code will work
the same as Xft. There is not penalty.

> Also - I noticed a mention of the windows and mac versions of Mozilla
> - keep in mind that these systems use built-in font capabilities.
> ...  In this case, wouldn't it behoove Mozilla/Unix to use system
> libraries rather than recreating the wheel?

Yes, where the libraries add value.

> [If one wants] to use TrueType fonts in non-Xft systems, figure out a
> way of supporting those so that other applications might also take
> advantage of it.

Support for TrueType in non-Xft systems is a good thing and I would be
glad to chat with anyone on how they might do this. I, myself, an
putting TrueType support in Mozilla. The code is open to the public and
people are encouraged to look at and reuse the pieces that are useful
in other contexts.

> Competition is good in some aspects - Pango vs. KDE, GTK+ vs. Qt.
> But in some instances, I don't believe its necessary.  This could be
> one of them.
>
> I don't mean for this message to hurt anyone's feelings or anything
> of the sort, but let's think about this.  Open Source w/o Open Minds
> is pretty narrow, isn't it?

My feelings are certainly not hurt. However, saying that one way (such
as Xft) is the only way way is closing the door to other possibilities.

I began this work thinking I would use Xft.

Look back at my comments of 2001-06-28 12:39 in this bug report. I
thought at that time that Xft was a likely choice. It was after I
began to study the Xft source that I slowly decided that for Mozilla
there was not enough value in Xft to justify using it. Xft is a
good (perhaps short term) way to get anti-aliasing into simple X
apps while the X server does not do support it. In the long run
Xft's good feature should be merged into Xlib and the X server.

Xft also has other good features but those are already done in 
Mozilla as they were needed long before Xft was written. Mozilla 
still needs features that Xft does not support and while I think 
adding them to Xft is a good thing Mozilla will have them far sooner 
than Xft. If Mozilla is going to have to build these features before 
Xft has them then what advantage does *Mozilla* get for waiting for 
Xft to catch up? It all boils down to this: a library must justify 
its use.

> One last note - I don't see empowering users to be able to use TT fonts in
> Mozilla w/o the ability to use them in other applications as very useful.

I believe that Redhat 6.2 Mozilla users will be happy to use TrueType
fonts even if they cannot use them in other places. Many people have
been more than happy that Mozilla displayed fonts/pages that xterm, etc.
could not.

> Why would a user who we portray as being unwilling to upgrade their
> X system be willing to install TT fonts solely for use in Mozilla?

Besides that fact that installing a new X server is non-trival,
changing the X server effects *every* application. Some may feel this
is a *good* thing and to those I heartily encourage them to take the
plunge. But should we force users to do this hard and risky task if
it is not necessary? No one is going to guarantee to the user that this
will not break something.

Installing a font (especially when automatic font download is added)
is a very-simple and low-risk task. Far more people will do this.

Even for those systems with a TrueType support in the X server
(and anti-aliasing when it arrives) or Xft with its client side
rendering will still need to download fonts.

I would be very surprised if Mozilla was not doing automatic
font download long-long before Pango, KDE, GTK+, or Qt. My guess
would be that once Mozilla trail blazes this on Linux these others
will leverage the technology. When that common leveraging happens
then Mozilla and the others will find a way to share the fonts.
This is a case where all sides benefit.

Alternate solutions are good for the open source community.

The good ones will survive and the poor ones will fade.

Comment 89

[Brice Ruth]

Back again. :)

First, I agree with blizzard - adding two draw operations (GetImage/PutImage)
for _every_ glyph that's displayed on the screen will be an amazing slow down. 
I'm not entirely sure how Mozilla's rendering code works - if it renders an
entire page or just the screen at a time, but I notice lags in re-rendering when
I scroll, right now, w/ Mozilla 0.9.2 on a 400MHz G3 w/ over 512MB RAM - plenty
of room for any application to get comfortable.  Will this method of providing
anti-aliased fonts slow down scrolling as well?  Will 'smooth scrolling' be a
pipe dream that people w/o 1.8GHz machines won't be able to realize?  

I'd also like to contribute my $0.02 on font downloading.  I don't want to see
any wide ranging decisions for the font system made based on the support for
downloading fonts.  Let's not forget that a huge percentage of sites (a) don't
specify fonts to use, so its up to the browser's default, (b) of the sites that
*do* specify fonts, they're typically sans-serif or serif, and in most cases,
Arial for sans-serif and Times for serif.  

Absolutely EVERY design/production book or site that you read for serious web
designers, HTML production persons, etc. indicates that using Netscape's
proprietary dynamic font system or Microsoft's version thereof is to be avoided
at ALL costs.  I don't see support for dynamic fonts in Macromedia Dreamweaver
nor in Adobe GoLive! (though I have far more experience w/ Dreamweaver).  

If you have text on screen that you want to use a non-standard font for, you
create a GIF rendered by Photoshop/Imageready or Freehand/Fireworks - its that
simple.  If you have dynamic content that you want to use a different font for
or you don't want to take the size/speed penalty for using graphic images, the
industry standard is to use Flash - Flash allows the embedding of font
information as vectors which are then displayed perfectly on a client's system,
regardless of their installed fonts.  

I'm not eager to pursue downloading of fonts on any platform.  In MacOS, with or
without ATM, the leading cause of system instability is corrupt fonts - you
think that downloading fonts via your browser might lead to more and more
corrupt fonts on your system?  Probably so.  I'm not sure how corrupt fonts are
handled in Win32 systems, but I wouldn't be surprised if some nasty things
happened there.  As for Linux systems, when I brought over TT fonts from my
Win98 system, I experienced a dramatic screw-up in how Mozilla rendered fonts,
eventually traced back to a mixture of 16-bit and 32-bit TT fonts.  How are
systems going to handle not being able to accept the format a font is provided
in?  I have an application that can manipulate a TT font into a postscript font
and transfer font formats between Win32/MacOS/UNIX - Mac's use font suitcases,
try providing it a .ttf file - I don't think it'll work.  And Win32 systems
don't support postscript fonts at all - w/o the use of ATM from Adobe. All in
all, its a can of worms that is absolutely unnecessary to open.  But, that's
just my opinion, as usual :)

I'm glad this dialog is taking place, fonts and font display is one of the areas
in Linux that is most lacking.  

Comment 90

[kill this account]

> First, I agree with blizzard - adding two draw operations (GetImage/PutImage)
> for _every_ glyph that's displayed on the screen will be an amazing slow down.
> I'm not entirely sure how Mozilla's rendering code works - if it renders an
> entire page or just the screen at a time, but I notice lags in re-rendering 
when
> I scroll, right now, w/ Mozilla 0.9.2 on a 400MHz G3 w/ over 512MB RAM - 
plenty
> of room for any application to get comfortable.  Will this method of providing
> anti-aliased fonts slow down scrolling as well?  Will 'smooth scrolling' be a
> pipe dream that people w/o 1.8GHz machines won't be able to realize?

I am currently pleased with the scrolling on my Redhat 6.2 system 500 MHZ 
system.

Why would anyone consider doing text with XPutImage more expensive than doing
graphics with XPutImage?

Is anyone saying that Mozilla has issues when scrolling images?

> I don't want to see any wide ranging decisions for the font system made based
> on the support for downloading fonts.

Certainly not a deciding factor but one that should be considered.

> If you have text on screen that you want to use a non-standard font for, you
> create a GIF rendered by Photoshop/Imageready or Freehand/Fireworks - its that
> simple.  If you have dynamic content that you want to use a different font for
> or you don't want to take the size/speed penalty for using graphic images, the
> industry standard is to use Flash - Flash allows the embedding of font
> information as vectors which are then displayed perfectly on a client's 
system,
> regardless of their installed fonts.

Downloading the image is way more costly than XGetImage. Again, is anyone 
suggesting that Mozilla has problems with scrolling images?

This seems to me to be "fear, uncertainty, and doubt" due to lack of
knowledge.

I would like to ask people to try it before deciding it cannot work.

> I'm not eager to pursue downloading of fonts on any platform.  In MacOS, with 
or
> without ATM, the leading cause of system instability is corrupt fonts 

If one feels that corrupt TrueType fonts are likely to cause instability
they should definitely turn off the TrueType support in the X server and
not use Xft.

I would recommend that instead of giving up the feature the renderer
be made tolerant.

Regarding OS crashes: The Linux/Unix kernel rarely crashes due to an
errant program so the situation is quite different from Mac.

Comment 91

[kill this account]

Let me say again that I think Xft is a fine tool for simple apps.

If people think Mozilla should use it I want to hear specific reasons
and examples of how *Mozilla* will benefit.

Simply saying that Xft is the "good" way to do things adds no value
to this discussion.

Simply saying that other apps. will use it is does not strike me as
an advantage to Mozilla. Mozilla has always been far ahead of the
rest of the Linux/Unix community when it comes to displaying text
and graphics on Unix system.

There is nothing wrong with being ahead. The good ideas are shared
and the poor ideas are re-thought.

If other tools like Xft have specific advantages and those are part
of the core system available to all Mozilla/Linux/Unix users I
agree that Mozilla should look carefully at using them.

If, however, those advantages are not available on the user's system
then Mozilla must decide to: 1) not use it, 2) use it only where
available, 3) provide the feature internally.

Let me see if I can summarize the current discussion:

1)  There is agreement that using FreeType2 is a good thing.

2)  There is agreement that fonts should be available to all
    applications.

3)  There is agreement that where the Render extension is available
    it should be used.

4)  There is agreement that anti-aliasing is good thing
    but perhaps is not critical.

5)  There is agreement that there is some TrueType support in the
    X servers.

6)  There is agreement the interesting data in the TrueType fonts
    is a good thing for Mozilla to use to produce better looking
    pages. The discussion I have heard so far is that Mozilla will
    directly access FreeType to get this data.

7)  There does not seem to be any discussion of Client side vs. Server
    side glyph rendering. I have not heard any comments that Xft should
    use server side rendering.

8)  I hope there is agreement that automatic font download is a
    good thing. If anyone knows of discussions going on in Pango,
    KDE, GTK+, Qt, etc. on this please say add references.

9)  There seems to be no discussion on what to do about printing.

10) There is disagreement on what level of features will be usable
    older systems due to performance considerations. The one person 
    that has tried XGetSubImage/XPutImage (me on a Redhat 6.2 500 
    MHz Pentium) seems to think it is okay. People who have not 
    tried it are concerned it will be too slow. Cleary, some testing 
    and tuning would help here.

11) There seems to be little discussion on the fact that Xft does not
    use FreeType glyph caching (at least I did not see this in the Xft
    sources so kindly point it out if I missed it). This is an important
    performance feature. I have written that code to use this.
    The only advantage XXft has here is it does not do the XGetSubImage
    but has the disadvantage that it re-renders that glyphs *every* time
    even if the current character is the same as the last character.

12) There seems to be little discussion on the fact that Xft is missing
    code to get the language groups for a font (at least I did not see
    this in the Xft sources so kindly point it out if I missed it). I
    have written the code to do this (actually, the current patch is
    missing the inverted lang. group lookup).

13) There seems to be little discussion on the fact that Xft is missing
    code to produce an accurate character map promptly (at least I did
    not see this in the Xft sources so kindly point it out if I missed it).
    Lacking this TrueType fonts are of no value. I have written that
    code. This is a huge performance issue. To do glyph fill Mozilla
    will scan the maps of all fonts for glyps. To get the true cmap
    for 200 TrueType fonts takes 1-5 minutes so it can not be done on
    the fly.

14) There is disagreement on whether it is reasonable to ask a user
    to upgrade their X server.

15) There no agreement on how fonts will be made available.
    It is not resolved how an automatic download from the browser
    will be made available to other apps.

16) The discussion on whether Xft font name look and aliasing is
    better or not than Mozilla font name aliasing needs to be
    discussed more.

17) There has been no discussion on where to merge in the
    TrueType support.

Comment 92

[Christopher Blizzard (:blizzard)]

> Why would anyone consider doing text with XPutImage more expensive than doing
> graphics with XPutImage?
> 
> Is anyone saying that Mozilla has issues when scrolling images?
> 
> 

The usual case with scrolling images doesn't use XPutImage.  We only use
XPutImage when we have an 8 bit composited image.  For an example of scrolling
have a look at:

http://www.libpng.org/pub/png/png-rgba32.html

It's pretty damn chunky on my laptop.  For simple masked images like transparent
gifs we use shared memory access so what you are thinking uses XPutImage isn't
actually using it.
> This seems to me to be "fear, uncertainty, and doubt" due to lack of
> knowledge.
> 
> 

No, it isn't.  An extra server round trip for an XGetImage and XPutImage for
each string draw is insane.

Comment 93

[Christopher Blizzard (:blizzard)]

> If people think Mozilla should use it I want to hear specific reasons
> and examples of how *Mozilla* will benefit.
> 

The most immediate is that Mozilla will use the same font configuration
information that all other apps using Xft will be. ( gtk2, qt, others. )  This
is very important and can't be understated.  We've got enough problems with
different apps using different fonts.

> Mozilla has always been far ahead of the
> rest of the Linux/Unix community when it comes to displaying text
> and graphics on Unix system.

That's a questionable assertion.  gtk2 is doing more than Mozilla is wrt text
rendering and qt has been using AA fonts and render for a long time now.

> 1)  There is agreement that using FreeType2 is a good thing.
> 

You will have to use FreeType2 in any case through Xft.  Also, FreeType2 isn't
available on Red Hat 6.2 systems by default.  You complain about people having
to upgrade their X servers but they aren't going to be upgrade free.

> 2)  There is agreement that fonts should be available to all
>     applications.
> 

I don't get this one.

> 3)  There is agreement that where the Render extension is available
>     it should be used.
> 

Yes.

> 4)  There is agreement that anti-aliasing is good thing
>     but perhaps is not critical.
> 

Yes.  Certainly not critical but nice to have.

> 5)  There is agreement that there is some TrueType support in the
>     X servers.
> 

There is some in varying degrees.

> 6)  There is agreement the interesting data in the TrueType fonts
>     is a good thing for Mozilla to use to produce better looking
>     pages. The discussion I have heard so far is that Mozilla will
>     directly access FreeType to get this data.
> 

Yes, through Xft or FreeType directly.

> 7)  There does not seem to be any discussion of Client side vs. Server
>     side glyph rendering. I have not heard any comments that Xft should
>     use server side rendering.
> 

It should and as far as I know, it does.

> 8)  I hope there is agreement that automatic font download is a
>     good thing. If anyone knows of discussions going on in Pango,
>     KDE, GTK+, Qt, etc. on this please say add references.
> 

I haven't heard anything about automatic font downloading except in Mozilla's
case and I'm not sure what it's going to take to make it happen.  Each user can
have their own Xft configuration file.  Maybe we need to work with Keith so that
Xft can rehash the font list or configuration file or something.

> 9)  There seems to be no discussion on what to do about printing.

Well, we're pretty much screwed with PostScript at the moment anyway.  No change
here!

> 
> 10) There is disagreement on what level of features will be usable
>     older systems due to performance considerations. The one person 
>     that has tried XGetSubImage/XPutImage (me on a Redhat 6.2 500 
>     MHz Pentium) seems to think it is okay. People who have not 
>     tried it are concerned it will be too slow. Cleary, some testing 
>     and tuning would help here.

I just noted that information in my previous comment.  It's not acceptable to
me.  Also, have you seen Nautilus?  It's pretty chunky.  Mozilla's performance
is bad enough as it is.  We don't need to make it any worse.

> 
> 11) There seems to be little discussion on the fact that Xft does not
>     use FreeType glyph caching (at least I did not see this in the Xft
>     sources so kindly point it out if I missed it). This is an important
>     performance feature. I have written that code to use this.
>     The only advantage XXft has here is it does not do the XGetSubImage
>     but has the disadvantage that it re-renders that glyphs *every* time
>     even if the current character is the same as the last character.

I'm not sure exactly what the problem is here.  Is this something that can be
pushed down into Xft?  Once again, Mozilla isn't a world unto itself.

> 
> 12) There seems to be little discussion on the fact that Xft is missing
>     code to get the language groups for a font (at least I did not see
>     this in the Xft sources so kindly point it out if I missed it). I
>     have written the code to do this (actually, the current patch is
>     missing the inverted lang. group lookup).

So let's fix Xft.  Keith is more than willing to take patches to fix problems
like this.  He's said so.

> 
> 13) There seems to be little discussion on the fact that Xft is missing
>     code to produce an accurate character map promptly (at least I did
>     not see this in the Xft sources so kindly point it out if I missed it).
>     Lacking this TrueType fonts are of no value. I have written that
>     code. This is a huge performance issue. To do glyph fill Mozilla
>     will scan the maps of all fonts for glyps. To get the true cmap
>     for 200 TrueType fonts takes 1-5 minutes so it can not be done on
>     the fly.

Once again, this needs to be looked at and I'm sure that Keith might be able to
help us find a concensus here.

> 
> 14) There is disagreement on whether it is reasonable to ask a user
>     to upgrade their X server.

It's not.  That's never been at issue.  If people don't support Render then they
shouldn't be using AA fonts.

> 
> 15) There no agreement on how fonts will be made available.
>     It is not resolved how an automatic download from the browser
>     will be made available to other apps.

That's true but we're going to have to deal with it in any case, whether or not
it's Xft or just plain FreeType2.

> 
> 16) The discussion on whether Xft font name look and aliasing is
>     better or not than Mozilla font name aliasing needs to be
>     discussed more.

*nod*

> 
> 17) There has been no discussion on where to merge in the
>     TrueType support.
> 

In what sense?

Comment 94

[Brice Ruth]

What source tree can I apply your patch against and what (if any) special things
do I need to do to support its compilation?  I'd like to see how the speed
compares on text intensive pages.

Comment 95

[Jeffrey Baker]

Just get the head of the Mozilla tree and apply the patch in gfx/src/gtk/

Comment 96

[keithp]

> I haven't heard anything about automatic font downloading except in Mozilla's
> case and I'm not sure what it's going to take to make it happen.  Each user
> can have their own Xft configuration file.  Maybe we need to work with Keith
> so that Xft can rehash the font list or configuration file or something.

Xft separates font matching from font loading; you can build a font pattern
that includes an arbitrary font name and hand it to the Xft load routine; it
will happily load from the specified file.  No magic is required, and you
continue to get the remaining benefits from Xft (X configuration, LCD
optimization and Render interface).

> 11) There seems to be little discussion on the fact that Xft does not
>     use FreeType glyph caching (at least I did not see this in the Xft
>     sources so kindly point it out if I missed it).

Xft doesn't need to use the glyph cache; it renders each glyph only once and
passes the results up to the X server where it is stored to be quickly
rendered with optimized AA text code paths again and again.  Render
shares identical glyphs to reduce memory usage, both intra- and inter-client.

> 12) There seems to be little discussion on the fact that Xft is missing
>     code to get the language groups for a font

My intent is to cache the available 10646 "characters" within the font and
provide both matching based on subsets and allow applications to quickly
determine which glyphs are available.  For glyphs not encodable by 10646,
the underlying FreeType APIs are used.  

As for which language groups are supported, I'm afraid I'm not familiar with
how to get such information out of FreeType.  Given the ability to extract
such, adding additional properties to fonts is straightforward.  Xft is
designed to allow for an extensible set of properties while remaining binary
and source compatible with existing applications.

> 13) There seems to be little discussion on the fact that Xft is missing
>     code to produce an accurate character map promptly (at least I did
>     not see this in the Xft sources so kindly point it out if I missed it).

I haven't run across TrueType fonts with these problems; but as I said, I'm
very interested in providing this information, and code which improves the
quality and reliability of the character maps would be greatly appreciated.

Comment 97

[Jeffrey Baker]

I tried out the patch on the trunk code with no noticeable effect.  At startup,
all of my fonts were catalogued which took about 10 seconds.  The display looked
the same as the way it's always looked.  I didn't get AA fonts, dunno if I was
supposed to expect them or not.  Didn't see any layout or display problems.  I
did get a lot of this assertion:

###!!! Break: at file nsFontMetricsGTK.cpp, line 3743
###!!! ASSERTION: failed to get converter type

Comment 98

[Decklin Foster]

I needed to change the following to get the patch to build:

-FREETYPE2_INCLUDES = -I/usr/local/include/freetype2 
+FREETYPE2_INCLUDES = `freetype-config --cflags`

-FREETYPE2_LIBS = -lfreetype
+FREETYPE2_LIBS = `freetype-config --libs`

(Makefile.in lines 105 and 110.)

Other than that, I got the same results as jwb, but no assertions. Using Debian
packages of xfree86 4.1.0-2, libfreetype6 2.0.2.20010514-1.

Comment 99

[kill this account]

> > This seems to me to be "fear, uncertainty, and doubt" due to lack of
> > knowledge.
> No, it isn't.  An extra server round trip for an XGetImage and XPutImage for
> each string draw is insane.

Do you have firsthand experience with my patch?

Without numbers or first hand experience running *this* particular
bit of code it seems like FUD to me.

I *am* running this and it seems okay.

Xft does a client-side glyph render, XPutImage, and a server 
anti-alias in software (if the hardware is not available).

My code does a a XGetSubImage, client-side glyph render, a client-
side software anti-alias, and a XPutImage.

The difference is: a XGetSubImage (and hardware AA when available).

I do remember seeing code in Xft that said give up FreeType
if Render is not available but I do not remember seeing code to give 
up AA if hardware is not available. You might care to look yourself
in xftdpy.c around line 76

Come on, why won't you try it before you decide it cannot be done.

Beside, what do you care? You and your buddies will be using the
render method.

Those old systems that run slow will be my problem.

> > If people think Mozilla should use it I want to hear specific reasons
> > and examples of how *Mozilla* will benefit.
> > 
> The most immediate is that Mozilla will use the same font configuration
> information that all other apps. using Xft will be. ( gtk2, qt, others. )  
This
> is very important and can't be understated.  We've got enough problems with
> different apps. using different fonts.

It is clear that we should use the font name in the TrueType file.

It is not clear that Xft's family name aliasing is better or 
worse than Mozilla.

Since font information is core to the GUI/OS shouldn't the font 
information should be managed by Xlib / the X server?

Are you saying that you expect Mozilla's font preferences to be
set via Xft?

Other than the fact that a Mozilla process may not have file 
privilege to add an automatically downloaded font there is no
reason why the fonts cannot be shared. Xft is certainly not 
required for this.

> You will have to use FreeType2 in any case through Xft.  

Why?

I certainly see no technical reason why Xft is required.

Xft is not ready. Let me quote the Xft man page:

    http://www.xfree86.org/4.1.0/Xft.3.html#toc5

    Xft will probably change radically in the future; 
    weak attempts will be made to retain some level of
    source-file compatibility. 

Until this changes it seems a very bad choice to tie Mozilla
to Xft. I do not mind sharing info but I do not want to
tie Mozilla to an unstable library.

I do see technical reasons why Mozilla needs to interact 
directly with FreeType2: Mozilla will want access to the info 
available from the FreeType2 API

It makes no sense for Xft to try to wrapper every FreeType2 API.

> > 7)  There does not seem to be any discussion of Client side vs. Server
> >     side glyph rendering. I have not heard any comments that Xft should
> >     use server side rendering.
> > 
> 
> It should and as far as I know, it does.

Kindly see xftglyphs.c around line 127

> > 9)  There seems to be no discussion on what to do about printing.
> 
> Well, we're pretty much screwed with PostScript at the moment anyway.  
> No change here!

I for one can not going to just throw up my hands and give up on
X printing. 

Mozilla must print.

Printing needs to be considered at the core of any font discussion
in X.

I believe that direct access the FreeType2 and the underlying 
files will be an important part of the X printing solution(s).

Any tool that sits on between FreeType2 and applications to 
provide new font support for X without considering printing seems 
very half-baked to me.

I would want some strong technical reason(s) to use a half baked
resource.

I may want to contribute ideas to that resource but I certainly 
would not tie Mozilla to half-baked solutions.

> > 10) There is disagreement on what level of features will be usable
> >     older systems due to performance considerations. The one person 
> >     that has tried XGetSubImage/XPutImage (me on a Redhat 6.2 500 
> >     MHz Pentium) seems to think it is okay. People who have not 
> >     tried it are concerned it will be too slow. Clearly, some testing 
> >     and tuning would help here.
> 
> I just noted that information in my previous comment.  It's not 
> acceptable to me.  Also, have you seen Nautilus?  It's pretty chunky.  
> Mozilla's performance is bad enough as it is.  We don't need to make 
> it any worse.

Do you have firsthand experience with *my* patch?

> > 11) There seems to be little discussion on the fact that Xft does not
> >     use FreeType glyph caching (at least I did not see this in the Xft
> >     sources so kindly point it out if I missed it). This is an important
> >     performance feature. I have written that code to use this.
> >     The only advantage Xft has here is it does not do the XGetSubImage
> >     but has the disadvantage that it re-renders that glyphs *every* time
> >     even if the current character is the same as the last character.
> 
> I'm not sure exactly what the problem is here.  Is this something that 
> can be pushed down into Xft?

The issue is bloat and performance. The current code in Xft is fast
but bloaty. My code is uses the FreeType face/glyph cache and is very 
nearly as fast and not bloaty.

I'm all for Xft being improved but I do find it odd that people tell me 
that Mozilla cannot have better code put directly into it but that this
better code must first be put in some 3rd party library then that 3rd
party library can be used in Mozilla.

To my knowledge we have never before required that Mozilla code be put 
in a 3rd party library so that 3rd party library can be included in 
Mozilla. 

Why are we adding this extra development burden and delay here?

Xft should benefit from Mozilla's code but lets not make Mozilla
development burden heavier without strong *Mozilla* specific 
benefits.

> > 12) There seems to be little discussion on the fact that Xft is missing
> >     code to get the language groups for a font (at least I did not see
> >     this in the Xft sources so kindly point it out if I missed it). I
> >     have written the code to do this (actually, the current patch is
> >     missing the inverted lang. group lookup).
> 
> So let's fix Xft.  Keith is more than willing to take patches to fix problems
> like this.  He's said so.

I am glad to help Xft but I still object to being told that my Mozilla
work cannot be directly added to Mozilla but instead must be added
to a 3rd party library, wait for that library to be stable, and only then
it can be used in Mozilla.

> > 13) There seems to be little discussion on the fact that Xft is missing
> >     code to produce an accurate character map promptly (at least I did
> >     not see this in the Xft sources so kindly point it out if I missed it).
> >     Lacking this TrueType fonts are of no value. I have written that
> >     code. This is a huge performance issue. To do glyph fill Mozilla
> >     will scan the maps of all fonts for glyphs. To get the true cmap
> >     for 200 TrueType fonts takes 1-5 minutes so it can not be done on
> >     the fly.
> 
> Once again, this needs to be looked at and I'm sure that Keith might be able 
to
> help us find a consensus here.

I have been talking with Keith about this and I think that the XFree86
team should be involved since fonts are really a GUI/OS issue.

He has a simple font catalog but I will be asking for a fairly dramatic
upgrade.

I am more than will to share my work but again let me repeat that it
make no sense to require Mozilla developers to add any Mozilla 
code to a 3rd party library, wait for that library, so it can be use
in Mozilla.

If this is a new policy I think we should start a very frank discussion 
with the core Mozilla team.

> > 14) There is disagreement on whether it is reasonable to ask a user
> >     to upgrade their X server.
> 
> It's not.  That's never been at issue.  If people don't support Render 
> then they shouldn't be using AA fonts.

Why shouldn't they have access to AA fonts?

Can you give a technical reason here. Lacking an actual technical 
reason or firsthand experience with a particular bit of code it 
seems like FUD to me.

My firsthand experience is that it seems to work fine without Render.

> > 15) There no agreement on how fonts will be made available.
> >     It is not resolved how an automatic download from the browser
> >     will be made available to other apps.
> 
> That's true but we're going to have to deal with it in any case, 
> whether or not it's Xft or just plain FreeType2.

This is why I want to include the XFree86 team on any public font
access libraries/APIs

> > 16) The discussion on whether Xft font name look and aliasing is
> >     better or not than Mozilla font name aliasing needs to be
> >     discussed more.
> 
> *nod*

Until that happens it does not make sense to require replacing 
Mozilla's (debugged in the Mozilla context) font name aliasing with 
Xft's (not debugged in the Mozilla context) aliasing.

> > 17) There has been no discussion on where to merge in the
> >     TrueType support.
> 
> In what sense?

At what layer in Mozilla.

The merge can happen at the gdk layer in which case all font 
access will go thru Xft or it can happen in a new nsFontGTK class.

If it replaces the gdk layer then all font access goes via Xft but
we get a whole new layer in the mozilla code. A new un-debugged 
layer of course will all the potential for speed issues, memory
issues, and bugs.

Adding it at the gdk layer also means all the existing Mozilla
debugged core X font code will have to have FreeType2 code
merged into it. As a person responsible for the Linux font code 
I would strongly prefer not dorking in a major new feature to the 
old code.

The merge can happen as a new nsFontGTK class in which case all
the existing Mozilla debugged core X font code can be left alone.

(assuming I have this info about Xft right)
Xft's font list API returns results which include the core 
fonts. If we merge at the nsFontGTK layer we will need to add
code to winnow out the core X fonts since those are to be
dealt with in the old code. 

I submit that we should not de-stabilize the old code.

I submit that if we add the FreeType2 support the old code we
will end up with a very messy class that tries to do two very
different things.

Erik and I discussed this a long time ago and we both agreed
that we should make a new nsFontGTK class for the FreeType 
support.

If we do this then Xft does not fit well.

Render and direct FreeType2 access do fit well.

Comment 100

[kill this account]

> ... Xft library [may have] some kind of caching code which sounds like it 
> just might be equivalent to the font catalog ... IF they turn out to 
> be the same thing, would it be possible for Mozilla to take advantage of Xft's
> cache 

Mozilla needs a font cache because it is fairly CPU expensive to generate the 
glyph bitmaps so if would help performance if the bitmap can be reused. However
saving all generated glyph bitmaps would bloat memory over time so Mozilla
needs some way to save them for some limited time.

FreeType2 has a LRU (Least Recently Used) font/glyph cache that maintains a 
constant memory footprint by dropping the LRU glyphs when space for new glyphs
is needed.

The font catalog is a summary on disk of a fonts properties.
Mozilla needs data not in the Xft font catalog.

Xft's author (Keith Packard), and I need to work with the XFree86 team to
find a way to share a common font catalog.

Comment 101

[kill this account]

> What source tree can I apply your patch against and what (if any) special 
> things do I need to do to support its compilation?  I'd like to see how the 
> speed compares on text intensive pages.

The 08/06/01 11:00 patch was current that day.

This is not ready for checking so people may experience some (hopefull) minor 
compile issues.

Comment 102

[kill this account]

> I tried out the patch on the trunk code with no noticeable effect.  At 
> startup, all of my fonts were catalogued which took about 10 seconds.  
> The display looked the same as the way it's always looked.  I didn't get 
> AA fonts, dunno if I was supposed to expect them or not.  Didn't see any 
> layout or display problems.  

You might want to get Cyberbit if you can find it.

Go the the font preferences and look for a font where the foundry
name start with an upper case letter. That should be a TrueType font.
Select those font and see if you can see AA text.

To get a "close" look at the glyphs try "xmag".

Comment 103

[Jeffrey Baker]

I'll have to try it again when I get back home, but last night I used only fonts
with uppercase foundry names (all my fonts are TT anyway), and the text was
decidedly not anti-aliased.  Is there a lower point size bound below which your
code disables AA?  If so, is it scaled against display resolution?  Maybe I need
to frob the use-my-fonts-not-theirs pref.

Comment 104

[kill this account]

> Xft doesn't need to use the glyph cache; it renders each glyph 
> only once and passes the results up to the X server where it is 
> stored to be quickly rendered with optimized AA text code paths 
> again and again. Render shares identical glyphs to reduce memory 
> usage, both intra- and inter-client.

This is an excellent scheme for saving CPU cycles.

Over time Mozilla opens many fonts and draws many different glyphs.

How does Xft manage client and server memory bloat as more
and more fonts and glyphs are loaded?

> > 12) There seems to be little discussion on the fact that Xft is missing
> >     code to get the language groups for a font
> 
> ... how to get such information out of FreeType.  

The OS/2 table has 64 bits for this

+  if ((fce->mCodePageRange1 & TT_OS2_CPR1_LATIN1) 
+      || (fce->mCodePageRange1 & TT_OS2_CPR1_MAC_ROMAN)
+      || (fce->mCodePageRange2 & TT_OS2_CPR2_WE_LATIN1)
+      || (fce->mCodePageRange2 & TT_OS2_CPR2_US))
+    printf("latin1 (iso8859-1), ");

See nsFontCatalogDumpFontCatalogEntry() in nsFontCatalog.cpp

> > 13) There seems to be little discussion on the fact that Xft is missing
> >     code to produce an accurate character map promptly (at least I did
> >     not see this in the Xft sources so kindly point it out if I missed it).
> 
> I'm very interested in providing this information

    for (i=0; i<0x10000; i++) {
      glyph_index = FT_Get_Char_Index(face, (FT_ULong)i);
      if (glyph_index !=0) {
        if (slot->format == ft_glyph_format_outline) {
          if ((slot->outline.n_contours==0) && 
                (!nsFontCatalogIsSpace(glyph_index))) {
                   character i is a blank non-space glyph

You might care to look at the nsFontCatalogFceFromFontFile 
routine for the complete example. 

Getting all the contour is the only way I currently know to tell
if a glyph is blank. Getting these contour counts takes a too much
CPU time to do at startup or lazily on-the-fly.

The fact that Mozilla needs this information and it is too expensive
to do at startup or lazily at run time is the primary item Mozilla 
needs from the font catalog.

Once we have a font catalog there is not reason not to save other
small bits of interesting information to make creating an in-memory
font catalog quickly.

Comment 105

[kill this account]

> Is there a lower point size bound below which your code 
> disables AA?  

Not at present. 

I know that the Mac does this so this is probably a good 
thing to consider.

> Maybe I need to use-my-fonts-not-theirs pref.

Might want to try displaying a plain text file to make sure
Mozilla is using the fonts specified in font preferences.

Comment 106

[kill this account]

If people have trouble getting a patched version to render AA
please let me know an I will produce a patch with debugging 
printf's

Comment 107

[keithp]

> Xft does a client-side glyph render, XPutImage, and a server 
> anti-alias in software (if the hardware is not available).

Xft doesn't use XPutImage; the Render extension includes a new object called
a GlyphSet which holds images for each glyph.  Xft renders the glyph in the
client and transmits the glyphs for storage within the GlyphSet.  Because
TrueType glyphs are represented only as alpha channels, the image transfered
is only 8 bits per pixel unlike window contents which are either 16 or 32
bits per pixel.

> The difference is: a XGetSubImage (and hardware AA when available).

The difference is that Render doesn't transmit glyph images to render
glyphs, only to place them in the X server glyphset.  Rendering is done by
passing glyph indices expressed as 1,2 or 4 byte values.  Xft consumes
essentially the same bandwidth for rendering as the core protocol.

Further, as Render doesn't require that information pass from the X server
to the client, it is entirely asynchronous and so works very well over
networked environments, even better than the core protocol can.

Finally, the software rendering need only fetch pixels from the screen which
are needed for rendering; transparent pixels are skipped while opaque pixels
are simply written.  For well hinted text, this means that the number of
PCI read cycles is usually quite small.  As PCI read cycles are many times
slower than write cycles, the resulting performance is far better than could
ever be achieved with client-side rendering.  Of course, hardware
acceleration can also be used which yields additional dramatic performance
gains.

> I do remember seeing code in Xft that said give up FreeType
> if Render is not available but I do not remember seeing code to give
> up AA if hardware is not available. You might care to look yourself
> in xftdpy.c around line 76

Hardware isn't really required; the ability to precisely control screen read
cycles is all that's really needed for reasonable text performance.  Even
relatively old machines can render tens of thousands of glyphs per second in
software.

It would be interesting to get measurements of text performance done on the
client side; subjective information isn't very helpful.

> It is clear that we should use the font name in the TrueType file.

Where the user is directly involved in font selection, the real face names
should be presented.  When applications receive face names from other
environments, the font system must attempt to match fonts as best it can.
As Panose numbers have fallen out of favor, we're essentially left with face
aliases of one kind or another.  This means we'll need a relatively large
database of suitable aliases customized to match available faces.  Sharing
this information among applications on the same screen will improve the
overall consistency of the display.

> It makes no sense for Xft to try to wrapper every FreeType2 API.

Of course not; Xft exposes the FreeType objects so that applications may
directly use the underlying system.  Xft provides font matching and Render
support on top of FreeType.  Just how much additional support is warrented
depends on how much commonality can be found among the various Xft users.

You'll note that Xft is a relatively small library, less than 10K lines of
code generating less than 100K of object.  It's dwarfed by the Mozilla
libraries of over 3 megabytes.

> (assuming I have this info about Xft right)
> Xft's font list API returns results which include the core 
> fonts. If we merge at the nsFontGTK layer we will need to add
> code to winnow out the core X fonts since those are to be
> dealt with in the old code. 

Xft only returns core fonts if the Render extension isn't present.  When
local bitmap font reading is added to Xft, it will return those fonts as
well.  There are still many locales not well served by TrueType fonts, but
Xft will still be useful as there are bitmapped fonts which cover the
appropriate 10646 space.  Because core X fails miserably with large fonts,
the Render extension and client-side fonts are necessary even in this case.

As yet undecided is what format these fonts will be delivered in; the only
currently available format capable of fully supporting 10646 is BDF, an
ASCII format which is rather expensive to incrementally access glyphs and
metrics from.  X will likely need a new binary font format tuned to manage
large 10646 encoded bitmapped fonts, Xft will support PCF, BDF along with
this new format.

> How does Xft manage client and server memory bloat as more
> and more fonts and glyphs are loaded?

Currently, Xft simply allows the server memory size to grow without bound.
Because glyphs are shared wherever possible, this hasn't proven to be a
problem so far, even when displaying rather complex documents with a large
number of unique glyphs.  As I expect this will eventually become an issue,
the Render extension provides for discarding glyphs from glyphsets which Xft
can use to reduce the server memory footprint.


Because Xft doesn't save the glyph images locally, there's no significant
impact in the application aside from the glyph metrics which are generally
small in comparison to the 8-bit glyph images.  Of course, when Xft discards
the server-side glyph images, it will probably discard the client-side
metrics at the same time.

However, in a typical system, the impact of glyph images on the X server
memory use is relatively small compared to the sauropods -- modern X
applications tend to store vast images within the server making our puny
glyph unloading seem pretty futile.


>    for (i=0; i<0x10000; i++) {
>          glyph_index = FT_Get_Char_Index(face, (FT_ULong)i);
>               if (glyph_index !=0) {

This seems to ignore the fact that many TrueType fonts present a broken
Unicode mapping; I've a large set of fonts (about 1200) which include both a
Unicode and an Apple-Roman mapping.  The Unicode map doesn't include any
characters which are in the Apple-Roman mapping.  Xft will (eventually) loop
through the available encoding tables looking for each glyph.

This particular project has been stymied by the difficulty in deciding
precisely how to adapt the existing Xft APIs to support a standard Unicode
encoding while also supporting direct access by glyph index.

> The fact that Mozilla needs this information and it is too expensive
> to do at startup or lazily at run time is the primary item Mozilla
> needs from the font catalog.

Any data which involves opening every font at application startup time is
too expensive -- I've users with thousands of TrueType fonts.  Xft needed a
font information cache just to store relatively simple font properties to
speed application startup.

> Once we have a font catalog there is not reason not to save other
> small bits of interesting information to make creating an in-memory
> font catalog quickly.

Indeed.  I think the Xft catalog holds everything needed except for the set
of available glyphs.  The internal structures already support the necessary
datatypes, all I was missing was a way to generate an accurate listing.

Comment 108

[Decklin Foster]

> If people have trouble getting a patched version to render AA
> please let me know an I will produce a patch with debugging 
> printf's

Please do; I'm having the same problem. If I open up prefs and use one of the
new capitalized font names, I get Times or Helvetica instead (depending on if I
picked "serif" or "sans serif" for the default font).

I copied one of your printfs and got:

sFreeTypeAntiAlias = 1, nsFreeType.cpp 192

so I know it's probably not a failure to set that variable, but I don't know
where else to look.

Comment 109

[Brice Ruth]

This doesn't seem to have anything to do with the patch, but since its
preventing me from testing it, I figured I'd ask the knowledgeable Mozilla
people here anyway :)

I applied the patch and everything was happy - I then did:

gmake -f client.mk build

From mozilla/.  Things seemed to chug along fine, until regExport was linked at
which time I got an error stating that libxpcom.so has an undefined symbol
PR_Assert.  I've tried tracking this down, but I'm at my wit's end.  I was able
to compile mozilla-0.9.3 from Mandrake's src.rpm w/o any problems.  I've
compiled 0.9.2 and 0.9.1 in the same manner, w/o any problems.

Any help would be appreciated!

Comment 110

[kill this account]

> Xft doesn't use XPutImage...
> 
> ... Render doesn't transmit glyph images to render glyphs ...
> Xft consumes essentially the same bandwidth for rendering as 
> the core protocol.
> 
> Further, as Render ... works very well over
> networked environments, even better than the core protocol can.
> 
> ... the resulting performance is far better than could
> ever be achieved with client-side rendering.  
> 
> ... [using Render] Even relatively old machines can render tens 
> of thousands of glyphs per second in software.

Very impressive. Clearly the thing to use (where available).

While we are still figuring out what to do about font naming and
printing why don't you start filling out the Render (w/wo Xft) 
code in the patch. You could actually make a working Mozilla with 
all your fast code. 

It would be an opportunity to see how to best fit Xft and 
Mozilla.

(Besides, let's stop arguing about who's way is better.  Lets use 
 all this energy and make some working code. The world is big
 enough for multiple solutions.)

I have a stub class in place for Render: nsFreeTypeRender

Basically copy&rename the needed parts from the nsFreeTypeXImage
class.

The Render class should be created here:

    NS_ASSERTION(0, "need to construct a render type FreeType object")

> Where the user is directly involved in font selection, the real face names
> should be presented.  

agreed

> When applications receive face names from other
> environments, the font system must attempt to match fonts as best it can.
> As Panose numbers have fallen out of favor, we're essentially left with face
> aliases of one kind or another.  This means we'll need a relatively large
> database of suitable aliases customized to match available faces.  Sharing
> this information among applications on the same screen will improve the
> overall consistency of the display.

Mozilla has font name aliasing.  Xft has font name aliasing.

As the actual person who has to fix the bugs related to font naming
I am very cautious about changing this kind of thing without a
direct benefit to Mozilla.

If someone else would like to volunteer to accept the responsibility 
for fixing the Mozilla bugs related to this we should talk.

> You'll note that Xft is a relatively small library, less than 10K lines of
> code generating less than 100K of object.  It's dwarfed by the Mozilla
> libraries of over 3 megabytes.

I have not issue with the size of Xft. 

I am concerned with the stability.

Would you care to make a statement about why it would be a good 
thing to tie Mozilla to a library that makes this statement (or 
let me know if there is a more recent statement superseding this):

http://www.xfree86.org/4.1.0/Xft.3.html#toc5

    Xft will probably change radically in the future; 
    weak attempts will be made to retain some level of
    source-file compatibility. 

> 
> > (assuming I have this info about Xft right)
> > Xft's font list API returns results which include the core 
> > fonts. If we merge at the nsFontGTK layer we will need to add
> > code to winnow out the core X fonts since those are to be
> > dealt with in the old code. 
> 
> Xft only returns core fonts if the Render extension isn't present.  When
> local bitmap font reading is added to Xft, it will return those fonts as
> well.  There are still many locales not well served by TrueType fonts, but
> Xft will still be useful as there are bitmapped fonts which cover the
> appropriate 10646 space.  Because core X fails miserably with large fonts,
> the Render extension and client-side fonts are necessary even in this case.

Since Mozilla already has a large base of code to composite fonts
having Xft try to do the same thing (font sets) under the hood 
just adds another layer of complexity (and potential bugs).

Replacing Mozilla's (nsFontMetricsGTK) font fill in code with Xft's 
would be very destabilizing for Mozilla. Since I do not see an 
advantage for Mozilla I strongly recommend against it.

You could of course create your own layer like the beos, motif, os2, 
qt, and xlib groups have.

eg: make a mozilla/gfx/src/xft that provides the same functionality
as any of these:

    mozilla/gfx/beos
    mozilla/gfx/gtk
    mozilla/gfx/gtk2
    mozilla/gfx/mac
    mozilla/gfx/motif
    mozilla/gfx/os2
    mozilla/gfx/photon
    mozilla/gfx/ps
    mozilla/gfx/qt
    mozilla/gfx/rhapsody
    mozilla/gfx/windows
    mozilla/gfx/xlib
    mozilla/gfx/xlibrgb
    mozilla/gfx/xprint

    

> As yet undecided is what format these fonts will be delivered in; the only
> currently available format capable of fully supporting 10646 is BDF, an
> ASCII format which is rather expensive to incrementally access glyphs and
> metrics from.  X will likely need a new binary font format tuned to manage
> large 10646 encoded bitmapped fonts, Xft will support PCF, BDF along with
> this new format.

Kindly keep me informed as things progress.

> > How does Xft manage client and server memory bloat as more
> > and more fonts and glyphs are loaded?
> 
> Currently, Xft simply allows the server memory size to grow without bound.
> Because glyphs are shared wherever possible, this hasn't proven to be a
> problem so far, even when displaying rather complex documents with a large
> number of unique glyphs.  As I expect this will eventually become an issue,
> the Render extension provides for discarding glyphs from glyphsets which Xft
> can use to reduce the server memory footprint.

Kindly keep me informed as things progress.

> Because Xft doesn't save the glyph images locally, there's no significant
> impact in the application aside from the glyph metrics which are generally
> small in comparison to the 8-bit glyph images.  Of course, when Xft discards
> the server-side glyph images, it will probably discard the client-side
> metrics at the same time.

sounds good.

> >    for (i=0; i<0x10000; i++) {
> >          glyph_index = FT_Get_Char_Index(face, (FT_ULong)i);
> >               if (glyph_index !=0) {
> 
> This seems to ignore the fact that many TrueType fonts present a broken
> Unicode mapping; I've a large set of fonts (about 1200) which include both a
> Unicode and an Apple-Roman mapping.  The Unicode map doesn't include any
> characters which are in the Apple-Roman mapping.  Xft will (eventually) loop
> through the available encoding tables looking for each glyph.
> 
> This particular project has been stymied by the difficulty in deciding
> precisely how to adapt the existing Xft APIs to support a standard Unicode
> encoding while also supporting direct access by glyph index.

Kindly keep me informed as things progress.

> > The fact that Mozilla needs this information and it is too expensive
> > to do at startup or lazily at run time is the primary item Mozilla
> > needs from the font catalog.
> 
> Any data which involves opening every font at application startup time is
> too expensive -- I've users with thousands of TrueType fonts.  Xft needed a
> font information cache just to store relatively simple font properties to
> speed application startup.
> 
> > Once we have a font catalog there is not reason not to save other
> > small bits of interesting information to make creating an in-memory
> > font catalog quickly.
> 
> Indeed.  I think the Xft catalog holds everything needed except for the set
> of available glyphs.  The internal structures already support the necessary
> datatypes, all I was missing was a way to generate an accurate listing.

Would you mind if we move the font catalog design to a different venue?

I don't have somewhere else in mind but this bug is already so long I
am considering closing it and opening another.

Comment 111

[kill this account]

> From mozilla/.  Things seemed to chug along fine, until regExport was linked 
> at which time I got an error stating that libxpcom.so has an undefined symbol
> PR_Assert.  I've tried tracking this down, but I'm at my wit's end.  I was 
> able to compile mozilla-0.9.3 from Mandrake's src.rpm w/o any problems.  I've
> compiled 0.9.2 and 0.9.1 in the same manner, w/o any problems.
> 
> Any help would be appreciated!

I am also having build problems at work (but not at home!?!)

Comment 112

[kill this account]

> From mozilla/.  Things seemed to chug along fine, until regExport was linked 
> at which time I got an error stating that libxpcom.so has an undefined symbol
> PR_Assert.  I've tried tracking this down, but I'm at my wit's end.  I was 
> able to compile mozilla-0.9.3 from Mandrake's src.rpm w/o any problems.  I've
> compiled 0.9.2 and 0.9.1 in the same manner, w/o any problems.
> 
> Any help would be appreciated!

I am also having similar run time linker problems at work (but not at home!?!)

Comment 113

[Jeffrey Baker]

Brian, I ran with your patch again and still, I'm not seeing AA fonts or really
any difference in rendering at all.  I loaded a text file off the disk.  Mozilla
uses my monospace font (Agfa Monotype Courier New 12pt) for the file, but it is
decidedly not AA.  Might there be an issue of me running at 16-bit, or havin the
Render extension available (it is)?

Also Mozilla seems to be parsing all of my fonts every time I load Mozilla.  At
least, it says "parsing xyz (123 glyphs)" for every font at every load.

Comment 114

[Jeffrey Baker]

More info:  I switched to depth 24/32, and no change.  The parsing-on-load
problem seems to be this:  if I pick my profile from the profile manager,
parsing happens every time, but if I use "mozilla -P jwb" no parsing occurs. 
Might be a bug here or in profile management.

Comment 115

[kill this account]

Attachment: patch; debugging added to nsFontCatalog.cpp

Comment 116

[kill this account]

I added runtime debugging statements to nsFontCatalog.cpp in the patch in
attachment http://bugzilla.mozilla.org/showattachment.cgi?attach_id=45257

These messages will report which fonts are queried / found in the FreeType
font catalog as well as activity when the font catalog is first being built.

The font catalog messages are output when the 0x20 bit is set in the 
NS_FONT_DEBUG environment variable; eg under bash:

    NS_FONT_DEBUG=20 ./mozilla

I added one FreeType draw message which will output when the 0x40 bit is set
in the NS_FONT_DEBUG environment variable; eg under bash:

    NS_FONT_DEBUG=40 ./mozilla

To see the font catalog messages and the FreeType drawing message set both bits

    NS_FONT_DEBUG=60 ./mozilla


FYI re: patching
1) the patch was made in the mozilla dir
2) previous patches will need to be uninstalled
3) to test the patch I run the following command (under bash)

   POSIXLY_CORRECT=1 patch --dry-run -p0 < patch.file gfx/src/gtk/Makefile.in

(setting POSIXLY_CORRECT should make patch find the Makefile.in in gfx/sc/gtk/
but due to a bug it finds Makefile.in in the current (mozilla) dir. I added the 
gfx/src/gtk/Makefile.in at the end to make patch find the right file)

4) to do patch I use the same command less the "--dry-run"

Comment 117

[Decklin Foster]

Here are my results with NS_FONT_DEBUG=60. I'm still seeing Helvetica instead of
anti-aliased Arial (what I selected).

http://www.red-bean.com/~decklin/temp/moz-freetype-debug.log

Comment 118

[kill this account]

Attachment: simple html spec'ing an arial font

Comment 119

[kill this account]

Let's see what it says 
(I'm including my analysis here in case it might help others)

> sFreeTypeDepth = 24, nsFreeType.cpp 175

This is good. As the code is only demo grade the alpha blending 
code only supports 24/32 bit frame buffers.

> look for TrueType fonts in /home/bstell/tmp_font, ...
> failed to stat dir /home/bstell/tmp_font, nsFontCatalog.cpp 1560

Where/how to find the font files is not resolved.

> look for TrueType fonts in /usr/local/lib/ttffonts, ...

Since that patch looks in /usr/local/share/fonts/ttfonts it is
clear you found and modified the line in unix.js

> no CodePageRange bits but 377 glyphs, coronet, ...
> no CodePageRange bits but 377 glyphs, marigold, ...

Interesting, a font that does not have any languages indicated
in the OS/2 table but has more fonts than Latin1 might indicate.
I will want to look at the font if possible.

It is clear from the missing catalog generation output that
the catalog was generated previously.

> looking for FreeType font matching 
> -Agfa Monotype-arial-*-*-*-*-*-*-*-*-*-*-iso8859-1, ...

Looks like this was called from FindStyleSheetSpecificFont().
Unless the page's font spec looked like this:
<font face="Agfa Monotype-arial-iso8859-1">
the font spec came from font prefs.

> found FreeType -arial-iso8859-1, nsFontCatalog.cpp 1455

The code found a TrueType/FreeType font. It should be used.

> looking for FreeType font matching 
> -Agfa Monotype-arial-*-*-*-*-*-*-*-*-*-*-iso8859-*, ...
> nsFontCatalogGetFontNames failed, nsFontCatalog.cpp 1464

This is okay. The font fill-in / glyph lookup code in Mozilla
is fairly complex and does many things trying to find fonts.

The TrueType font catalog lookup supports the more interesting
lookups:

  -foundry-family-...-registry-encoding
  -foundry-family-...-*-*

but not

  -foundry-family-...-registry-*

(wild card just the encoding)

Do note that when the "-foundry-family-...-registry-*"
(wild card just the encoding) search does not give a font the 
Mozilla code next tries "-foundry-family-...-*-*" (wild card 
the registry and the encoding) which the font catalog does
support.

I see fonts being found but I do not see the
"nsFreeTypeXImage::DrawString, nsFreeType.cpp 576" messages.

Do you see anti-aliased fonts be showing in the menus or
buttons?

Try this:

Make sure Arial is *not* selected in font prefs and then
kindly try displaying the simple html in attachment 45426 [details] and
attach moz's output with NS_FONT_DEBUG=60

I would expect to see something like this:

looking for FreeType font matching -*-arial-*-*-*-*-*-*-*-*-*-*-*-*
found FreeType <foundry>-arial-*-*, nsFontCatalog.cpp 1424

Comment 120

[Decklin Foster]

> Since that patch looks in /usr/local/share/fonts/ttfonts it is
> clear you found and modified the line in unix.js

Actually, I just set

user_pref("freeType.font.dir.1", "/usr/local/lib/ttffonts");

in my profile's prefs.js (that's the location of the fonts on my machine), but I
assume the effect is the same.

> Interesting, a font that does not have any languages indicated
> in the OS/2 table but has more fonts than Latin1 might indicate.
> I will want to look at the font if possible.

http://www.red-bean.com/~decklin/temp/coronet.ttf
http://www.red-bean.com/~decklin/temp/marigold.ttf

> It is clear from the missing catalog generation output that
> the catalog was generated previously.

Yes; here's the catalog that was generated.

http://www.red-bean.com/~decklin/temp/mozilla_font_summary.txt

> the font spec came from font prefs.

Right; I tried loading a page with no font information. I've also loaded pages
specifying other TT fonts (still aliased), but not while generating that log.

> Do you see anti-aliased fonts be showing in the menus or
> buttons?

No, they're not showing up anywhere.

> I see fonts being found but I do not see the
> "nsFreeTypeXImage::DrawString, nsFreeType.cpp 576" messages.

This is puzzling to me as well. Setting NS_FONT_DEBUG=40 produces no debugging
messages at all.

> Make sure Arial is *not* selected in font prefs and then
> kindly try displaying the simple html in attachment 45426 [details] and
> attach moz's output with NS_FONT_DEBUG=60

That attatchment displays in Arial, but aliased. Here's the results of running

NS_FONT_DEBUG=60 mozilla
'http://bugzilla.mozilla.org/showattachment.cgi?attach_id=45426'

and hitting Ctrl-+ several times to make sure the font really was aliased (I
used xmag to verify), then quitting.

http://www.red-bean.com/~decklin/temp/moz-freetype-debug-2.log

Any ideas?

Comment 121

[kill this account]

Decklin and I tracked down his error to FT_Load_Glyph returning an error code 
0x422 which did not appear to be a valid error code.

From email on the freetype <[e-mail]> mailing list David Turner 
wrote:
>   the error codes have been changed between 2.0.2 (the version
>   of FT2 you're user has installed on its system) and 2.0.4,
>   which is why "0x442" isn't defined anymore..
> 
>   (and it was a TrueType-specific thing at that time)
> 
>   I suggest asking your user to update its version of FreeType
>   to see if it solves the problem. If not, let us know and we
>   will look deeper into this problem..

Comment 122

[kill this account]

Attachment: patch; at compile checks for FreeType2 dev env; at runtime loads libfreetype.so if avail

Comment 123

[kill this account]

attachment 47990 [details] [diff] [review] adds:

1) At build time: if configure finds FreeType includes then the 
   nsFreeType code is compiled in else not.

2) At run time: if PR_LoadLibrary can load libfreetype.so then the
   nsFreeType code is enabled else not.

3) Add user pref and code: minimum size to anti-alias FreeType glyphs.

4) Adding run-time debugging printf's.

5) Miscellaneous code cleanup.

Comment 124

[kill this account]

Attachment: png image showing http://home.netscape.com/ja using a Union Way font

Comment 125

[kill this account]

Attachment: patch; added support for embedded bitmaps

Comment 126

[kill this account]

Attachment: same patch as before but with gfs/src/gtk/Makefile.in first

Comment 127

[kill this account]

I find that even with POXISLY_CORRECT set patch tries to apply the
gfs/src/gtk/Makefile.in diffs to ./Makefile.in which is wrong.

I can add the path to the command line but it only appears to work if
it is the first parameter.

Hence the revised patch with gfx/src/gtk/Makefile.in moved to the top.

Here is the command line I use:

  POSIXLY_CORRECT=1 patch --dry-run -p0 gfx/src/gtk/Makefile.in < patch.file

Comment 128

[kill this account]

Also note that since the patch changes configure.in that after the patch is
applied, the autoconf command must be run (normally this is done automatically
by leaf) and a full rebuild must be done.

Comment 129

[kill this account]

oops, the correct command is without the '--dry-run'

  POSIXLY_CORRECT=1 patch -p0 gfx/src/gtk/Makefile.in < patch.file

Comment 130

[kill this account]

I find that my Redhat 7.2 system will not build unless I remove this from
config/autoconf.mk.in

  FREETYPE2_INCLUDES      = @FREETYPE2_INCLUDES@

Comment 131

[Nathaniel Daw]

I have been playing with this code and noticed a number of minor buglets. Note
that I compiled against freetype 2.01 and subsequently updated to run against
2.03, which is probably less than ideal. But here's some things i've noticed; I
haven't done much digging in the source yet.

* First, it bases the decision on whether to activate antialiasing on whether it
can load freetype.so, rather than freetype.so.6 or whatever it was compiled
against. Had it been dynamically linked directly at compile time, it would want
the latter. The problem with the former is that in Redhat at least, the .so
links come in the devel packages, so someone can very well have the library
installed but have this test fail.

* Second, I have problems getting antialiasing to appear at all for default
fonts selected in the preferences box, which seem to relate to the code having a
hardcoded list of foundry names that doesn't match what e.g. ttmkfdir thinks the
foundry names for a font are, together with some kind of case-sensitive matching
on them. E.g. I am offered the chance to select both

Agfa Monotype-andale mono-iso8859-1 and
monotype-andale mono-iso8859-1 (which is what X thinks the font is); only the
former gets antialiased.

Similarly, between Microsoft-georgia and microsoft-georgia, only the capitalized
one works.

* I have seen descending glyphs disappearing completely and ascending ones
clipped (see http://www.cs.cmu.edu/~daw/mozilla1.png -- note the "Bugzilla"
header) in header text.

* I have seen truly gruesome lack of hinting in some text, which I suspect
relates to the font being rendered at a large size then scaled down later in
rendering?!? See http://www.cs.cmu.edu/~daw/mozilla2.png -- the font is
garamond, and though it is antialiased there it is obviously in bad shape. The
same font (indeed, the same page) looks great in other apps like konqueror,
antialiased or not. See http://www.cs.cmu.edu/~daw/konq.png

Comment 132

[kill this account]

> ... I compiled against freetype 2.01 and subsequently 
> updated to run against 2.03, which is probably less than 
> ideal. 

You need to compile and run against 2.0.4. I already know
the 2.0.2 does not work due to a major change between it
and 2.0.3 (why didn't they at least change the middle
number?). I have asked the FreeType developer group how I 
might distinguish the two at run time but they have not
replied.

> * First, it bases the decision on whether to activate 
> antialiasing on whether it can load freetype.so, rather 
> than freetype.so.6 or whatever it was compiled against. 
> Had it been dynamically linked directly at compile time, 
> it would want the latter. The problem with the former is 
> that in Redhat at least, the .so links come in the devel 
> packages, so someone can very well have the library
> installed but have this test fail.

We certainly cannot link to the library at build time for
2 separate reasons: 

  1) This would require all Linux developers to upgrade 
     their systems on the day that this got checked in
     which would make them very unhappy.

  2) This would require all Linux users to upgrade their
     system when they install the new client.

I do not claim to be an expert on Linux librariy extensions. 
If I can get a confirmation on this I will change it.

> * Second, I have problems getting antialiasing to appear 
> at all for default fonts selected in the preferences box, 
> which seem to relate to the code having a hardcoded list 
> of foundry names that doesn't match what e.g. ttmkfdir 
> thinks the foundry names for a font are, together with 
> some kind of case-sensitive matching on them. E.g. I am 
> offered the chance to select both
>
> Agfa Monotype-andale mono-iso8859-1 and monotype-andale 
> mono-iso8859-1 (which is what X thinks the font is); only 
> the former gets antialiased.

Those starting with lower case, come from xfs which does not 
anti-alias.

As far as listing both: Is anyone suggesting we should not
allow the user to pick the fonts that are reported by 
XListFonts?

Those that start with upper case come from this patch which
does support AA and the foundry names come from MIcrosoft's
registered vendors list 
http://www.microsoft.com/typography/links/vendorlist.asp
(which seems to have new members even since I last looked).

> * I have seen descending glyphs disappearing completely and 
> ascending ones clipped 

It works correctly on my system. Would you kindly rebuild 
with FreeType 2.0.4 and see if this continues?

> * I have seen truly gruesome lack of hinting in some text, 

I'm glad to finally hear someone talking about the rendering
quality. I tried to follow the FreeType example code. As
you point out the type quality appears differently. I do
admit that I am not yet wildly happy with the quality.

Would you kindly install FreeType 2.0.4, force a rebuild of
the code in the patch and then run both apps with FreeType 
2.0.4 and report back with the difference?

Could you also attach a screen dump of the same text output
with the reference display code "ftview"? So far when I have
done this I have not seen a difference but I am very
interested in this.

Thanks

Comment 133

[Nathaniel Daw]

I'll try out the 2.0.4 rebuild as requested. On a couple of your other points:

I didn't mean to suggest that you should link at compile time. Rather, when you
dynamically load the library at runtime the file you should load is
libfreetype.so.6 rather than libfreetype.so. My point about compile time dynamic
linking was just that freetype.so.6 is the file that it would look for if you
did it that way, so it is presumably the right choice for your runtime loading
as well. As I understand it, the foo.so file is only there to point the compile
time linker to the major version to link against, the dynamic runtime linker
loads the .so.x file to ensure that it loads the same major version the program
was compiled against, and this in turn points to foo.so.x.x, the actual library
with a minor version number so the user can swap in a minor upgrade.

By analogy, it seems your code should figure out at compile time what
libfreetype.so points to, e.g. libfreetype.so.6, and this is the file it should
check for and load at runtime. This is important, as I said before, because
otherwise a redhat user who has the freetype library installed (which has .so.x
and .so.x.x), but not the freetype-devel package (which is the one that has the
.so symlink) will otherwise not be able to use your code. Your code will
incorrectly conclude that the user doesn't have the library because he doesn't
have the symlink used by the compile-time linker.
Secondly, on the issue of the font menu, I didn't mean to imply that you
shouldn't allow access to non-freetype fonts (though this wouldn't be the end of
the world IMHO -- it is how kde works I think). But it is bizarre and confusing
that the same fonts show up in the menu twice, and even stranger that how the
font looks depends on whether you choose the one with a capital letter. If you
are combining fonts from two sources, shouldn't you prune duplicates, favoring
the freetype version? This is of course harder because you have a different idea
of the foundry names than does xfs. Another point is, since you use different
names, when someone updates to your code, their fonts will not become
antialiased even if they were already truetype, until they go select them under
a different name. Not a big deal, but it seems it would be nice if you could
catch requests to an xfs font that is the same as a freetype font, and route it
through your renderer.

Comment 134

[kill this account]

> If you are combining fonts from two sources, shouldn' you prune duplicates,
> favoring the freetype version? 

No, some users feel strongly that AA fonts are blurry not smoothed and will 
want to use the non-AA version.

> Another point is, since you use different names, when someone updates to your 
> code, their fonts will not become antialiased even if they were already 
> truetype, until they go select them under a different name. Not a big deal, 
> but it seems it would be nice if you could catch requests to an xfs font that 
> is the same as a freetype font, and route it through your renderer.

Careful, I believe the family name is the same but the font name is different. 
CSS almost always specs the family name and not the foundry name.

We should not *force* the user to use an AA font if available.  We definitely 
want to allow the user to select between them when the font selection comes 
from preferences; ie: no CSS spec. We will probably also need to add a control 
to select the AA or non-AA when the font spec comes from HTML/CSS.

Comment 135

[Brice Ruth]

> No, some users feel strongly that AA fonts are blurry not smoothed and will 
> want to use the non-AA version.

The user should either turn on or turn off anti-aliased fonts - if anti-aliased
fonts are turned on, duplicates should be removed.  I don't believe that 98% of
the users of Mozilla and 99.999% of the users of NS6.x on FreeType systems will
want to be confused by duplicates ... the only reason to have *duplicates* would
be if someone wanted to select anti-aliased fonts for some uses and non AA fonts
for others.  I'd say that this is going to happen rarely to not at all, right?

> Careful, I believe the family name is the same but the font name is different. 
> CSS almost always specs the family name and not the foundry name.
>
> We should not *force* the user to use an AA font if available.  We definitely 
> want to allow the user to select between them when the font selection comes 
> from preferences; ie: no CSS spec. We will probably also need to add a control 
> to select the AA or non-AA when the font spec comes from HTML/CSS.

Just one simple checkbox to select either AA or non-AA and maybe a selection to
indicate at what font-size to stop doing AA (since I'm guessing your code
controls this, too) - this should probably default to 8 or 9 (currently defaults
to 14 in some KDE systems which cuts it off *way* to early, IMHO).  Let's not
make the AA system so absurdly difficult to use that it becomes unusable for a
novice.  If you *want* to allow more advanced controls, expose them via prefs.js
settings, but make things default to popular settings - let's compromise.

Comment 136

[kill this account]

> the only reason to have *duplicates* would be if someone wanted to select 
> anti-aliased fonts for some uses and non AA fonts for others.  I'd say that 
> this is going to happen rarely to not at all, right?

Other way 'round. Some of the AA fonts look pretty good when rendered by
FreeType. Other look fairly poor. I am fairly sure I for one will want some
fonts AA and some non-AA.

Comment 137

[Brice Ruth]

> Other way 'round. Some of the AA fonts look pretty good when rendered by
> FreeType. Other look fairly poor. I am fairly sure I for one will want some
> fonts AA and some non-AA.

Shouldn't this be an 'Advanced' option?  As long as Times, Verdana, Arial,
Helvetica and Courier AA well, you've got most of your users covered.  Chances
are that if a font does AA well in FreeType, it probably doesn't AA well
elsewhere (possibly in Windows, possibly in MacOS - though I've not seen a *bad*
font yet).  If so, a designer will know not to use that particular font in HTML
- choosing instead to present it as Flash text or an image (GIF/PNG).  Let's
optimize for the common case, make sure that everything smokes with that and
*then* worry about tweaking it for a few advanced scenarios - I don't want to
cut out the advanced user who has special needs, I just want things to *rock*
for the average user.

Comment 138

[Jeffrey Baker]

The size cutoff may seem inappropriate to you, but it is resolution dependent. 
The target should make sense at some kind of "normal" resolution, sa 96 DPI.

Comment 139

[Brice Ruth]

> The size cutoff may seem inappropriate to you, but it is resolution dependent. 
> The target should make sense at some kind of "normal" resolution, sa 96 DPI.

Where exactly does this 96dpi number come from?  I've seen it tossed around
quite a bit, but every designer worth his/her salt that I've talked to says that
when designing for web, you design to 72dpi which is standard screen res.  I
believe, on my laptop (Dell Latitude CPi D266XT) which has a 12.1" screen & runs
at 1024x768, I get a screen res of 75x75 reported by XFree86 (can't be
*positive* on that, but its either that or 72x72).  Clarify for me, please.

Comment 140

[Boris Zbarsky [:bzbarsky]]

> which has a 12.1" screen & runs at 1024x768, I get a screen res of 75x75 
> reported by XFree86

XFree86 is confused, I would say.  That resulution comes out to 1280 pixels along 
the diagonal, which with a 12.1" diagonal is about 106dpi

For the more common (imo) case of 1024x768 on a 15-inch monitor (14.5" diagonal) 
we get 88dpi.

For 800x600 on the same 14.5" diagonal we have about 69dpi

For 1280x1024 on a 16.5" diagonal (hopefully no one is crazy enough to run that 
on smaller screens) we have 99dpi

96dpi is fairly average for people running high-res setups.

Comment 141

[keithp]

For complete control over when to use AA and even when to use core fonts, the
Xft library has a (overly) sophisticated matching and editing mechanism that can
do what has been suggested here.  Using Xft has the advantage that user
customizations through Gnome or KDE will be followed in Mozilla as well.  As
Brian says, some fonts look great rasterized by FreeType and others are very
hard to read

As for display resolutions; I've seen average DPI values slowly increase from
75dpi in 1990 to 90 to 100 dpi today.  In addition, AA text (obviously) looks
fuzzier the larger the pixels are, I believe people complaining about AA text
are likely to be running their monitors closer to 75 dpi than 100dpi.  A large
part of that reason is that web pages are all designed for this resolution and
*the browsers can't magnify images*.  If the browser had a way of scaling the
entire web page by 100/75, users could take advantage of the higher quality text
display at 100dpi without suffering from tiny images.

Now that all of my text (aside from this, as I'm using Mozilla to access
bugzilla) is anti-aliased, the pixels on my screen have essentially disappeared
and I am left looking the underlying objects.  The fact that I run at 1600x1200
doesn't affect anything aside from the clarity of text.  Solve the remaining
issues relating to running at higher screen resolutions and users will be more
accepting of tiny pixels which will make them happier with AA text.

Comment 142

[db]

> If the browser had a way of scaling the
> entire web page by 100/75

Forgive me for saying it here, but opera does have this feature and it's
wonderful. I couldn't find a mozilla bug suggesting this, maybe I should file
one. I guess it can be a rather big change though, depending on how mozilla
works now.

Comment 143

[Christopher Blizzard (:blizzard)]

<star wars>Stay on target.</star wars>

Comment 144

[kill this account]

Lets get it working first and then tune it up.

I've opened the following bugs:

bug 100564 - user control to enable/disable font anti-aliasing

bug 100565 - fontlist: merge Anti-Aliased fonts with non-AA fonts 

Kindly add yourself to the cc: list as appropiate.

Comment 145

[kill this account]

This bug now has over 4,000 lines of comments and I find it hard to scroll. 

Thus I am opening bug 100570 as "Part 2".

Kindly cc: youself as appropiate.

Sorry for any inconvenience.

thanks

*** This bug has been marked as a duplicate of 100570 ***

Comment 146

[kill this account]

I've also opened this bug:

bug 100572 - integrate font control with system font controls (Xft)

Kindly add yourself to the cc: list as appropiate.

Comment 147

[Teruko Kobayashi]

Changed QA contact to ylong since she is QA contact on bug 100572.

Comment 148

[kill this account]

I have spent some time looking at TrueType printing and bug 90385 is not
blocked by this bug

Comment 149

[Matt Perry]

v