90813 - anti-alias bitmap scaled fonts to make them less ugly

Status: VERIFIED FIXED

(Core :: Internationalization, defect)

Description

[kill this account]

Traditional X bitmap scaled fonts are very ugly.
If the Xserver has 16/24 bit pixels we should anti-alias the scaled bitmaps
as these look much better than plain scaled bitmaps.

Comment 1

[kill this account]

Attachment: image showing 16px Japanese anti-aliased bitmap scaled to 12 px

Comment 2

[kill this account]

Attachment: sample program that anti-alias-scales a bitmap; tar/gz

Comment 3

[kill this account]

Attachment: a bigger image of Japanese anti-aliased bitmap scaled from 16px to 12 px

Comment 4

[kill this account]

Attachment: image of Japanese anti-aliased scaled 16 px to 22 px

Comment 5

[kill this account]

Attachment: image of regular bitmap scaled Japanese for comparison

Comment 6

[kill this account]

Attachment: image showing 1) regular 16 px Japanese, 2) AA scaled to 12, 3) AA scaled to 22, 4) regular bitmap scaled to 12px

Comment 7

[Teruko Kobayashi]

Changed QA contact to [e-mail].

Comment 8

[kill this account]

The scaling in the attachment.cgi 42297 does the scaling by expanding
the source into a a temporary buffer that is the Cartesian product of
the source and destination. For a source image of Sw*Sh and a
destination of Dw*Dh the temporary buffer is Tw=Sw*Dw wide and
Th=Sh*Dh high; eg: for a 20x30 source expanding into a 30x45 it would
use a temporary buffer of ((20*30) * (30*45)) bytes = 810k points.
It takes (Sh*Dh)*(Sh*Dh) operation to expand into the the temp buffer
and the same number to compress into the dest bugger for a total
of 2*(Sh*Dh)*(Sh*Dh) operations (1.6 million for this example).

If the expansion/compression is first done horizontally and *summed*
into a temporary buffer that is the width of the destination and
the height of the source it takes (Sw*Dw)*Sh operations. To 
expanded/compress the temp buffer into the destination takes
Tw*(Th*Dh) = Dw*(Sh*Dh) operations. The total is now
(Sw*Dw)*Sh + Dw*(Sh*Dh) (in this example (20*30)*30 + 30(30*45) =
18000 + 40500 = 58.5K operation or 3.7% of the Cartesian product).

Comment 9

[kill this account]

so the calculations is reduce from 2*n*n*n*n to 2*n*n*n 
(
from n**4 to n**3)

Comment 10

[kill this account]

Attachment: tar/gz; sample prog now scales in n**2 operations instead of n**4

Comment 11

[kill this account]

Attachment: image; scale 16->12 Japanese text

Comment 12

[kill this account]

one note:

Be aware that the blurring in the sample images may be slightly different than 
what would be seen if this was integrated into mozilla. 

Exactly which pixels get blurred depends on the scaling factor and the exact 
positioning of the image that was scaled. The glyphs in the sample image were 
copied off the display so there x,y position depends on the image placement and 
the inter-line spacing.

In an integrated version the glyphs would always be drawn off screen and then
copied. Thus the x,y placement would always be the same but perhaps different 
than the sample images.

Comment 13

[kill this account]

The reduction from n**4 to n**2 is an important speed consideration.

To scale a 16x16 glyph to a 12x12 glyph:

The original algorithm takes 73728 operations and a 36864 byte temporary buffer.
((16*12)*(16*12) + (16*12)*(16*12))

The new algorithm takes about 768 operations and a 208 byte temporary buffer.
(((16*12)*2) + ((12*16)*2))

Comment 14

[kill this account]

Attachment: tar/gz; now handles colormap on 24/32 bit frame buffer systems

Comment 15

[kill this account]

Attachment: sample image showing colors; 16->12 scaling

Comment 16

[kill this account]

Attachment: image showing scaling 16 px src to 1-32 px (

Comment 17

[kill this account]

Attachment: tar/gz; demo code to show range of sizes

Comment 18

[kill this account]

Attachment: tar/gz; demo prog with anti-jagging turned at size 20+

Comment 19

[kill this account]

Attachment: png image showing sizes 1-33

Comment 20

[kill this account]

Attachment: png image showing sizes 34-48

Comment 21

[kill this account]

Attachment: png image showing sizes 34-48

Comment 22

[kill this account]

Attachment: png image showing sizes 49-60

Comment 23

[kill this account]

Attachment: png showing sizes 61-64

Comment 24

[kill this account]

Attachment: image comparing without/with anti-jagging

Comment 25

[kill this account]

Attachment: image: www.china.com/zh_cn standard fonts

Comment 26

[kill this account]

Attachment: image: www.china.com/zh_cn with anti-aliased scaled bitmap fonts

Comment 27

[kill this account]

Attachment: for reference: the complete patch file (less unix.js)

Comment 28

[kill this account]

Attachment: for reference: the patch file for unix.js

Comment 29

[kill this account]

Since the checkin is ~4500 lines I have broken it down into
sub bugs to make the checking process more managable.

bug 107018 covers the alpha blending (transparency) parts (~1200 lines)
bug 107019 covers the code for a anti-aliased glyph object (~260 lines)
bug 107020 the changes to nsFontMetricsGTK.cpp and nsRenderingContext.cpp (~1200
lines)
bug 107025 covers the layer that contains regular and anti-aliased fonts object
(~1697 lines)

Comment 30

[kill this account]

Attachment: for reference: the complete patch file (less unix.js) updated with MathML fixes

Comment 31

[kill this account]

Since this bug has no relationship in any way with bug 34242 I am removing 
that.

Comment 32

[kill this account]

Attachment: for reference: the unix.js file with minor tweaks suggested by momoi

Comment 33

[kill this account]

Attachment: for reference: the complete patch file (less unix.js) updated

Comment 34

[kill this account]

Attachment: for reference: the patch file for unix.js with minor tweaks suggested by momoi

Comment 35

[Roland Mainz]

Just a dumb question about the patch (I did not look at it in detail yet): Are
you creating one pixmap per glyph and keep it all the time until the aa-font
isn't used anymore ?

My fear is that we may run out of XIDs ...

Comment 36

[kill this account]

This code creates one pixmap per font/size. When a scaled glyph is first 
needed the non-scaled glyph is drawn into the pixmap, the pixels are copied to 
the client with XGetImage, and scaled (on the client). The scaled glyph is 
hashed/cached on the client.

Comment 37

[Roland Mainz]

OK... but it would be better to store the pixmaps on the server if possible.
What about creating _one_ pixmap large enougth to hold all glyphs - sort of
"table" containing all glyphs. This would save tons of image transfers over the
wire...

Comment 38

[kill this account]

we need the scaled glyphs on the client since the alpha blending occurs on
the client.

Comment 39

[Roland Mainz]

> we need the scaled glyphs on the client since the alpha blending occurs on
> the client.

Yes... I see... this code is hopefully not used to render all glyphs (or better:
the most common type of glyphs (like ISO-Latin-1)) - otherwise our remote X11
performance would degrate significantly.

----

We should disable this code automagically if the Xserver's screen resolution is
high enougth (>=150DPI !?) ... 

Comment 40

[kill this account]

This seems unrelated to screen resolution. This seems related to having or
not having an appropiately size font.

Comment 41

[kill this account]

For western fonts there tend to be a good spread of font sizes so this code
should rarely be used in that case.

For CJK fonts there tend to be a very limited spread of font sizes so this
code should be used more often.

Comment 42

[kill this account]

Attachment: for reference: patch (less unix.js) with fixes from bug 107020

Comment 43

[Frank Tang]

Here is the performance and footprint impact by using 10/31 patches
running the cowtools on 533MHz machine with 128M RAM on RedHat JA 7.2. Opt build

without the patch:
the 'top' command show me after cowtools show me the report.
28100 ftang     12   0 39120  38M 13008 S     0.9 30.8   5:07 mozilla-bin
'cowtools' show me :
Test id: 3BE22BD0DA
Avg. Median : 1641 msec         Minimum     : 354 msec
Average     : 1750 msec         Maximum     : 8405 msec

with the patch:
the 'top' command show me after cowtools show me the report.
28163 ftang     11   0 35584  34M 13008 R    37.2 28.0   5:07 mozilla-bin
'cowtools' show me :
Test id: 3BE22EC66E
Avg. Median : 1600 msec         Minimum     : 353 msec
Average     : 1772 msec         Maximum     : 8605 msec

I will post the details report in the attachement.

Comment 44

[Frank Tang]

Attachment: cowtools details result without patch

Comment 45

[Frank Tang]

Attachment: cowtools details report without patch

Comment 46

[Frank Tang]

Attachment: cowtools details result with patch

Comment 47

[Roland Mainz]

ftang:
Are this results with local X11 (server+client on the same machine) or remote
X11 (different machines).

Remote X11 performace is the part I am worrying about (many many universities
still have 10baseT-based Xterminals (no joke)) ...

Comment 48

[Frank Tang]

Attachment: cowtools details report with patch

Comment 49

[Frank Tang]

>ftang:
>Are this results with local X11 (server+client on the same machine) or remote
>X11 (different machines).
Local X11.

>Remote X11 performace is the part I am worrying about (many many universities
>still have 10baseT-based Xterminals (no joke)) ...
I know. I don't see any peformance masurement targeting on that matrix. We are 
willing to measure that if it is in the performance measurement matric in 
general. 

Do you have the percentage of how many Linux users are on same machine and how 
many are running remotely ?

Comment 50

[Roland Mainz]

> Do you have the percentage of how many Linux users are on same machine and how 
> many are running remotely ?

That cannot be counted in a per-machine/user manner. The remote X11 stuff is
used by large installations (big companies, universites, internet café's
etc.)... usually those guys who have much $$$$$$ and are spending their money
into thin-client stuff...

Comment 51

[kill this account]

> Remote X11 performace is the part I am worrying about (many many universities
> still have 10baseT-based Xterminals (no joke)) ...

This code is used when there are no "nearly correct size" fonts. This is
generally much more common in CJK (ideograph) fonts since there are so
many characters per font that making lots of sizes is very time/disk
consumming. Since western fonts are small there typically are lots of sizes 
available.

So, how many of these systems are using CJK fonts (or have a poor selection
of font sizes available)?

Since the bandwidth is similar to images:

  Has the image (pictures) performance on these system been a problem?

Comment 52

[kill this account]

Attachment: for reference: the complete patch file (less unix.js), de-rotted and with current changes

de-rotted the bits

Comment 53

[kill this account]

the code for this is all in

Comment 54

[Yuying Long]

Attachment: Screen shot for yahoo-japan page, notice some of the font are not anti-alias

In most CJK pages, anti-alias and not anti-alias bitmap fonts are mixed
together.

I wonder if there is a reason for that?

Comment 55

[kill this account]

> In most CJK pages, anti-alias and not anti-alias bitmap fonts are mixed
> together. I wonder if there is a reason for that?

yes, the code tries to use non-scaled where possible for memory and
performance reasons. This is controlled by the unix.js file:

  pref("font.scale.aa_bitmap.undersize", 80);
  pref("font.scale.aa_bitmap.oversize", 120);

These setting say: if there is a non-scaled font within 80% to 120%
of the desired size use it.

Thi can be set on a per language basis with a pref such as this:

  pref("font.scale.aa_bitmap.undersize.ja", 90);
  pref("font.scale.aa_bitmap.oversize.ja", 110);

Comment 56

[Adam D. Moss]

Cheers for that Brian... I've just discovered to evil joy to be had from:
user_pref("font.scale.aa_bitmap.oversize", 100);
user_pref("font.scale.aa_bitmap.undersize", 100);

Nice.  :)

Comment 57

[kill this account]

care to attach a screen shot?

Comment 58

[Adam D. Moss]

Attachment: test0 screenshot with 100/100 aa-text thresholds

Comment 59

[kill this account]

nice looking.

how is the performance?

Comment 60

[Adam D. Moss]

Performance is subjectively fine, especially for the
number of X-server round trips I expect this is causing!

Might suck on a remote X session, but eh.  I'll try
that tomorrow if I get a chance.

Comment 61

[kill this account]

Adam: one small favor: people are likely to be confused by the
term "aa-text" and assume that it means TrueType (outline scaled)
text that is anti-aliased. As such could we call it aasb (anti
aliased scaled bitmap) since it really is "sort of" a hack?

Comment 62

[Adam D. Moss]

I can confirm that aasb performance is kinda nasty
on a remote X-display.  =)  So it's not a good idea
on such a setup to enable the 100/100 threshold, but
it's fine with the defaults which only apply aasb to
the occasional font.

Comment 63

[Frank Tang]

>In most CJK pages, anti-alias and not anti-alias bitmap fonts are mixed
>together.
Is that a issue ?

The goal of the project is to make CJK display text with the desired size and
reasonable quality and performance. 

------- Additional Comment #47 From Roland Mainz 2001-11-07 12:19 -------
>Remote X11 performace is the part I am worrying about (many many universities
>still have 10baseT-based Xterminals (no joke)) ...
------- Additional Comment #62 From Adam D. Moss 2001-11-20 02:24 -------
>I can confirm that aasb performance is kinda nasty
>on a remote X-display.  =)

How is ghostscript performance over remote X11 ? Is it also kinda nasty too ?
How many remote X11 users still using ghostscript? Will you use ghostscript over
remote X11? If ghostscript is slow over remote X11 as what I assume, will you
stop using ghostscript on local X11 because it perform slowly over remote X11?

The other thing I want to mention is the performance issue with remote X11 could
be solve easily for those university, as long as the admin install enough number
of font size for bitmap font on the X display (or the font server), the anti
aliasing font won't be used at all. The remote X11 performnace only suck when
the X display server do not have enough font size in the bitmap font. 

------- Additional Comment #62 From Adam D. Moss 2001-11-20 02:24 -------
>I can confirm that aasb performance is kinda nasty
>on a remote X-display.  =)
How nasty with and without aasb over remote x display ?
What is the number ? how many % does aasb impact ?

Comment 64

[Adam D. Moss]

>>I can confirm that aasb performance is kinda nasty
>>on a remote X-display.  =)
>
>How is ghostscript performance over remote X11 ? Is it also kinda nasty too ?

I have never used ghostview remotely and am not in a position to do so.

>How many remote X11 users still using ghostscript?

I don't know!

>Will you use ghostscript over remote X11?

What do you mean?  I suspect you're picking on ghostscript
because the later versions support antialiasing so you assume
that it would suffer comparably to Mozilla, but the mechanics
AFAIK are quite different (significantly more client-side,
or I'm shocked).

>If ghostscript is slow over remote X11 as what I
>assume, will you stop using ghostscript on local X11 because it perform
>slowly over remote X11?

By what logic would I?  Sorry, but I can't see either the sense or
relevance in this question.

>>I can confirm that aasb performance is kinda nasty
>>on a remote X-display.  =)
>
>How nasty with and without aasb over remote x display ?

Over a 10Mbps local LAN it's totally fine on pages where only
a few sparse strings require aasb.  On the rare pages which are
very aasb-heavy (try bonsai checkin summaries with the aasb
kick-in threshold bumped to simluate the effect of a poor
font selection) it becomes unpleasant to scroll or paint.  A
page-down takes three or four seconds to respond.  Assuredly
unpleasant but not unusable.

>What is the number ? how many % does aasb impact ?

That would definitely depend on the X client<->server network
latency and, of course, the amount of aasb use on the page in
question.  If you can construct a benchmark testcase with
non-standard font sizes then I'll be happy to supply numbers with
the non-aasb-tolerance prefs set to 100/100 to eliminate most
of the font list variance between systems so we can all compare
meaningful figures.

Comment 65

[Roland Mainz]

I'd like to nominate this bug to get the evil "highrisk" keyword ...

Comment 66

[kill this account]

If there is there a way to tell that the the server is remote
perhaps we should have aasb disable by default for remote displays.

Adam: any ideas on how to test for a remote X server?

Comment 67

[Adam D. Moss]

Perhaps you could test gdk_get_use_xshm() -- you may get false-negatives as to
whether we're running locally, but at least you shouldn't get false-positives. 
I've never used it for the purposes of detecting display localness, but from a
quick browse of the gdk/gtk source it looks like it should be pretty close.

Comment 68

[Adam D. Moss]

Alternatively of course you can always parse $DISPLAY and check that the
hostname before the colon resolves to the same as 'localhost', or is null.  But
the form of this string can vary depending on the transports available to client
and server, and that all seems a bit icky and hacky.  Would probably turn out no
more reliable than a call to gdk_get_use_xshm().

Comment 69

[kill this account]

Any idea what percentage of X servers would correctly work with 
gdk_get_use_xshm?

Comment 70

[Roland Mainz]

bstell wrote:
> Any idea what percentage of X servers would correctly work with 
> gdk_get_use_xshm?

0% ? I would never trust a GDK/GTK+ function in that case...
Please examine the $DISPLAY variable - that's guranteed to be safe ...

Comment 71

[Roland Mainz]

Adam D. Moss:
> Perhaps you could test gdk_get_use_xshm() -- you may get false-negatives as to
> whether we're running locally, but at least you shouldn't get false-positives.

GDK still makes Xnest crash due a bug in their MIT-SHM handling code. I'd like
to avoid doing the same mistake ...

Comment 72

[kill this account]

I don't think anyone suggested we use shm.

$DISPLAY is possible but because there are many valid ways to say the same 
thing it might be messy (but doable).

Comment 73

[Roland Mainz]

I filed bug 112506 to get a prefs item to turn AASB "on"/"off" in the GUI...

Comment 74

[Adam D. Moss]

> GDK still makes Xnest crash due a bug in their MIT-SHM handling code.

Whatever you say -- but academic.  gdk_get_use_xshm() gets
whether GDK thinks it's safe to use xshm (and hence is definitely
not remote), it doesn't *use* xshm, it just returns a GDK-internal
boolean.  Frankly if GDK's shm-images causes a particular server
to crash then gtkfe-using mozilla will already have crashed or
shortly will, plain and simple, because it uses them heavily
elsewhere.

Comment 75

[Yuying Long]

The aasb fonts on 12-13 trunk build.

Mark it as verified.