68779 - Xprt PS DDX/Xprint generates _huge_ PostScript output files with low quality font glyphs...

Status: VERIFIED FIXED

(Core Graveyard :: Printing: Xprint, defect)

Description

[Roland Mainz]

Xprint implementation (last tested with mozilla-2001-02-12-08-Mtrunk but applies
to all previous versions) generate huge output files.

A simple attempt to print a plaintext file (file:///var/log/syslog) with 973893
bytes via Mozilla's Xprint module and Xprt's PostScript DDX results (Solaris
7/108376-18) in a 94MB (!!!!) PostScript file. 
It looks like the file is "rendered" as a whole big image instead of rendered as
text (e.g. via XDrawString()) - HP LaserJet IIISi rejects this file... ;-( ) ?

Last but not least: Rendering fonts as "image" looks awful... ;-(

The behaviour I would expect is that Mozilla renders this plaintext file more or
less completely via XDrawString() and this text strings would reappear in
PostScript source (Xprt's PS DDX normally uses printer fonts and puts text given
via XDrawString() 1:1 into the PostScript output).

Two possible reasons for this problem:
a) Xprint code in Mozilla renders everything as image
b) Xprt is not properly configured and does not know how to map screen fonts -->
printer fonts and therefore renders the screen fonts as images (which results in
this awful-looking monster-scaled pixel fonts).

Adding waquar to CC: as this is more or less related to bug 57820...
Marking "critical" because no state-of-the-art PostScript printer can handle
94MB (OK... we have one - but guess how usefull it is to create >=94MB print
jobs).

Comment 1

[Roland Mainz]

It is definately problem [b], found a workaround (details on demand; now getting
1.1MB PostScript from 1MB plaintext file... :-) but no solution for all printer
model-configs. 

Need help from Xprint gurus from HP/Sun to discuss some things. Helllp !!

Comment 2

[Roland Mainz]

This may be solved via setting "*xp-listfonts-modes:
xp-list-internal-printer-fonts" (to force build-in printer fonts) before
XpSetContent(), but...
... ding... this does not work... ;-(

Looks like a bug in Xprt (both Solaris 7+108376-18 and Xprt build from X11R6.5.1
sources) - setting "xp-listfonts-modes" (tested all combinations(=4:) of
"xp-list-internal-printer-fonts"/"xp-list-glyph-fonts") seems to be completely
ignored.

Workaround is to set fontpath (Xprt option -fp) to dir of printer model-configs
fonts/ dir and add alias for "fixed" and some cursor fonts - but this is not
very usefull because one Xprt cannot be shared between different printer
models... ;-(

Comment 3

[Roland Mainz]

CC Momoi who is tracking international issues of UNIX printing.
CC jdunn because he may be interested, too (based on comments in bug 49947)
CC tajima because (AFAIK) he's interested in Xprint for Mozilla, too

----

(Sorry, I am little bit angry about this issue, but...)

Background:
Number of fonts returned by XListFonts() after XpSetContext() cannot be
controlled via "xp-listfonts-modes" as defined by specs (unfortunaltely they
have many "may... may... may..."...) - that's causing all the trouble with job
sizes up to 700MB (and beyond) for some web pages (see ~94MB example listed here
in the bug) and making a workaround nearly _impossible_.

Unfortunately the specs are a little bit unclear how to handle printer model
fonts - and how an application can force XLoadFont() to return internal printer
fonts only.

Based on this bug and some discussion with people from the early Xprint days... 
we can dump the whole Xprint (including Mozilla's Xprint module) to /dev/null
due these bugs in X11R6.5.1 code because Xprint (not only in Mozilla) becomes
unuseable when generating such large job sizes for (even) small pages - except
someone will fix these bugs.

I wrote a small "workaround" on the application side (which does (from within
the application via XSetFontPath()) a "xset fp=
$XPCONFIGDIR/C/print/models/$CURRENT_XP_MODEL_NAME/fonts/" to change font path
to contain internal printer fonts only if "xp-listfonts-modes" page/document
attribute is set to "xp-list-internal-printer-fonts" directly before
XpSetContext()) - but this issue needs to be fixed on the Xlib/Xprt side to be
spec conformant.

Comment 4

[Masaki Katakai]

Yes, I understand this is problem in Xprt side, not Mozilla.

I remember when I tried to use printer's fonts, not bitmap from X server
on Solaris, Xprt could work for only English fonts. I could not get any
Japanese printer's fonts in outputs when I used Japanese characters.
I'll ask Xprt folks in Sun.

Comment 5

[Roland Mainz]

(spend neary the last 9 hours to dig-out that this is a _bug_ in X11R6.5.1...
grrrr...)

To [e-mail]: Is it possible to discuss this short and quickly via
PM, please ?

Comment 6

[Roland Mainz]

Attachment: xprintutil.c - small collection of Xprint util libs

Comment 7

[Roland Mainz]

Please take alook at the attached "xprintutil.c".
The function XpuSetContext() is a "wrapper" for XpSetContext() and tries to
_hack_ Xprt to return only printer-internal fonts if you've set
XpuSetOneAttribute(pdpy, pcontext, XPDocAttr, "*xp-listfonts-modes",
"xp-list-internal-printer-fonts", XPAttrMerge);

Does that solve your problem ?

Comment 8

[Roland Mainz]

To quote XpuSetContext() comment here:
-- snip --
XpuSetContext() - XpSetContext() wrapper which tries to force 
fontpath to contain only fonts specified by 
"xp-listfonts-modes" page/document attribute.
WARNING: This is only a "workaround" for X11>= R6.5.1 - and
this is _evil_ as it changes the font path "globally" instead
of changing the font path in a per-"print context" manner 
(this can only be handled from Xserver (=Xprt) side).

The fix in Xserver (=Xprt) side would be to force XListFonts() 
to return only fonts which match the requirements set by 
"xp-listfonts-modes" page/document attribute (unfortunaltely
the specs say XListFonts() _may_ do this and the _real_ 
_authority_ is XLoadFonts() - but is is unuseable from 
real-world applications (like such complex beasts like 
Mozilla)...

Solution: Change the "may" for XListFonts() in specs to "must" 
and _IMPLEMENT_ this !!
-- snip --

Comment 9

[Roland Mainz]

> This is only a "workaround" for X11>= R6.5.1 - and
Sorry... should be: '"workaround" for X11 <= R6.5.1 - and'

Comment 10

[Roland Mainz]

To Masaki Katakai:
(Question... again:)
Does 
-- snip --
XpuSetOneAttribute(pdpy, pcontext, XPDocAttr, "*xp-listfonts-modes",
"xp-list-internal-printer-fonts", XPAttrMerge);
XpuSetContext(pdpy,pcontext);
-- snip --
instead of 
-- snip --
XpSetContext(pdpy, pcontext);
-- snip --
solve your problem (forcing build-in fonts solves the problem listed here - but
I do not have a PS printer which has build-in japanese fonts) ?

Comment 11

[Masaki Katakai]

I understand xprintutil.c is wrapper, so do we need changes
in Xprt codes in Mozilla? Please tell me the instruction
detail for build?

Btw, have you succeeded to use printer's fonts (for English) by using this?

XpuSetOneAttribute(pdpy, pcontext, XPDocAttr, "*xp-listfonts-modes",
"xp-list-internal-printer-fonts", XPAttrMerge);

Yes, actually I have also succeeded to use English builtin fonts,
but the same attribute did not work Japanese. I've checked the
font and printer definition files in Xprt side and I found
there is no definition for Japanese. Xprt side should provide them.

Comment 12

[Roland Mainz]

> I understand xprintutil.c is wrapper, 

Yes, it is - but XpuSetContext() is only a _hack_ to force Xprt to do what I
would expect from it: Return _only_ printer-internal fonts (e.g. XListFonts() -
specs say it _may_ do this... (and XLoadFonts() is the real authority) but IMHO
it _must_ do this even in XListFonts()&co. to fit the needs of complex apps.
like Mozilla) when "*xp-listfonts-modes: xp-list-internal-printer-fonts" is set.

> so do we need changes in Xprt codes in
                           ^^^^

I assume you mean "Xprint"... :-)

> Mozilla? 

> Please tell me the instruction detail for build?

#if 0
XpSetContext(pdpy,pcontext);
#else
/* WARNING: XpuSetContext() is a illegal _hack_ until we find a 
 * real solution for this problem...
 * (this may be a bug in Xprt or misinterpretation of the Xprint 
 * server specs in Xprt sample server implementation).
 */
XpuSetOneAttribute(pdpy, pcontext, XPDocAttr, "*xp-listfonts-modes",
"xp-list-internal-printer-fonts", XPAttrMerge);
XpuSetContext(pdpy,pcontext);
#endif

xprint.c refereces only xprint.h which contains only the prototypes and one or
two debugging macros (should I attach it, too ?)...

> Yes, actually I have also succeeded to use English builtin fonts,
> but the same attribute did not work Japanese. I've checked the
> font and printer definition files in Xprt side and I found
> there is no definition for Japanese.

Uhm... does the printer have build-in japanse fonts ?

> Xprt side should provide them.
Try to ask HP staff. AFAIK HP has _tons_ of Xprt configs for their own printers.
... and it would be nice to release these configs to the public that other
people can use them...

Comment 13

[Roland Mainz]

Fixing subject to include "Xprt PS DDX".
Set target milestone to "Mozilla 1.0" based on [e-mail]'s rants&crys
in bug 49947.
 
To kataki&&tajima: 
1. Is it possible that Xprt's PS DDX get's fixed until Mozilla release 1.0 ?
2. Please join bug 49947 (CC:). There are some notes about changing your code a
little bit to be able turn --with-xprint on per default for all Unix-Platforms
which have libXp.so

----

I added one of my rare "votes" to this bug because I'd like to see this issue
fixed (in Xprt)...

----

About my xprintutil.c:
Assuming the code in Mozilla.org's CVS is the most "upto-date" version of Xprint
in Mozilla...
... are you interested that I hack the code to use my wrappers in xprintutil.c
to make Xprint support in Mozilla little bit more useable (like supporting some
page layout attributes from unix print dialog, using $XPSERVERLIST instead of
$XPDISPLAY, allowing printername@host:displaynum (like [e-mail]:4) and
some error checking to avoid some silly crashes...) ?

----

To bug owner (Katakai): Do you want to accept this bug ?

Comment 14

[Masaki Katakai]

I'll close this bug as WONTFIX.

I understand this is not a bug in Mozilla and we should not define
xp-list-internal-printer-fonts attribute in Mozilla because this is
an attribute specific to printers. By the setting the attribute in
configuration file of Xprt and selecting the target printer by Mozilla,
Xprt will use printer's fonts instead of bitmap. Of course, we should
describe in Xprt notes of Mozilla how to use printer's built-in fonts
to reduce the size of outputs.

Comment 15

[Roland Mainz]

Yes and no.
Yes... it is a bug/issue in current Xprt implemenation which blows-up the
PostScript/PCL job size to unuseable size - and dramatically degrading quality.
No... Mozilla should _set_ both xp-list-internal-printer-fonts and
xp-list-glypth-fonts to _force_ the printer driver to use both internal printer
fonts and glypth fonts - see below:

(I already posted the following to another Sun engineer:)

-- snip --
I am going to write a small proposal how this may be fixed in a
spec/standard-conformanant way - but I need some days...
...short: Xprt behaviour for "xp-listfonts-modes" should be improved:
1. Application sets "*xp-listfonts-modes:
xp-list-internal-printer-fonts", then XpSetContext(). Xprt
(XListFonts()&co.) should only list internal printer fonts and font path
for this print context (e.g. font paths may be "modified" by
XpSetContext() in a per print-context manner) should only list the
source of these fonts (e.g.
$XPCONFIGDIR/C/print/models/$XP_PRINTER_MODEL/fonts/).
2. Application sets "*xp-listfonts-modes: xp-list-glyph-fonts", then
XpSetContext(). Xprt (XListFonts()&co.) should only list glyph fonts and
font path for this print context should only list the source of these
fonts.
3. Appliication sets both "xp-list-internal-printer-fonts" and
"xp-list-glyph-fonts". Both types of fonts should be listed - but font
path for this print context should be arranged that the match order of
fonts equals the appearance order of xp-list-* attribute values.
"*xp-listfonts-modes: xp-list-internal-printer-fonts
xp-list-glyph-fonts" means that internal printer fonts should occur
before glyph fonts in both font path and XListFonts()&co. output;
"*xp-listfonts-modes: xp-list-glyph-fonts
xp-list-internal-printer-fonts" asks for the reverse order.
4. "*xp-listfonts-modes: xp-list-internal-printer-fonts
xp-list-glyph-fonts" should be the default value for the
*xp-listfonts-modes attribute unless otherwise set by model config...
5. There should be a simple way how to determinate if a font is an
internal printer font or not.
-- snip --

Additionally _quality_ may be improved by rearraging the default font paths for
Sun's Xprt: Internal printer fonts first (models/$XPMODEL/fonts), then "normal"
scaleable vector fonts - to avoid that 100DPI bitmap fonts will be scaled-up to
300/600/1200DPI instead of using available internal printer and scaleable vector
fonts...

Without fixing the font path after XpSetContext() in one of the ways described
above it will be hopeless (at least it looks like that...)... ;-(

BTW: I fixed the subject to include "... with low quality font glyphs..."
because it's the same issue - Mozilla/Xprt select the _wrong_ (set of) fonts...

Comment 16

[Roland Mainz]

Quality issues:
I got much better results after removing bitmap font dirs from font path and
only use scaleable fonts. One good fix for this problem would be to allow only
"scaleable" fonts...

Comments ?

Comment 17

[Roland Mainz]

I still can't get the current implementation of Xprt (Solaris 7/108376-21) to
use printer build-in fonts for printing a simple ASCII file.
2920 bytes of ASCII text result in 288383 bytes of PostScript code, e.g. 98.7613
times more bytes as in the original file.
Same input file but Xprt _forced_ to use build-in printer fonts by setting font
path to $XPCONFIGDIR/C/print/models/SPSPARC2/fonts,/home/gisburn/Xp_miscfonts 
and adding an alias for
"fixed"(="-adobe-courier-medium-r-normal--199-120-1200-1200-p-1190-iso8859-1")
results in 7374 bytes of postscript code - I'll attach this file later as an
example how this _should_ look like (uhm... the file size - not the buggy text
layout... =:-)

I used the following config (which tries to _avoid_ to use any 75dpi/100dpi
bitmap fonts by using only font dirs with _scaleable_ fonts - the current
default font path used by Xprt is (sorry) _insane_):
-- snip --
% ls -l Xp_miscfonts/
total 8
lrwxrwxrwx   1 cursor.pcf.Z -> /usr/openwin/lib/X11/fonts/misc/cursor.pcf.Z
-rw-r-----   1 fonts.alias
-rw-r-----   1 fonts.dir
lrwxrwxrwx   1 olcursor.pcf.Z -> /usr/openwin/lib/X11/fonts/misc/olcursor.pcf.Z
% cat Xp_miscfonts/fonts.alias 
! alias for "fixed" font
! original from /usr/openwin/lib/X11/fonts/misc/fonts.alias looks like this:
! fixed   "-misc-fixed-medium-r-semicondensed--13-120-75-75-c-60-iso8859-1"
fixed -*-r-*--*-*-*-*-*-*-iso8859-1
! EOF.
% MYFP=/home/gisburn/Xp_miscfonts; for i in
$(OWFP=/usr/openwin/lib/locale/$LANG/OWfontpath; [ -f $OWFP ] && cat $OWFP; echo
/usr/openwin/lib/X11/fonts/*) ; do if [ -f $i/fonts.scale ] ; then
MYFP=${MYFP},${i} ; fi ; done ; export
XPCONFIGDIR=/home/gisburn/cwork/Xprint/Xprt_config/XpConfig; Xprt -fp $MYFP
-audit 4 :1
-- snip --

The resulting file has more or less good quality because Xprt doesn't offer any
scaled-up 75DPI fonts - but it still prefers to use the scaled fonts over the
printer build-in fonts (unless I remove all except build-in printer fonts - but
that only helps for one model-config - multiple different model configs in one
server may cause the problem that one application using one model uses fonts
from another model... ahhhgllll...).
And there's the problem that the "fixed" font (which is usually an alias)  is
stored as bitmap imgages in the resulting PostScript file.

Conclusions:
1. Xprt _must_ be fixed to use printer build-in fonts when available - see my
comments from 2001-03-07 00:41 in this bug here about *xp-listfonts-modes
2. model fonts should always include an alias for "fixed" if they contain a
matching font - and this "fixed" alias font should replace the alias for "fixed"
currently in used if a print context is active (e.g. after XpSetContext) - but
is that technically possible (AFAIK yes, font path can be modified at runtime)
3. Mozilla should only use fonts (XQueryFonts/XLoadFonts/XListFonts) after
XpSetContext() has been called (e.g. a print context is active for that
particular display).

Misc:
a. [e-mail]: Can Alan.C*@sun.com comment this issue, please ? At
least issues [1] and [2] above should be fixed at Xprt (config) side...
b. Some some reason a simple hello_world_xprint program can use build-in printer
fonts without any _pains_ (like described here). This may be either [3] (e.g.
Mozilla queries fonts before XpSetContext() is called) or Mozilla has some very
restrictive use of fonts (maybe it likes to use exactly the same fonts as used
on the framebuffer display !?). This is a reason for me to reopen this bug and
investigate this issue...

Comment 18

[Roland Mainz]

Attachment: 7374 bytes of PostScript (using build-in printer fonts) from 2920 bytes of ASCII text file

Comment 19

[Masaki Katakai]

Please provide a small example to reproduce 1 and 2 issues.
I'll file bugs against Xprt side for Sun bugtraq and will
wait evaluation.

If the issue in Mozilla is only 3, we should change the
summary to that to understand what issue is in Mozilla
(not Xprt side).

Comment 20

[Roland Mainz]

Example for [2001-03-14 16:08/2] (I added date/time of comment to avoid any
confusion) is easy, I'll attach a simple hello_world.xprint.c and the resulting
PostScript file as tar.gz
You'll see that using "fixed" font always uses a bitmap font instead of printer
buildin fonts (which results in the problem that bitmap fonts are bitmap images
in PostScript code - blowing-up the size a lot) - even if the printer model
contains it's own alias for "fixed"... and this leads to the conclusion that
Xprt does either...
a) ...does not include any printer model fonts (e.g. ignores them)
b) ...does not change the alias for "fixed"
c) ...appends alias for "fixed" instead of putting it in font of the font list
d/e/f/g/h/i/j...) [insert tons of other possibilities here what _may_ happen...
;-( Without adding debugging code to Xprt it's just guessing which font get's
used]

Best idea (for Xprt) would be to change fontpath at XpSetContext() time to
restrict it to only those font sources really used (e.g. font path of _current_
printer model and remaining font path (currently it lists all font dirs from all
configured print models)) and rehash font list internally to be sure that old
stuff get's flushed (and restore font path&font list to it's "old" glory when
print context has been removed from display).

It's very difficult for me to debug these issues because I do not know a way to
find the _source_ (font dir (physical location on disk)) of a font without
adding debug code to Xprt (as I pointed out somewhere else my resources are
currently _very_ limited, I can't work with X11R6.5.1 builds and Mozilla at the
same time... ;-( ).

Example for [2001-03-14 16:08/1] is a lot more difficult - but in fact it's
covered by changing the (default) font path.
Examples:
a. Setting "*xp-listfonts-modes: xp-list-internal-printer-fonts" can be emulated
by Xprt -fp
/home/gisburn/cwork/Xprint/Xprt_config/XpConfig/C/print/models/SPSPARC2-TEST001/fonts,/home/gisburn/Xp_miscfonts
- assuming the printer has only the "SPSPARC2-TEST001" model configured. In this
case only printer build-in fonts for the SPSPARC2-001 model are used
(SPSPARC-TEST001 is like SPSPARC-001 except that fonts.alias contains an alias
for "fixed
(="-adobe-courier-medium-r-normal--199-120-1200-1200-p-1190-iso8859-1"))
xlsfonts how results only build-in printer fonts

b. "*xp-listfonts-modes: xp-list-glyph-fonts": This cannot be archived by
hacking the config - you cannot remove all build-in printer fonts except
changing the font path for the current display from within the application.

Currently the "xp-listfonts-modes" attribute seems to do nothing - changing it
seems to have no effect to the number of fonts available...

Comment 21

[Roland Mainz]

Attachment: hello_world_xprint.c hacked to use "fixed" font alias - and the resulting PostScript file showing that "fixed" font is always stored as bitmap images in PostScript code

Comment 22

[Roland Mainz]

RFE
(http://puck.informatik.med.uni-giessen.de/people/gisburn/work/rfe_solaris.html#date_20010315):
Add an option to Xprt do store debugging information into resulting PostScript
or in a seperate log/file (adding plaintext comments in PCL would be a little
bit difficult) - including used Xp-attributes and used fonts (name, model (if
font is from a printer model) and physical source (e.g. which fontpath element).

Comment 23

[Masaki Katakai]

Actually, without setting -fp option, even when I set fonts.alias to use
printer's font for "fixed", "fixed" always uses bitmap.

This seems to happen when we use "fixed" font because when I tried to use
another name "katakai" for courier, it can be replaced with printer's font.

fixed "-adobe-courier-medium-r-normal--199-120-1200-1200-p-1190-iso8859-1"
katakai "-adobe-courier-medium-r-normal--199-120-1200-1200-p-1190-iso8859-1"

I've filed new bug that "can not use "fixed" printer font" against Xprt
of Sun's bugtraq system, internal bugid is 4425922.

For debugging option, an RFE has been also files in Sun's bugtraq, bugid is
4425924.

Comment 24

[Roland Mainz]

To Masaki Katakai: Is it possible to attach bugid 4425922 and 4425924 (and if
you're going to file more RFs for issues below - please attch them, too), please
? I don't have access to Sun's bug database anymore... ;-(

More Xprt RFEs:
- Xprint config should have a "generic postscript" model (maybe a variant of
SPSPARC2 ?)
- Xprint config should have a "unicode postscript" model (problem: Where to get
the matching font glyph info from ?)
- Xprt's default font path and/or font lookup/match algorithm should be
arranged/modified to use _scaleable_ fonts first. Current default font path
matches low-resolution fonts very often (better: too often... ;-( ) which
results in a very poor quality. Background: Xserver'S default font path and and
font lookup is designed for speed - use fast-to-load bitmap fonts, then
scaleable font. But Xprt need high-resolution fonts which usually are not
available as "bitmap fonts" (600DPI bitmaps fonts... ? =:-) And this issue leads
to the next RFE:
- Solaris should ship with scaleable fonts for all locales (at least my Solaris
7 system has no scaleable fonts for japanses)

Comment 25

[Roland Mainz]

*** Bug 68317 has been marked as a duplicate of this bug. ***

Comment 26

[Roland Mainz]

Just one note: X11R6.6 has been released, see ftp://ftp.x.org/pub/R6.6/tars/

Comment 27

[Roland Mainz]

Some measurements:

I printed a plain text file
(http://puck.informatik.med.uni-giessen.de/people/gisburn/unix/x11/x11r6.5.1_to_x11r6.6.gudiff.txt)
with Mozilla5 and got the following results
(job_size/text_file_size=ratio):

no. | job size  | text file size | ratio
----+-----------+----------------+--------
[a] | 298265006 | 2951613        | 101.05
[b] |  56882096 | 2951613        |  19.27
[c] |   4378562 | 2951613        |   1.48
----+-----------+----------------+--------
[d] |  10760728 | 2951613        |   3.64


[a] is Xprt from Solaris 7 patch 108376-21 (X11R6.5.1 !?)
[b] is Xprt from X11R6.6
[c] is Xprt from Solaris 7 patch 108376-21 where I forced it to use
printer-buildin fonts
[d] is the native PostScript print module in Mozilla5
(mozilla/gfx/src/ps/)


[c] looks for me like the optimum - it is the fastest on all PS printers
(+sdtimage), has the smallest job size (even beats the native PS module
(hand-written and optimized... :-) - but [c] beats [d] by factor 2.45), tools
like "ps2txt" are able to extract the plain text from the file (+sdtimage page
overview is able to give an "overview" over the text) and we would be able to
get rid of the myth that "Xprint" always uses GFX fonts...

Comment 28

[Masaki Katakai]

Assign to Roland

Comment 29

[Roland Mainz]

Accepting...

Comment 30

[Roland Mainz]

This issue is now under control. 
Xprt in >= X11R6.6 has some tricks to avoid that GFX fonts blow-up the print job
size - and Mozilla's Xprint module now operates correctly at printers
resolution, chooses proper fonts for that DPI and scales images properly to
destination size (using Xprint API XpSetImageResolution()).

Marking as "FIXED".

Comment 31

[Masaki Katakai]

Mark this as VERIFIED.

Currently printer's built-in font are used by
Xprt setting. Also bug 88554 has been fixed.