96875 - string in titlebar is truncated when it contains characters with accents

Status: VERIFIED FIXED

(Core :: Internationalization, defect, P3)

Description

[Gunnar Wolf]

I have found different web pages containing accented characters in the title,
and the title always appears truncated to the immediately previous character. In
the web page I give as an example, the title should read "UNIDAD DE PROGRAMACIÓN
Y EVALUACIÓN", and it reads only "UNIDAD DE PROGRAMACI".

I am running Mozilla 0.9.3 on Linux Debian Unstable (Sid), on an Alpha system. I
do not have an alternate window manager installed (I use WindowMaker 0.65.1),
but for other windows with accented characters, the accented characters do not
appear on the title bar, but the text after them does appear (i.e., if I run
"rxvt -title 'UNIDAD DE PROGRAMACIÓN Y EVALUACIÓN'" I get a window with the
title "UNIDAD DE PROGRAMACIN Y EVALUACIN".

Comment 1

[Reinout van Schouwen]

*** Bug 97279 has been marked as a duplicate of this bug. ***

Comment 2

[Reinout van Schouwen]

This bug is not Linux-only. I've encountered it on Solaris too, see bug 97279.

Comment 3

[R.K.Aa.]

I see the whole title in both this bug and the dup, with a current CVS build
running under RH7.1, XFree86 4.0.3 and the MS web-collection of truetype fonts
installed.

Comment 4

[Christian Reis]

Reinout, Reporter: I can't dupe this on Linux. It very much seems to me that it
is a local font configuration option, and I would like you both to check on the
availablity of the full Latin-1 charset in the fonts you are using for your
windowmanager titles. I use WindowMaker myself, and the titlebar font I use is 

-*-helvetica-bold-r-normal-*-12-*-*-*-*-*-*-*

Can you check please and guarantee this is not the case and that indeed mozilla
is setting an incorrect titlebar message?

I would also like you to run xprop and click on the browser window, and paste in
the WM_ICON_NAME and WM_NAME items for our review. Mine for that URL are:

WM_ICON_NAME(STRING) = "UNIDAD DE PROGRAMACI\323N Y EVALUACI\363N - Mozilla
{Build ID: 2001083108}"
WM_NAME(STRING) = "UNIDAD DE PROGRAMACI\323N Y EVALUACI\363N - Mozilla {Build
ID: 2001083108}"

And they render correctly.

Please follow-up.

Comment 5

[Gunnar Wolf]

Hi,

I have the full charset on the window titles... I just tested opening "rxvt
-title ?????" and it worked correctly.

Comment 6

[Christian Reis]

Could you please post the xprop output as I requested?

Have you tested with another windowmanager?
Please try and repost.

Comment 7

[Gunnar Wolf]

I exported the display to a Solaris machine running Openlook, and it also gets
truncated.

As a sidenote, I installed Galeon (which is based on Mozilla) on this same
machine, and it works correctly - I get the full title, both in Openlook and in
WindowMaker.

Comment 8

[Christian Reis]

Still waiting for xprop output and window manager switch :)

BTW: did you grab a fresh mozilla for galeon, or did you use the same
build as the mozilla base?

Comment 9

[Reinout van Schouwen]

Christian: 

I still confirm this with the nightly Solaris 2.6 build (ID 2001083110).

Title bar font: -adobe-helvetica-bold-r-*-*-*-120-*-*-*-*-*-* 
WM_ICON_NAME(STRING) = "UNIDAD DE PROGRAMACI"
WM_NAME(STRING) = "UNIDAD DE PROGRAMACI"(some other fonts I tried show the same
result)

I don't know how I could check for full Latin-1 compatibility, but  the problem
only shows up in the titlebar and nowhere else (also true for mail/news).

Comment 10

[Gunnar Wolf]

I have the following package versions:

mozilla - 0.9.3+0-3
mozilla-browser - 0.9.3+0-3
galeon - 0.11.5-3

(compiled by the Debian developers, I am just a regular user, and not too
enthusiast on compiling large packages ;-) )

Comment 11

[Christian Reis]

Reinout: now we're talking! 

Just to be certain, let's see if you can find out is that font is. I
have on my system the following variant:

-adobe-helvetica-medium-r-normal-*-*-140-*-*-p-*-iso8859-1

Which has the Latin-1 ending right there. Try running:

xlsfonts | grep helvetica | grep normal | grep iso8859-1

And seeing if you get output that shows the Latin-1 variant right there.
I'm quite convinced you will, and it does seem like a browser bug, so
I'll look for a dupe in the meantime. Thanks a lot for your help!

Comment 12

[Reinout van Schouwen]

Attachment: Output of xlsfonts

Comment 13

[Reinout van Schouwen]

OK, stupid question probably. Does iso8859-1 equal Latin-1 support?
Anyway- yes most of the fonts end with iso8859-1. I'll attach the xlsfonts
output for you.

Comment 14

[Christian Reis]

Yep, a whole bunch of latin-1 (which is iso-8859-1, yes, just easier to
type) fonts. So I'm going to reassign this (though I have no idea why
it's happening), mark as new, and change the subject.

Reinhout, thanks for the help! Let me just ask you another pair of
questions: 

- Does this _always happen_ if the title contains accents? Note that
  there are two ways accents can occur: a) using HTML entities, like
  á for instance b) using a latin-1 character directly like á.
  Both should work, of course, but your problem may be related to one or
  the other.

- Can you reproduce this on local documents, stored on your local drive,
  opened through the file:/// method?

Comment 15

[Christian Reis]

Roy, just want to point out a thing: the xprop WM string isn't being
set, so it means we're truncating the string somewhere inside mozilla
AFAICT, which is why I'm marking NEW. I can't get the reporter to
feedback on Linux, but Reinhout has confirmed this on Solaris.

Comment 16

[Reinout van Schouwen]

Christian:

Before reporting Bug 97279 I tested it both locally and remote with both
entities and Latin-1 chars. I couldn't find a condition where it didn't occur.

Comment 17

[Reinout van Schouwen]

Additional question... currently this bug is listed as occuring on DEC/Linux.
That probably keeps it from some peoples' radar that would be interested in
Solaris/sparc bugs, for instance. Is it somehow possible to list more than one
platform/OS for a bug, but not All?

Comment 18

[Gunnar Wolf]

Replying to Christian:

> - Does this _always happen_ if the title contains accents? Note that
>   there are two ways accents can occur: a) using HTML entities, like
>   á for instance b) using a latin-1 character directly like á.
>   Both should work, of course, but your problem may be related to one or
>   the other.

I created a file containing just this:
<HTML><HEAD><TITLE>T&iacute;tulo</TITLE></HEAD><BODY></BODY></HTML>
I did it using also the í character instead of &iacute;, with the same results -
It still gets truncated. 

> - Can you reproduce this on local documents, stored on your local drive,
>   opened through the file:/// method?

Yup...

Comment 19

[Gunnar Wolf]

Replying to Reinout:

> Additional question... currently this bug is listed as occuring on DEC/Linux.
> That probably keeps it from some peoples' radar that would be interested in
> Solaris/sparc bugs, for instance. Is it somehow possible to list more than one
> platform/OS for a bug, but not All?

I just tested it on a Mozilla 0.9.1 I have available on a i386 running Debian
Potato (Mozilla was installed as part of Ximian Gnome), and I got the same
behavior. The machine is not mine, so I did not do extensive tests, but at least
http://upe.iztacala.unam.mx still appears with the title truncated, as I
reported when starting the thread.

Comment 20

[Roy Yokoyama]

I have a whole bunch of titlebar text truncation bugs reported in few days.  
They may be related.
I'll collect all the bugs and consolidate them.
97671, 94011, 94024, 96875

Comment 21

[Roy Yokoyama]

97882, too.

Comment 22

[Roy Yokoyama]

=== assigning to jbetak

Comment 23

[[e-mail] (away - not reading bugmail)]

accepting...

Comment 24

[[e-mail] (away - not reading bugmail)]

pushing out

Comment 25

[Christian Reis]

I guess I deserve this. After pestering everybody about this bug being
unreproducible, I now have it with every site that has entities in
<TITLE>. Example sites:

http://www.infogratis.com.br/
http://www.async.com.br/articles/kiko13092000.php
http://www.async.com.br/address.php
http://home.uol.com.br/

And other major brazilian sites I visit. 

This happened over the course of last week, and now I'm trying to track
down why this is happening.

Comment 26

[Frank Tang]

give to nhotta, this looks like similar to the bug we fix for mac. Can you look
at the code on Linux a propose a patch? work with bstell if you don't have a build.

Comment 27

[nhottanscp]

yes, it needs nsIUnicodeEncoder::SetOutputErrorBehavior. I think that should be
an default behavior for encoder since almost everybody need it.

Comment 28

[nhottanscp]

Attachment: Patch, call SetOutputErrorBehavior to replace unmapped character by '?'.

Comment 29

[Gunnar Wolf]

Ummm... Wouldn't it be better to translate -if an ugly patch is needed-
international characters to the closest one known? At least for the most common
(acute/grave/diaeresis/tilde)... Translating them to the unmodified literal.
(anyway, I understand this would be a temporary patch)

Comment 30

[Frank Tang]

Comment on attachment 53686 [details] [diff] [review]
Patch, call SetOutputErrorBehavior to replace unmapped character by '?'.

r=ftang

Comment 31

[Frank Tang]

>yes, it needs nsIUnicodeEncoder::SetOutputErrorBehavior. 
>I think that should be an default behavior for encoder since almost everybody 
need it.
Not true. Unix font system need it without convert to '?'. There are many other 
places need it without convert to '?'. But that have nothing with your patch 
anyway. your patch looks good r=ftang

>Ummm... Wouldn't it be better to translate -if an ugly patch is needed-
>international characters to the closest one known? At least for the most common
>(acute/grave/diaeresis/tilde)... Translating them to the unmodified literal.
>(anyway, I understand this would be a temporary patch)
I don't mind if you want to work on such patch. We already have a 
transliteration facility in nsIEntityConverter. But I think that kind of 
improvement is not worthy. What we really should do is generate ICCCM string or 
UTF-8 string.

Comment 32

[Frank Tang]

and notice we already try UTF-8 in some limited way in our current code. Some 
window manager know how to handle UTF-8.

Comment 33

[Christopher Blizzard (:blizzard)]

Comment on attachment 53686 [details] [diff] [review]
Patch, call SetOutputErrorBehavior to replace unmapped character by '?'.

sr=blizzard

Comment 34

[nhottanscp]

Checked in to the trunk.

Comment 35

[Yuying Long]

The original truncation problem is not reproducible on 11-16 linux trunk build,
and for accent characters are not display properly is handled by some other bug. 

Mark this as verified.