138638 - Accented and special chars (®, ©) are not displayed correctly in Sidebar

Status: VERIFIED FIXED

(Core :: Internationalization, defect)

Description

[marina]

*** observed with 2002-04-19 trunk build ***
Steps to reproduce:
- go to http://www.google.com;
- enter "albert ellis";
- get results and look at the sidebar where the results are displayed, note that
® and ™ are displayed garbled. This doesn't happen in case i would do the search
directly in the sidebar, in that case the display will be correct.Google does
utf-8 search, doesn't it? Screen shot to follow

Comment 1

[marina]

Attachment: here is a screen shot to showthe problem, make sure you scroll to see the display in the sidebar

Comment 2

[Roy Yokoyama]

marina: regression? If so, then please nominate this for nsbeta1 and indicate
the regression build date?

Comment 3

[marina]

i am looking at 6.2. build that we shipped on MacOSX and the display here is
correct, so it is a regression, nominating.I'll invistigate when the regression
started

Comment 4

[marina]

somehow i don't see this problem in the latest trunk neither in the branch...
gone, i wish i knew what fixed it.

Comment 5

[nhottanscp]

WFM?

Comment 6

[marina]

yes, WFM

Comment 7

[marina]

i spoke too soon... confused the test case, i searched directly in the Sidebar (
and this had no problem). The wrong display is when you search in the browser
and the reasults are displaying  in the Sidebar. Let me see when did that
started...( it is OK in 6.2 )...Sorry for the confusion

Comment 8

[nhottanscp]

Trademark is shown without the problem in the sidebar using 4/26 commercial branch.

Comment 9

[marina]

i am testing this right now and i can see the trade mark displaying fine with 
a new profile but not with my current one, even in 6.2 , so it looks like the 
profile got corrupted. I have to do one more test with importing IE favorites, i 
guess i started to notice this after i was verifying import of the bookmarks 
from IE. More comments later

Comment 10

[marina]

well, after repeating all tests that i performed for bookmark bug i didn't get a 
new profile corrupted, so i have no clue why my current profile displays the 
trade mark and other special chars corrupted in the sidebar... I am going to 
mark it as WFM

Comment 11

[marina]

i am still hezitant to close this bug. Talked to Tao and his guess is that after
the fix for double-byte search went into the trunk it could have some
sideeffects. The way i install daily builds could affect this as well:i use  the
same dir for branch and trunk ( i unistall them previously) but because i launch
with the same profile the prefs that are getting written into panels.rdf,
localstore.rdf and prefs.js could somehow got confused. Cc'ing to Frank and
Shanjian for possible clues.

Comment 12

[marina]

i see it on the trunk but not on the branch with today's builds.

Comment 13

[Roy Yokoyama]

>Talked to Tao and his guess is that after
>the fix for double-byte search went into the trunk
What's the bug number so that I can quickly take look at the patch?

Comment 14

[marina]

cc'ing Tao for the answer..

Comment 15

[marina]

i am seeing this on the branch now (2002-05-28)with a new profile, i'll attach a 
screen shot in a moment

Comment 16

[marina]

Attachment: here is a screen shot to illustrate the problem, the meta tag points to UTF8, could that be that in the left frame Latin1 is used?

Comment 17

[nhottanscp]

In the screen shot, the sidebar display is corrupted. How about the item in the
search result page, was that shown correctly? That is not included in the screen
shot.

Comment 18

[marina]

yes, the results in the serach page are displayed corretly, screen shot to follow

Comment 19

[marina]

Attachment: look at the search page on the right, article #42 is displayed correctly but is garbled in the sidebar

Comment 20

[Frank Tang]

nhotta, please work with [e-mail] to change the 
google.src file in mozilla tree and also
http://www.google.com/mozilla/google.src so we will use UTF-8 for communication. 

Comment 21

[Frank Tang]

if we fix the file on the http://www.google.com/mozilla/google.src , we may not
even need land the change into the mozilla tree (but I think we should) 

Comment 22

[marina]

BTW: same happens with Netscape search

Comment 23

[nhottanscp]

I cannot reproduce this with google.
But I can reproduce this with Netscape search.

Comment 24

[marina]

it is much easier to reproduce with Netscape search, so changing the URL for the
test case

Comment 25

[nhottanscp]

Netscape search result page is UTF-8.
I think the sidebar search result view depends on a system's charset (i.e. need
Chinese system to show Chinese result). 

A separate (non browser) problem is that the Netscape home page search does not
accept non ASCII queries. As a result, we don't often encounter the sidebar
problem currently.

Comment 26

[nhottanscp]

There is a problem when the search code parses the result page. The code creats
a Unicode converter but it does not canonicalize the charset name which is
specified in the .src file. That has to be fixed.
So if I change the charset name from the current "utf-8" to "UTF-8" then the
sidebar shows TM correctly.

Comment 27

[nhottanscp]

Attachment: Map charset name to charset atom, this fixes the charset name canonicalization problem.

Comment 28

[Robert John Churchill]

Comment on attachment 86311 [details] [diff] [review]
Map charset name to charset atom, this fixes the charset name canonicalization problem.

r=rjc

Comment 29

[Alec Flett]

Comment on attachment 86311 [details] [diff] [review]
Map charset name to charset atom, this fixes the charset name canonicalization problem.

sr=alecf

Comment 30

[Frank Tang]

adt2 since it is very easy to see from the main ui even in English build with
english text. 

Comment 31

[nhottanscp]

checked in to the trunk

Comment 32

[scottputterman]

Please verify this on the trunk.

Comment 33

[Rui Xu]

Verified with 2002061204 trunk on SC WinXP.

Comment 34

[scottputterman]

adding adt1.0.1+.  Please get drivers approval before checking in.

Comment 35

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 86311 [details] [diff] [review]
Map charset name to charset atom, this fixes the charset name canonicalization problem.

Please land this on the 1.0.1 branch.  Once there, remove the
"mozilla1.0.1+" keyword, and add the "fixed1.0.1"

Comment 36

[nhottanscp]

checked in to 1.0.1 branch

Comment 37

[Rui Xu]

Verified fixed on branch 2002-06-18-08