155668 - POP: Japanese name folder looks weird after restart

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect)

Description

[Jeesun Cho]

POP: Japanese name folder looks weird after restart

Steps:
1. Setup a POP account
2. Create a JA name folder (in my case, ねこ - see this in Shift_JIS）
3. The new folder is displayed correctly
4. Exit and restart the browser
5. Notice that the JA folder name looks weird

Note: 
On IMAP account, there's no such problem.
Latin-1 folders look fine after restart.
If you open a POP account in a new profile, a new folder won't show before you
close/reopen mail window (see bug 155658 )

Comment 1

[nhottanscp]

>5. Notice that the JA folder name looks weird
Please attach a screen shot.

Comment 2

[Jeesun Cho]

Attachment: A screenshot of folders

Right below Trash folder, there's "mS" folder. It should show the JA name I
gave upon creation.

Comment 3

[nhottanscp]

This is a regression from 7.0PR1. Jeesun, please find out when this has started.

Reassigned to naving, cc to putterman. Can this be related to the recent POP
folder related change?

Comment 4

[nhottanscp]

The folder name which are displayed as random ASCII ("mS" in the screen shot),
it shows fine as Japanse when I switch back to 7.0 PR1. So it look like this is
a display problem.

Comment 5

[Navin Gupta]

How does it work for latin-1 folders then ?

Comment 6

[Jeesun Cho]

I first saw this on 0701 branch build.
0627 branch build does NOT show this problem. 
So it came back inbetween 0628 branch and 0701 branch.
For trunk, both 0627 trunk and 0628 trunk have the same problem.

Comment 7

[Jeesun Cho]

>How does it work for latin-1 folders then ?
As indicated in the report, Latin-1 folders work fine.

Comment 8

[Navin Gupta]

I know this has been caused by setting user given name as prettyName which we
store as PRUnichar * rather than using the disk name. I am wondering why it
works for latin-1 and not for japanese. 

Comment 9

[Navin Gupta]

When we store this name in the db (summary file) we just do
ToNewCString(PRUnichar*) I think that causes loss in case of japanese. nhotta, how
should we convert here, perhaps to utf8 would fix it, right ?

Comment 10

[nhottanscp]

Yes, I think so, storing as UTF-8 would not lose the data.

Comment 11

[nhottanscp]

06/28 branch does not have the problem, it started from 06/29 branch.

Comment 12

[Navin Gupta]

Can I have japanese, chinese names to play around with ? thx. 

Comment 13

[Jeesun Cho]

See this in Shift_JIS.
I used ねこ, a JA name.

Comment 14

[nhottanscp]

Local folder name is dependent to default OS setting. So you need to switch the
default to test. For Japanese, you can copy/paste text from home.netscape.co.jp.
Please contact me for detail.

The problem can also be reproduced by using a symbol character. I tried a
trademark and it turned to a double-quote after the re-launch.
Here is how to type trademark on Windows.
Hold Atl key then hit numeric pad 1, 5, 3 then release Alt key.

Comment 15

[Navin Gupta]

Attachment: proposed fix

I think this should fix it. nhotta, Can you try this fix w/ your testcases ?
thx.

Comment 16

[nhottanscp]

I applied the patch to a debug build on Windows NT4 Japanese. 
It fixes the problem. I created a couple of Japanese folder names and quit,
after I re-launch the folder names are still shown correctly.

But for Japanse folders which already exist, I had to remove summary files
(.msf) to make them shown correctly. We have to do something about that because
all the exisiting non Latin1 local folder have their name stored broken in
summary files.

Comment 17

[Navin Gupta]

yeah, that is a tricky issue. trunk builds from 6/27 onwards and branch builds
from 6/29 onwards have this problem. I don't see how we can fix it other than
just asking users to delete their msf files. 

Comment 18

[scottputterman]

A small # of users will have been hit by this (and no one from using a major
milestone such as 1.1, 1.0.1, or Mach V).  So, I think we can focus on fixing it
for newly created folders.

Comment 19

[nhottanscp]

Actually the folders I created by 6.2 and PR1 cannot be shown correctly even
with the patch, I have to delete .msf files.

Comment 20

[Navin Gupta]

Acutally all 6.x builds will have this problem because we were never converting
mailboxName to UTF8 as far as I can see. So we were storing lossyConverted char*
in the db.

I can go back and undo setting of prettyName which will fix this bug for cases
where disk names match the prettyNames (which should cover most cases). 

Comment 21

[Navin Gupta]

Attachment: proposed fix, v2

Don't set the prettyName as mailboxName.

Comment 22

[Navin Gupta]

David, Cavin, Can I get reviews for last patch ? thx. 

Comment 23

[nhottanscp]

I removed the first patch and applied the last patch. I created a folder by PR1
and that is shown correctly without removing .msf.

One question, the pretty name change was a part of bug 153982 fix. Would
reverting it not affect the fix of bug 153982?

Comment 24

[Navin Gupta]

No, it would not affect fix for bug 153982. It was not totally related to that
bug but it was the right thing to do because we should be storing prettyName in
case disk name doesn't match prettyName. When you create a local folder where
disk name obtained after hashing is not the same as prettyName, it will show
hash name
in the folder pane now (I think this was the same behavior before bug 153982 was
fixed).

Comment 25

[Cavin Song]

Comment on attachment 90160 [details] [diff] [review]
proposed fix, v2

r=cavin.

Comment 26

[scottputterman]

Let's get an sr= on this so it can land in the trunk.  How risky is this patch?

Comment 27

[David :Bienvenu]

Comment on attachment 90160 [details] [diff] [review]
proposed fix, v2

sr=bienvenu

Comment 28

[Jeesun Cho]

QA contact over to me.

Comment 29

[scottputterman]

Navin, can you land this on the trunk?  After it's landed, Jeesun, can you
verify this?  Navin, are there any other areas besides this actual bug that we
should test around to reduce risk?

Comment 30

[Navin Gupta]

fixed on trunk. 

As I have mentioned before we may see the hash names instead of actual folder
names in some cases when you start-up after creating a new folder. I think we
will have to relnote this. 

Comment 31

[Jeesun Cho]

Verified on 0709 trunk build
1. I created a folder using this build. After restart, it was displayed correctly.
2. I first created a folder using PR1. And it is also shown correctly in today's
trunk build.

Comment 32

[Jaime Rodriguez, Jr.]

adt1.0.1+ (on ADT's behalf) approval for checkin to the 1.0 branch. pls check
this into asap, then replace "mozilla1.0.1+" with "fixed1.0.1".

Comment 33

[Judson Valeski]

please checkin to the 1.0.1 branch. once there, remove the "mozilla1.0.1+"
keyword and add the "fixed1.0.1" keyword.

Comment 34

[Navin Gupta]

fixed on branch

Comment 35

[Jeesun Cho]

Verified on 0711 branch build