23153 - Int. chars in pathname of profile folder brakes the mozregistry.dat file

Status: VERIFIED FIXED

(Core Graveyard :: Profile: BackEnd, defect, P1)

Description

[Henrik Gemal]

If you delete your mozregisty.dat and creates a new profile where the profile
directory is c:\program files\mozilla\זרו and the exit Mozilla and restart
Mozilla the Create New Profile will show again.
All the files "prefs.js", "history.dat", etc will be created the right place but
the mozregistry.dat is corrupt.
I'll attach the corrupted one.

Comment 1

[Henrik Gemal]

Attachment: Corrupt mozregistry.dat

Comment 2

[selmer (gone)]

Bhuvan, I thought this was supposed to work...

Comment 3

[racham]

This worked when tried with japanese version while verifying a separate i18n
bug. I will check it again.

Comment 4

[Henrik Gemal]

More investigation shows that the mozregistry.dat doesn't get corrupted the
profile just doesn't get saved.
The same thing happens if I enter int. chars into the profile name.
Custom folder: "c:\program files\mozilla\users"
Profile name: "Hænrik Gømal"

I can start Mozilla but the profile doesn't get written...

Comment 5

[selmer (gone)]

Bumping this to the top of the M14 list.

Comment 6

[racham]

Adding dan veditz to the cc list.

Dan,

Please cc me on the nsRegisrty revamp bug you are planning.
That bug should also include the need to handle
PRUnichar and UTF-8 conversions.

Comment 7

[Henrik Gemal]

This seems to be working now.
I've successfully create a profile called "Hænrik Gæmal"
I've also successfully create a profile called "læp" in a directory called
"c:\program files\mozilla\æøå\"

Comment 8

[Henrik Gemal]

I just realized even though that I could succesfully create these profiles the
profiles doesn't show up in the Profile Selection the next time I started
Mozilla!

Comment 9

[racham]

Making this dependent on 23859.

Comment 10

[leger]

Moving all Profile Manager bugs to new Profile Manager Backend component.
Profile Manager component to be deleted.

Comment 11

[leger]

Putting on PDT+ radar for beta1.

Comment 12

[racham]

Reassinging the bug to gayatri. Thanks gayatri!
I will work with gayatri on this bug after I'm done 
with the other 2 pdt+ bugs (19621, 23390).

Comment 13

[gayatrib]

There is still work to be done to fix this bug--I need a couple of more days. 
Changing the fix date to 2/17. This is a large scale change and there is a lot 
of testing that needs to be done. Selmer was going to talk to jar about this.

Comment 14

[gayatrib]

On reviewing the situation again, changing the expected checkin date to 2/18. 
Adding one more day i.e.

Comment 15

[gayatrib]

Due to dependency on bug 23859, the closing on this bug got delayed. Expected 
checkin date is 2/23--giving an extra day for testing as the changes are 
extensive.

Comment 16

[gayatrib]

I have all the code changes ready and I've had initial rounds of code reviews 
and testing done on this. I worked on fixing some of the memory leaks and am 
currently trying to debug into a couple of crashers that I am running into. Once 
this is done, I should be able to checkin. 

Changing the target checkin date to wednesday, 03/01. It should be ready-to-go 
by then.

Comment 17

[Jim Roskind]

Can we have a status-whiteboard update on this bug?  There are other
dependencies, and we need to get all the ducks lined up.
Thanks ;-)

Comment 18

[gayatrib]

I am waiting on a code review. I am hoping that I will be able to checkin 
today/tomorrow--depending on when I get an ok on it.

Comment 19

[Jim Roskind]

Alec: When will we have the code review completed, so that this can land? Any 
anticipated problems?
Thanks,
Jim

Comment 20

[Alec Flett]

I have already given lots of feedback on the code. I haven't seen a patch since
my last feedback on friday.

Comment 21

[Daniel Veditz [:dveditz]]

The latest code I've received from Gayatri works when combined with my patch to 
profileSelection.js to work around the RDF problems. Waiting for a review on 
that one and for a final version of Gayatri's changes.

Comment 22

[selmer (gone)]

The Ja char problem came down to nsFileSpec not being I18N ready.  We give up 
for beta1.

Comment 23

[Phil Peterson]

PDT-

Comment 24

[bobj]

selmer, Pls be specific about how nsFileSpec is not I18N ready.  We are
making use of it in other areas.
Cc'ing ftang and nhotta.

Comment 25

[selmer (gone)]

We attempted to use the one and only interface that takes an nsString parameter 
and it would not give back a good pathname that could be used to access or 
create a directory.  Without the directory, none of the rest of what we do 
works.  Gayatri can give more details on what we called and how, she should 
still have the code on her machine.

Comment 26

[nhottanscp]

The same problem when creating a local mail folder (bug 29543).
Need change in nsFileSpec to do a conversion from a file system charset to 
unicode or rewite to store path string in unicode internally.

Comment 27

[gayatrib]

The problem is that nsFileSpec uses single byte strings all over. We need double 
byte for it to work for japanese characters. We tried calling SetLeafNode call 
in nsFileSpec, but it boils down to using char*.

One solution would be to use nsIFile--but again, they support UTF8--but it might 
not work for Japanese char set. And also there are other problems with using 
nsIFile. I need to do code like this:

//Basically I need to have dirSpec to be type nsCOMPtr<nsIFile> :
// the current code cannot deal with double byte strings

NS_WITH_SERVICE(nsIFileLocator, locator, kFileLocatorCID, &rv);

nsFileSpec dirSpec;
nsCOMPtr <nsIFileSpec> defaultRoot;

rv = locator->GetFileLocation(
                nsSpecialFileSpec::App_DefaultUserProfileRoot50,
                getter_AddRefs(defaultRoot));

defaultRoot->GetFileSpec(&dirSpec);
if (!dirSpec.Exists())
    dirSpec.CreateDirectory();

dirSpec += profileName; // profileName PRUnichar*

How would I successfully change this code to work for nsIFile. It looks like 
currently there is no way to go from a nsIFile to an nsILocalFile. Such a 
utility needs to be created in xpcom/io.


The other solution is that there is code in nsWebShell that could probably be 
used to serve our purpose as well--it seems to be used to convert a Unicode 
"file:" URL into a real string. 
http://lxr.mozilla.org/mozilla/source/webshell/src/nsWebShell.cpp#1731

This code needs to be moved into nsStdURL::GetFile/SetFile (a necko util) so 
that can be shared by all. This is filed as bug 29341.

Given all the above restrictions, it did not look like this was doable in beta 
time frame. The best path to be followed needs to be decided as yet. Do we add 
code to xpcom/io, do we use necko...? 

Comment 28

[gayatrib]

In the baove comment I meant to write:
It looks like currently there is no way to go from nsIFileSpec to nsILocalFile. 
I wrote nsIFile to nsILocalFile by mistake... Sorry.

Comment 29

[Daniel Veditz [:dveditz]]

nsLocalFile--at least on windows--is **NOT** UTF8. It takes a char* in the 
native filesystem charset, leaving it up to the caller to figure that out.

This is an improvement over nsFileSpec which took a wstring and dropped the 
high byte of each char (i.e. assumed Latin-1). It still pushes the conversion 
task out to each caller who arguably shouldn't have to know about the 
filesystem charset.

It also makes nsIFile nearly useless from javascript on non-western systems 
even though it is technically "scriptable". xpconnect will see the "string" 
API's and always apply a Unicode->Latin-1 conversion to javascript data.

Comment 30

[bobj]

dveditz: ... It still pushes the conversion task out to each caller who
dveditz: arguably shouldn't have to know about the filesystem charset.

We provide the nsIPlatformCharset class, so the caller does not need to know
the specific filesystem charset for the current platform, it only needs to
know that it needs to convert to (or from) "the" filesystem charset.

nhotta, pls provide an example where this is used.  Is this a good one?:
  http://lxr.mozilla.org/seamonkey/source/mailnews/base/util/nsMsgI18N.cpp#215
  215 // Charset used by the file system.
  216 const nsString& msgCompFileSystemCharset()
  ...

dveditz: It also makes nsIFile nearly useless from javascript on non-western
dveditz: systems even though it is technically "scriptable". xpconnect will
dveditz: see the "string" API's and always apply a Unicode->Latin-1 conversion
dveditz: to javascript data.

I'm not sure if we have made this scriptable yet.  nhotta?

Comment 31

[bobj]

Other examples of this usage would be in the file browser widget and RDF.

Comment 32

[gayatrib]

The profile manager is now i18n friendly. 
Some issues are:
It works well for european characters.
It might not work for japanese--if the directory name is japanese. If the 
profile name is japanese and directory name english it should be fine. This is 
because nsIFile is not yet i18n friendly.

Marking it fixed. If there are any issues, please open separate bugs.

Comment 33

[bobj]

gayatrib> It works well for european characters.

Be sure to test on a Mac, since it uses MacRoman and not Latin1.
Also there are FS differences between Win9x and NT too.

gayatrib> This is because nsIFile is not yet i18n friendly.

What are your issues, other than the ones dveditz mentioned above and I tried
to answer?  nhotta, Please assist gayatrib on this.

Comment 34

[gayatrib]

bobj>What are your issues, other than the ones dveditz mentioned above and I 
tried to answer?

No other issues. I hadn't refreshed my mind on the preceeding comments before 
writing mine. Sorry.

Comment 35

[Grace Bush]

Mac and Win builds for 20000414