Can't move non-ascii subfolder on IMAP

VERIFIED FIXED in mozilla1.0.1


17 years ago
11 years ago


(Reporter: jeesun, Assigned: naving)


({dataloss, intl})

Windows XP
Dependency tree / graph

Firefox Tracking Flags

(Not tracked)


(Whiteboard: [adt1 rtm] [ETA 06/28])


(1 attachment, 5 obsolete attachments)



17 years ago
Can't move non-ascii subfolder under special folders on IMAP
Platform: JA windows XP
Input: JA chars

1. On IMAP account, create a non-ascii folder and subfolder under it.
2. Move them (both at the same time by highlighing the 1st level folder) by
dragging and dropping them to Sent or Drafts folder.
3. Both will appear under Sent folder for example, but when you click on the 2nd
level folder, you'll get an error: The mail server responded: Mailbox does not exist

English folders work fine.

Comment 1

17 years ago
This seems related to bug 153982
Keywords: intl

Comment 2

17 years ago
btw: this error can be reproduced with any folder on the IMAP tree (not only
with special folders). Try to drag the newly created non-ascii subfolder to the
top of the tree ( or to any ascii/non-ascii folder of the
first level: you'll get the same error- seems that the copy/move of the
non-ascii subfolder is not allowed.

Comment 3

17 years ago
You're right, Marina. Change the summury accordingly.
Summary: Can't move non-ascii subfolder under special folders on IMAP → Can't move non-ascii subfolder on IMAP

Comment 4

17 years ago
changing qa contact.Thanks!
QA Contact: marina → jeesun

Comment 5

17 years ago
This is dataloss. Is there anyway to recover the data?
Nominating for nsbeta1. Reassign to naving, cc to putterman
Assignee: nhotta → naving
Keywords: dataloss, nsbeta1

Comment 6

17 years ago
I can reproduce this with Latin1 characters.

parent folder "ссс", subfolder "ррр".

Comment 7

17 years ago
1) In nsImapMailFolder::RenameSubFolders,

6905      rv = AddSubfolderWithPath(&utf7LeafName, dbFileSpec,
The argument utf7LeafName contains non UTF-7 after the call. Then the following
call to append to CString fails because utf7LeafName contains non ASCII.

2) In nsImapMailFolder::AddSubfolderWithPath,
323   uri.Append(*name);
324   char* uriStr = ToNewCString(uri);

this doesn't seem to be right since 'name' may be non ASCII

I got assersions for both places with non ASCII subfolder name.

Naving, could you take a look at this?

Comment 8

17 years ago
>I can reproduce this with Latin1 characters.
>parent folder "ссс", subfolder "ррр".
Parent folder does not have to be non ASCII, subfolder has to be non ASCII.

Comment 9

17 years ago
this is an important intl functionality, it is not a regression from 6.2 (is
reproducable in the shipped versoins)

Comment 10

17 years ago
Posted patch Convert subfolder to UTF-7. (obsolete) — Splinter Review
The dataloss problem is fixed with this patch. But I still see the assersion
for AddSubfolderWithPath using ToNewCstring.

Comment 11

17 years ago
I suggest we take this as nsbeta1+ and mark it as [adt1] since this could be a
huge data loss (the loss of a whole folder of messages) . Since I am not the
engineer manager of naving, I will leave that to his manager to mark that. 
Blocks: 141008

Comment 12

17 years ago
N6.2 have this problem too. So this is not a regression frrom 6.2 to 7.0

Comment 13

17 years ago
accepting. working on a fix.

Comment 14

17 years ago
If they are really losing their folder then this is a pretty critical bug.
Keywords: nsbeta1nsbeta1+
Whiteboard: [adt1 rtm]

Comment 15

17 years ago
Scott, please also look at bug 153982, similar problem also happens for POP.

Comment 16

17 years ago
Posted patch proposed fix (obsolete) — Splinter Review
I made some changes to nhotta's last patch. We need to use modutf7 for online
Regarding assertion the problem is in AddSubfolderWithPath where it does
ToNewCString. It should actually convert name to modutf7 when constructing
uriStr. I think this patch is ok to fix this bug.

Comment 17

17 years ago
Cavin, David, Can I get reviews ? thx.

Comment 18

17 years ago
I think rdf uri's have to be in utf8 format so we cannot do what I suggested to
fix the assertion. 

Comment 19

17 years ago
Comment on attachment 89115 [details] [diff] [review]
proposed fix


Comment 20

17 years ago
This fixes the assertion for me in AddSubfolderWithPath. still investigating if
this is correct

@@ -321,7 +321,7 @@
-  char* uriStr = ToNewCString(uri);
+  char* uriStr = nsCRT::strdup(NS_ConvertUCS2toUTF8(uri.get()).get());
   if (uriStr == nsnull) 

Comment 21

17 years ago
Attachment #89115 - Attachment is obsolete: true


17 years ago
Blocks: 143047
Whiteboard: [adt1 rtm] → [adt1 rtm] [ETA 06/27]
Target Milestone: --- → mozilla1.0.1

Comment 22

17 years ago
Comment on attachment 89115 [details] [diff] [review]
proposed fix

Attachment #89115 - Flags: superreview+
Attachment #89115 - Flags: review+

Comment 23

17 years ago
fixed on trunk
Closed: 17 years ago
Keywords: adt1.0.1
Resolution: --- → FIXED
jeesun - cna you pls verify this fix on the trunk, then  mark it as resolved?

Comment 25

17 years ago
I'll pick up tomorrow's trunk build and verify this.

Comment 26

17 years ago
Reopen this. Try this slightly different scenario. I'm still experiencing data
loss. See the attached screenshot.
1. Create a non-ascii folder and a subfolder under it (ゆき、 ゆき2)
2. Move both to a 1st level folder (ifolder)
3. Then create ゆき3 under ゆき2
4. Notice the tree structure looks weird (The original ゆき2 is grayed-out and
the same folder was created again. See the screenshot.)
5. Move the whole ゆき1,2,3 folders back to the top of the tree, out of
6. ゆき3 disappeared
Resolution: FIXED → ---

Comment 27

17 years ago
Posted image non-ascii folder screenshot (obsolete) —
Please see this page in Shift_JIS

Comment 28

17 years ago
Please create a reproducible test case for your comment #26, so everyone can

Comment 29

17 years ago
Jeesun, plesae use Latin1 characters (e.g. á) for your test case (when the
problem is generic for non ASCII characters), so pepole who do not have Japanese
system support can reproduce and debug, thanks.

Comment 30

17 years ago
Sure! This is reproducible using latin-1 chars too. Just use latin-1 chars where
I mention JA chars.
My intention was to give you an example to describe the problem and reproducible
steps in more detail. 

Comment 31

17 years ago
I don't know what is latin-1 and what is japanese. Can you give a list of
reproducible steps with folder names in latin-1.

Comment 32

17 years ago
1. Create a latin-1 folder, for example, tàmü and a subfolder, tàmü2 under it 
2. Move both to a 1st level folder, say its name is ifolder.
3. Then create yet another subfolder, tàmü3 under tàmü2
4. Notice the tree structure looks weird (The original tàmü2 is grayed-out and
the same folder was created again.)
5. Move the whole tàmü, tàmü2, tàmü3 folders back to the top of the tree, out of
6. tàmü3 disappeared
removing the nomination because this bug has been reopened.
Keywords: adt1.0.1

Comment 34

17 years ago
working on a fix. 

Comment 35

17 years ago
I have a fix for problem described in comment #32. But I am seeing other
problems where LSUB command doesn't tell us all the children so when we
*restart* tàmü3 disappears. This is relevant portion of the imap log for the
last rename command
when doing comment #32. Remember we do lsub command before we do rename. Relevant
code to look at is RenameHierarchyByHand in nsImapProtocol. David, what do you 
think ?

2512[4ffa200]: nsmail-2:S-ifolder/t&AOA-m&APw-:SendData: 6 lsub ""

2512[4ffa200]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: * LSUB
(\HasNoChildren) "/" ifolder/t&AOA-m&APw-/t&AOA-m&APw-1/t&AOA-m&APw-2

2512[4ffa200]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: 6 OK

2512[4ffa200]: nsmail-2:S-ifolder/t&AOA-m&APw-:SendData: 7 close

2512[4ffa200]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: 7 OK

2512[4ffa200]: nsmail-2:A:SendData: 8 rename "ifolder/t&AOA-m&APw-" "t&AOA-m&APw-"

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 8 OK Completed

2512[4ffa200]: nsmail-2:A:SendData: 9 subscribe "t&AOA-m&APw-"

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 9 OK Completed

2512[4ffa200]: nsmail-2:A:SendData: 10 unsubscribe "ifolder/t&AOA-m&APw-"

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 10 OK Completed

2512[4ffa200]: nsmail-2:A:SendData: 11 subscribe

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 11 NO Mailbox does not exist

2512[4ffa200]: nsmail-2:A:SendData: 12 unsubscribe

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 12 OK Completed

2716[50e0c50]: nsmail-2:NA:CreateNewLineFromSocket: * OK IMAP4
service (Netscape Messaging Server 4.15 Patch 4 (built Dec  7 2000))

2716[50e0c50]: nsmail-2:NA:SendData: 1 authenticate plain

2716[50e0c50]: nsmail-2:NA:CreateNewLineFromSocket: + 

2716[50e0c50]: nsmail-2:NA:SendData: Logging suppressed for this command (it
probably contained authentication information)
2716[50e0c50]: nsmail-2:NA:CreateNewLineFromSocket: 1 OK User logged in (no

2716[50e0c50]: nsmail-2:A:SendData: 2 XSERVERINFO MANAGEACCOUNTURL

2716[50e0c50]: nsmail-2:A:CreateNewLineFromSocket: * XSERVERINFO

2716[50e0c50]: nsmail-2:A:CreateNewLineFromSocket: MANAGELISTSURL NIL

2716[50e0c50]: nsmail-2:A:CreateNewLineFromSocket: 2 OK Completed


Comment 36

17 years ago
If I try to break up the hierarchy and subscribe for each children folder then 
also it doesn't fix it. cc jgmyers for input as well. 

3040[51627b0]: nsmail-2:S-ifolder/t&AOA-m&APw-:SendData: 6 
lsub "" "ifolder/t&AOA-m&APw-/*"

3040[51627b0]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: * LSUB 
(\HasNoChildren) "/" ifolder/t&AOA-m&APw-/t&AOA-m&APw-2/t&AOA-m&APw-3

3040[51627b0]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: 6 OK 

3040[51627b0]: nsmail-2:S-ifolder/t&AOA-m&APw-:SendData: 7 close

3040[51627b0]: nsmail-2:S-ifolder/t&AOA-m&APw-:CreateNewLineFromSocket: 7 OK 

3040[51627b0]: nsmail-2:A:SendData: 8 rename "ifolder/t&AOA-m&APw-" "t&AOA-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 8 OK Completed

3040[51627b0]: nsmail-2:A:SendData: 9 subscribe "t&AOA-m&APw-"

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 9 OK Completed

3040[51627b0]: nsmail-2:A:SendData: 10 unsubscribe "ifolder/t&AOA-m&APw-"

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 10 OK Completed

3040[51627b0]: nsmail-2:A:SendData: 11 subscribe "t&AOA-m&APw-w-/t&-AOA-m&-APw-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 11 NO Mailbox does not exist

3040[51627b0]: nsmail-2:A:SendData: 12 unsubscribe "ifolder/t&-AOA-m&-APw-/t&-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 12 OK Completed

3040[51627b0]: nsmail-2:A:SendData: 13 subscribe "t&AOA-m&APw-w-/t&-AOA-m&-APw-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 13 NO Mailbox does not exist

3040[51627b0]: nsmail-2:A:SendData: 14 unsubscribe "ifolder/t&-AOA-m&-APw-/t&-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 14 OK Completed

2624[518f7b0]: nsmail-2:A:SendData: 12 select "t&AOA-m&APw-"

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: * FLAGS (\Answered \Flagged 
\Draft \Deleted \Seen)

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: * OK [PERMANENTFLAGS 
(\Answered \Flagged \Draft \Deleted \Seen \*)]  

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: * 0 EXISTS

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: * 0 RECENT

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: * OK [UIDVALIDITY 

2624[518f7b0]: nsmail-2:A:CreateNewLineFromSocket: 12 OK [READ-WRITE] Completed

Comment 37

17 years ago
This patch fixes the bug but there are some problems as described in my earlier

comments. AddSubfolderWithPath should actually take char* instead of
nsAutoString if we want to avoid converting back and forth from unicode to utf7

and vice-versa. But we can do that in a separate bug.

Comment 38

17 years ago
this subscribe failed:
2512[4ffa200]: nsmail-2:A:SendData: 11 subscribe

2512[4ffa200]: nsmail-2:A:CreateNewLineFromSocket: 11 NO Mailbox does not exist

and here:
3040[51627b0]: nsmail-2:A:SendData: 11 subscribe "t&AOA-m&APw-w-/t&-AOA-m&-APw-

3040[51627b0]: nsmail-2:A:CreateNewLineFromSocket: 11 NO Mailbox does not exist

presumably, that's the cause of the problem. I can't tell if your patch fixes
that problem or not.

Comment 39

17 years ago
The last patch doesn't fix the problem of tàmü3 disappearing on restart. 

>This patch fixes the bug but there are some problems as described in my earlier

Actually if I try the same thing with ascii folder names it works fine. So this 
problem (tàmü3 disappearing on restart) appears to be a *server issue* because
the server doesn't tell us the children correctly on doing lsub command. 
For a comparison please look at comment #35 and this comment, thx. 

2624[4c02a10]: nsmail-2:S-ifolder/A:SendData: 7 lsub "" "ifolder/A/*"

2624[4c02a10]: nsmail-2:S-ifolder/A:CreateNewLineFromSocket: * LSUB 
(\HasChildren) "/" ifolder/A/B

2624[4c02a10]: nsmail-2:S-ifolder/A:CreateNewLineFromSocket: * LSUB 
(\HasNoChildren) "/" ifolder/A/B/C

2624[4c02a10]: nsmail-2:S-ifolder/A:CreateNewLineFromSocket: 7 OK Completed

2624[4c02a10]: nsmail-2:S-ifolder/A:SendData: 8 close

2624[4c02a10]: nsmail-2:S-ifolder/A:CreateNewLineFromSocket: 8 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 9 rename "ifolder/A" "A"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 9 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 10 subscribe "A"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 10 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 11 unsubscribe "ifolder/A"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 11 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 12 subscribe "A/B"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 12 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 13 unsubscribe "ifolder/A/B"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 13 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 14 subscribe "A/B/C"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 14 OK Completed

2624[4c02a10]: nsmail-2:A:SendData: 15 unsubscribe "ifolder/A/B/C"

2624[4c02a10]: nsmail-2:A:CreateNewLineFromSocket: 15 OK Completed

2980[4d68730]: nsmail-2:NA:CreateNewLineFromSocket: * OK IMAP4 
service (Netscape Messaging Server 4.15 Patch 4 (built Dec  7 2000))

2980[4d68730]: nsmail-2:NA:SendData: 1 authenticate plain

Comment 40

17 years ago
Cc to taka for the server problem.

Comment 41

17 years ago
Navin, did you look in the subscribe UI and see if your subscribe was
successful? It looks to me from the log that it failed.

Comment 42

17 years ago
From comment 35:

2512[4ffa200]: nsmail-2:A:SendData: 11 subscribe

This appears to have been garbled by the client.  Shouldn't it instead be:

11 subscribe "t&AOA-m&APw-w-/t&-AOA-m&-APw-1/t&-AOA-m&APw-2"

to match what the server reported in the previous LSUB response?

Comment 43

17 years ago
yes, subscribe UI failed. Good point, jgmyers. 

Comment 44

17 years ago
a point that I tried to make twice :-(

Comment 45

17 years ago
The difference was so subtle I couldn't make out (until it was explicitly 
pointed out). Sorry about that!

patch upcoming.

Comment 46

17 years ago
sorry, I should have been more clear - the subscribe should not fail, and the
reason it failed was that we were subscribing to a folder that didn't exist
because the name was wrong - but that imap mod-utf7 is a bear to read.

Comment 47

17 years ago
The cause of garbled folder name could be in charset convertor.
Just my guess.

Comment 48

17 years ago
Posted patch proposed fixSplinter Review
There are two parts to the fix. One is to pass unicode string converted from
mod-utf7 to AddSubfolderWithPath. The next part is in imapProtocol code where
the server is giving us names in modutf7 so we don't need to convert the server
name again to mod-utf7. That was causing subscribe to fail.

Cavin, David, Can I get reviews ? thx.
Attachment #89090 - Attachment is obsolete: true
Attachment #89136 - Attachment is obsolete: true
Attachment #89313 - Attachment is obsolete: true
Attachment #89354 - Attachment is obsolete: true

Comment 49

17 years ago
Comment on attachment 89444 [details] [diff] [review]
proposed fix

sr=bienvenu - however, this does require a lot of testing of mod-utf7 folder
names (e.g., A&B), creating, renaming, deleting, subscribe/unsubscribing, etc.
Attachment #89444 - Flags: superreview+

Comment 50

17 years ago
This code change will just affect renaming/deleting/moving imap folders and 
that is all it would affect. I have tested these three, looks ok. 

Comment 51

17 years ago
*** Bug 154446 has been marked as a duplicate of this bug. ***

Comment 52

17 years ago
Comment on attachment 89444 [details] [diff] [review]
proposed fix

Attachment #89444 - Flags: review+

Comment 53

17 years ago
If I am reading comments correctly, there is a bug in charset
convertor from Modified UTF-7 to UCS-2.  We should double check
the convertor works okay and open a new bug against it if it doesn't.


Comment 54

17 years ago
fixed on trunk. 
Closed: 17 years ago17 years ago
Resolution: --- → FIXED

Comment 55

17 years ago
yes, it does seem like the round trip might not quite be working from imap
mod-utf7 -> ucs2 -> imap mod-utf7, unless somewhere we're just converting to
utf7 and not imap mod-utf7

Comment 56

17 years ago
we were passing modutf7 to CreateUtf7ConvertedString when it was expecting 8 
bit ascii, so I think it is ok now unless we have some other places doing the 
same thing. 

Comment 57

17 years ago
Verified on 0628 trunk build.


17 years ago
adt1.0.1+ (on ADT's behalf) approval for checkin to the 1.0 branch. pls check
this in asap, then add the "fixed1.0.1" keyword.
Keywords: adt1.0.1adt1.0.1+
Whiteboard: [adt1 rtm] [ETA 06/27] → [adt1 rtm] [ETA 06/28]

Comment 59

17 years ago
Comment on attachment 89444 [details] [diff] [review]
proposed fix

a=chofmann for 1.0.1 add the fixed1.0.1 keyword after checking in to the
Attachment #89444 - Flags: approval+


17 years ago

Comment 60

17 years ago
fixed on branch.

Comment 61

17 years ago
Verified on branch build.
Keywords: verified1.0.1
Product: MailNews → Core
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.