115091 - Mail subfolder(IMAP) can not have a # sign in the name.

Status: RESOLVED FIXED

(MailNews Core :: Backend, defect)

Description

[Sam Rogers]

When a # sign is use in the name of a mail folder it freezes/locks the contents
of the folder. You can open the folder and see the subject srings and usually
message folder information. But when you open the messeage if comes up blank as
if the message had been zeroed out. When you remove the # sign from the folder
name you can then open and veiw all messages. I sell comic books on eBay and use
mail folders to track each book. I need to be able to have a subfolder name SW
Marvel #1 to denote the issue number of that book.

Comment 1

[Dimitrios]

Confirming on 2001121303, Win98. Create a local folder "folder #1" and move some
messages in it. Try to preview or view any of these messages. Their content is
blank as reporter described. Doesn't seem to be a dupe of bug 94124, here the
folder doesn't start with a "#". Platform->All.

Comment 2

[Dimitrios]

JS console, when viewing a message of this folder on standalone window:
"Error: popup has no properties
Source File: chrome://editor/content/editorApplicationOverlay.js
Line: 82"

Comment 3

[scottputterman]

I'm going to mark this a dup of 94124 and change the summary for that one.

*** This bug has been marked as a duplicate of 94124 ***

Comment 4

[Sam Rogers]

I am marking this bug to be reopened. This is not the same as bug 94124 as when
you restart mozilla and the mail window you can see the folder (test #1) under
the parent folder, but as before you still can not do anything with it. Mr.
Putterman I don't think ran a simple test to see if they were the same. Bugzilla
should have a way to group similar bugs together as in a parent child sort of
thing. I can see where 94124 and this bug look similar, but if I were a betting
man I would say that the code will be different. If you mark this bug a dup and
only 94124 gets fixed then you still have this bug floating out there. I would
say fix 94124 first but test that 115091 is fixed or not before marking as dup.

Comment 5

[scottputterman]

A folder with # anywhere in the name including the beginning exhibits this behavior.

Comment 6

[scottputterman]

rassigning to cavin.

Comment 7

[esther]

Marking this bug dependent on 94124 per comment by reporter to make sure this
gets tested when 94124 is fixed. Note: 94124 is nsbeta1+ also.  Also note:
comments in bug 94124 mention that "#" in any part of the folder name is a
problerm referening this bug.  

Comment 8

[Dirk O. Brückner]

Cant confirm it use 2002021203/win2k with an Imap over Ssl Server (University
Mainz/Germany).
Can create folder. move mail into folder , see mail in the folder and move them
back to somewhere else .. but have problems deleting the mail .. but this MUST NOT
be a bug in mozilla ..

Comment 9

[Dimitrios]

Dirk, one of the problems in this bug is that you can't delete messages in that
folder (you confirmed that). You can only delete the folder itself. Still seeing
the bug on 2002021203.

Comment 10

[Michael Buckland]

Discussed at 2/19/02 bug meeting w/ Eng/PjctMg/Marketing: decided to move to
1.2, and change nsbeta+ to nsbeta-

Comment 11

[Michael Buckland]

Fixing dependency which I mis-entered on 2/20.

Comment 12

[Daniel Wang]

*** Bug 199648 has been marked as a duplicate of this bug. ***

Comment 13

[Jo Hermans]

*** Bug 204738 has been marked as a duplicate of this bug. ***

Comment 14

[Sam Rogers]

Has this been fixed?? I see not comments to state other wise. Cavin are you
still working on this. This bug was filed in 2001. Why is that it take over two
and half years to fix this bug??

Sam Rogers - orginal author of this bug.

Comment 15

[Jo Hermans]

*** Bug 217216 has been marked as a duplicate of this bug. ***

Comment 16

[Jo Hermans]

*** Bug 217441 has been marked as a duplicate of this bug. ***

Comment 17

[Scott MacGregor]

*** Bug 215017 has been marked as a duplicate of this bug. ***

Comment 18

[Matthias Versen [:Matti]]

*** Bug 227379 has been marked as a duplicate of this bug. ***

Comment 19

[Malte Rücker (mbr)]

*** Bug 228691 has been marked as a duplicate of this bug. ***

Comment 20

[Paul Tomlin]

Still appears to be bug in Thunderbird 0.8 (20040913)/Windows

I had just created a load of subfolders with # as the first char.
I could put mail into the folder but couldn't read it (empty window).

I'm guessing this is an RDF URI issue, being that the # symbol is an anchor and
all :-)

Comment 21

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #4)
> I am marking this bug to be reopened. This is not the same as bug 94124 as when
> you restart mozilla and the mail window you can see the folder (test #1) under
> the parent folder, but as before you still can not do anything with it.

Sam Rogers, your problem(problem which is different from bug 94124) still exists?
Or your problem became same phenomenon as bug bug 94124?
Or "#" related problem has been resolved?
   
As I wrote Comment #19 bug 94124, "#" related problem seems to be fixed by patch
for Bug 262018.
(I can't imagine other than patch for Bug 262018.)
Since the patch was checked-in on 2004-11-18, please be careful for build number
in problem recreation test and fix verification test.
 

Comment 22

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #20)
Paul Tomlin, your case(RDF URI issue) is probably Bug 262018, which is already
fixed as I said in comment #19.

Comment 23

[Sam Rogers]

(In reply to comment #21)
> (In reply to comment #4)
> > I am marking this bug to be reopened. This is not the same as bug 94124 as when
> > you restart mozilla and the mail window you can see the folder (test #1) under
> > the parent folder, but as before you still can not do anything with it.
> 
> Sam Rogers, your problem(problem which is different from bug 94124) still exists?
> Or your problem became same phenomenon as bug bug 94124?
> Or "#" related problem has been resolved?
>    
> As I wrote Comment #19 bug 94124, "#" related problem seems to be fixed by patch
> for Bug 262018.
> (I can't imagine other than patch for Bug 262018.)
> Since the patch was checked-in on 2004-11-18, please be careful for build number
> in problem recreation test and fix verification test.
>  

Has someone tested it in Unix land. This bug is 3 years old. I gave up using
Imap because at the time I need this to be fixed. So at the present I do not
have Imap up and running on my server or a Imap e-mail account. I might be able
to do the later but not until after Christmas and the New Year. What I would
think a fix to this would be, would be not only check for # exceptions but for
all non-alphanumeric cases. This would include begining, middle and end of a
folder name, as I have seen that there are other folder name bug issues out
there. But that is not a requirement to fix this bug. One does however have to
test for # on all OS's to make sure it is fixed to resolve this bug. I run on
primaryly on Linux (I also use a Mac X, but no stink'n PC) so someone had better
make sure they test on it.

So I will repeat one last time this bug can not be resolved until it has be
tested for # at the beginning of a folder name. A good programer and/or tester
would however test for # anywhere in the name.

Sam Rogers.

P.S.
   I do not understand why it has taken so long to try and fix what looks to me
as a simple bug. Cavin I don't know what you've been busy with, but you've let
this slide. Personally it looks to me like you got **** at me for not letting
you close this per another open case. But it could just look that way and I
could be wrong.

Comment 24

[WADA:World Anti-bad-Duping Agency]

I've understood why folder disappears when bug 94124 but does not disapper when
this bug.
As Bug 273753 Comment #1 says, there is a logic for starting "#" in addition to
 starting "."/ending "~" in /mailnews/local/src/nsLocalMailFolder.cpp.
> 193 nsShouldIgnoreFile(nsString& name)
> 194 {
> 195   PRUnichar firstChar=name.CharAt(0);
> 196   if (firstChar == '.' || firstChar == '#' ||
>           name.CharAt(name.Length() - 1) == '~'  )
> 197     return PR_TRUE;
This is the reason why folder disappears only when starting "#"(bug 94124).
So if this logic was disabled, bug 94124 would become this bug.
( Dependency should have been reverted :-) )

And this bug was resolved by not using "#" in file name (use hashed name
instead), by patch for Bug 262018.
Then bug 94124 won't occur anymore even above logic still exists.

This is true when local file, as I reported to bug 94124 comment #18.
But I cannot know whether probllem when IMAP is resolved or not, because I don't
have IMAP account.

Comment 25

[zug_treno]

*** Bug 320898 has been marked as a duplicate of this bug. ***

Comment 26

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #23)
> This bug is 3 years old. I gave up using Imap because at the time I need this to be fixed.

Sam Rogers, your original and current problem is issue on IMAP only?
(comment #23 is the first comment you mentioned IMAP, and 3 years since you opened this bug...)

Problem when IMAP is different from problem when POP3, including "Mail folder name" related problem, in many situations.
And issues such as Bug 95114 and Bug 84045 are usually involved in problem when "#" in IMAP mail folder.
Please note that bug 94124 is for "LOCAL file name/file path" related issue.

Comment 27

[WADA:World Anti-bad-Duping Agency]

Mail from Sam Rogers(bug opener) to me arrived today. 
> I am sorry I do not remember my bugzilla password and do not want to hassel
> with it. Basically I have given up on Mozilla, but your right I forgot to
> mention in the original description that this was a imap only problem.
> Regards,
>    Sam Rogers.

Removing Bug 94124 from "depends on:".
Adding Bug 95114 and Bug 84045 to "depends on:".

To all comment posters who experienced problem on LOCAL folder name with "#":
To reporters/comment posters of many bugs which were DUPed to this bug:
   See Bug 94124 instead of this bug, IMAP only case, if LOCAL folder case.    

Comment 28

[Magnus Melin [:mkmelin]]

This works fine for me on Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a2pre) Gecko/20070107 Thunderbird/3.0a1 ID:2007010703 on IMAP too. 

->WFM (probably fixed by bug 262018)

Comment 29

[Magnus Melin [:mkmelin]]

Reopening, not sure what I saw in comment 28, but IMAP folders with # doesn't work on trunk (and not branch either).

Tested on Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a9pre) Gecko/2007092904 Thunderbird/3.0a1pre ID:2007092904

Comment 30

[ovidiu]

please consider bug 434850

Comment 32

[Nikolay Shopik]

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b1pre) Gecko/20080927031346 Shredder/3.0b1pre
Still there

Comment 33

[Nikolay Shopik]

Could we have some attention to this bug too as we have kinda similar bug 370951 which is planed to be fixed for b2. This is regression (worked in 2.0) you can have look into my dupe bug 434850 where much discussion aswell.

Comment 34

[David :Bienvenu]

it would be good to figure out what's going on here - this failure implies a problem with our uri escaping for imap folders.

Comment 35

[Nikolay Shopik]

I've just check my acount where folder with "#" symbol names and it appears to be showing w/o problems. Seems somebody do some commits which affects this bug. I will check on clean install if I can reproduce it again.
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b3pre) Gecko/20081210 Shredder/3.0b2pre

Comment 36

[Nikolay Shopik]

Problem is sligly changed. You can subscribe to any folders with "#" sign, but if you have more than 5 folders with this sign, if you select first folder then second and third you appears to be notice what content of forth folder are same as third and fifth.
If you startup TB w/o connection to internet you will see random numbers instead names of folders.

Comment 37

[David :Bienvenu]

from a little debugging, it looks like if I create five folders, folder #1-5, folders #2-5 all have the online name of folder #1, so we treat any command involving them as if they were run on folder #1. Very strange.

Comment 38

[David :Bienvenu]

we are munging the .msf file names for these folders, because we treat '#' as an illegal char in filenames. That's related to the numeric folder names if we start up dis-connected, and it may also explain this bug. We're definitely trying to set the online name

Comment 39

[Nikolay Shopik]

(In reply to comment #37)
> from a little debugging, it looks like if I create five folders, folder #1-5,
> folders #2-5 all have the online name of folder #1, so we treat any command
> involving them as if they were run on folder #1. Very strange.

Exactly, and # is illegal charter(need to be escaped) for POSIX systems on Windows its ok to call folder with such name.

Comment 40

[David :Bienvenu]

Attachment: fix folder #1, folder #2... case

I'm not really sure what's going on here, but I noticed that sometimes the 8 bit char version of NS_MsgHashIfNecessary is used, and sometimes the unicode version. They don't produce the same hashed file names. This was producing inconsistent offline store and .msf file names, and generally producing confusing results. I switched almost all the code to use the unicode version, and that seems to have fixed this issue. The remaining code that uses the 8 bit char version doesn't seem to be called anymore (nsImapMailFolder::CopyFolder) but we can deal with that in a spin-off bug.

Comment 41

[[e-mail]]

Comment on attachment 355501 [details] [diff] [review]
fix folder #1, folder #2... case

Wrong patch?

Comment 42

[David :Bienvenu]

Attachment: right patch

oops, thx, this is the right patch.

Comment 43

[David :Bienvenu]

Comment on attachment 355586 [details] [diff] [review]
right patch

I do need to remove the XXX comment as well.

Comment 44

[[e-mail]]

Comment on attachment 355586 [details] [diff] [review]
right patch

sr=me with this all done in Unicode, i.e.
>+  nsCString nativeName;
>+  NS_CopyUnicodeToNative(proposedDBName, nativeName);
>+  nativeName += SUMMARY_SUFFIX;
>+  dbPath->AppendNative(nativeName);
proposedDBName.AppendLiteral(SUMMARY_SUFFIX);
dbPath->Append(proposedDBName);

>-    dbPath->GetNativeLeafName(proposedDBName);
>+    dbPath->GetNativeLeafName(nativeName);
dbPath->GetLeafName(proposedDBName);

>-  proposedDBName.SetLength(proposedDBName.Length() - NS_LITERAL_CSTRING(SUMMARY_SUFFIX).Length());
Keep this (but possibly change the length calculation).

>-  dbPath->SetNativeLeafName(proposedDBName);
>+  dbPath->SetNativeLeafName(nativeName);
dbPath->SetLeafName(proposedDBName);

>-        // the parentName might be too long or have some illegal chars
>-        // so we make it safe.
>-        // XXX Here it's assumed that IMAP folder names are stored locally
>-        // in modified UTF-7 (ASCII-only) as is stored remotely.  If we ever change
>-        // this, we have to work with nsString instead of nsCString
>-        // (ref. bug 264071)
>-        nsCAutoString safeParentName;
>-        LossyCopyUTF16toASCII(parentName, safeParentName);
>-        NS_MsgHashIfNecessary(safeParentName);
>-        path->AppendNative(safeParentName);
>-
>-        // path is an out parameter to CreateDirectoryForFolder - I'm not
>-        // sure why we're munging it above.
The reindentation hides the fact that you're just deleting this section...

>   nsCAutoString proposedDBName;
>   LossyCopyUTF16toASCII(newLeafName, proposedDBName);
>   // warning, path will be changed
>-  rv = CreateFileForDB(proposedDBName, pathFile, getter_AddRefs(dbFile));
>+  rv = CreateFileForDB(newLeafName, pathFile, getter_AddRefs(dbFile));
Does proposedDBName still get (usefully) used?

Comment 45

[David :Bienvenu]

Attachment: fix addressing Neil's comments

address Neil's comments, carrying forward sr, requesting r from Standard8.

Comment 46

[Mark Banner (:standard8)]

Comment on attachment 355611 [details] [diff] [review]
fix addressing Neil's comments

Something still isn't right, at least with my courier imap server.

STR:

1) Create two folders on imap server not using TB, named "testhash#1" and "testhash#2"
2) Start up TB, look in profile directory
=> there are "testhash.msf", "testhash03db7f9d.msf" and "testhash67a1bc76.msf" files.
3) Move one message into each folder
=> Selecting folders lists both messages in both folders.
=> On disk, now have a single "testhash" file containing both messages.

>+  NS_ConvertASCIItoUTF16 leafName(folderName);
>+  nsAutoString folderNameStr;
>+  nsAutoString parentName = leafName;
>+  // use RFind, because folder can start with a delimiter and
>+  // not be a leaf folder.
>+  PRInt32 folderStart = leafName.RFindChar('/');
>+  if (folderStart > 0)
>+  {
>+      nsCOMPtr<nsIRDFService> rdf(do_GetService(kRDFServiceCID, &rv));
>+      NS_ENSURE_SUCCESS(rv, rv);

nit: as you're touching this section anyway, you may as well make it 2-space indented

>+      nsCOMPtr<nsIRDFResource> res;
>+      nsCOMPtr<nsIMsgImapMailFolder> parentFolder;
>+      nsCAutoString uri (mURI);
>+      parentName.Right(leafName, leafName.Length() - folderStart - 1);
>+      parentName.Truncate(folderStart);
>+
>+      rv = CreateDirectoryForFolder(getter_AddRefs(path));
>+      if (NS_FAILED(rv)) return rv;

nit: again as you're touching these lines, please put the return on the next line with a blank line afterwards (2 places).

Comment 47

[David :Bienvenu]

Attachment: fix for case where folder created on different machine

this should fix the case where the folders are created on a different machine. What was happening was that even though we'd set the file path, ParseUri was crunching the path we'd already set up. So the only difference between this patch and the previous one is that we don't allow ParseUri to crunch an existing mPath.

You'll probably need to go through the whole exercise again, because once we've created the bad .msf file, or put a bad entry in panacea.dat, it's difficult to recover.

Comment 48

[Mark Banner (:standard8)]

Comment on attachment 361004 [details] [diff] [review]
fix for case where folder created on different machine

That's better. r=me with the comments in comment 46 addressed.

Comment 49

[David :Bienvenu]

fix checked in, with comments addressed.