379101 - Mail data and sub-folders are lost, if folder has ending period(dot) or trailing space in its name, when upgrade to Tb 2.0 from Tb 1.5 or former(Linux/Mac)

Status: NEW

(MailNews Core :: Database, defect)

Description

[Colin Lau]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1

I was upgrading from SeaMonkey 1.1 to 1.1.1 and noticed that some of the local mail sub-folders are not showing up.  This affects folders whose parent folders have multiple periods in their names (e.g. "A.B.C.").

Reproducible: Always

Steps to Reproduce:
1. In the Mail component of SeaMonkey 1.1, create a top-level folder (e.g. "Work")
2. Create a second-level folder with several periods (e.g. "A.B.C.")
3. Create some third-level folders (e.g. "Sub-folder1", "Sub-folder2", etc.)
4. Open the same profile using the Mail component of SeaMonkey 1.1.1
Actual Results:  
When opening the profile using SeaMonkey 1.1, the following result is obtained:
---Local Folders
 |-Work
  |-A.B.C.
   |-Sub-folder1
   |-Sub-folder2
  |-Other second-level folders

When opening the same profile using SeaMonkey 1.1.1, the following appears instead:
---Local Folders
 |-Work
  |-A.B.C. (the folder is empty)
  |-Other second-level folders

Expected Results:  
All sub-folders should appear in the navigation tree regardless of the parent folder's name.

Comment 1

[Andrew Schultz]

This regressed between 1.8 branch builds of seamonkey between 2007-01-17 and 2007-01-25, so bug 229522

If I just create an A.B.C. folder with 1.1, I get the following in my Work.sbd directory:

A.B.C.
A.B.C..msf

If I then bring up a later build with the bug, I get:

A.B.C.
A.B.Ce15fb51d
A.B.Ce15fb51d.msf
A.B.C..msf

Comment 2

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #1)
> A.B.Ce15fb51d
Caused by a part of patch for Bug 229522, a part for problem of Bug 117840(ending dot issue on Win).
"Ending dot" case should have been limited to MS Win.
Next is the quickest approach?
 - Remove "ending dot" and ship it with Tb 2.0.0.1, and re-open Bug 117840.  
   RCS file: /cvsroot/mozilla/mailnews/base/util/nsMsgUtils.cpp
   -#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER  ".~"
   +#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER  "~"

Workaround :
 1) Delete garbage of A.B.Ce15fb51d / A.B.Ce15fb51d.msf
 2) Rename A.B.C. / A.B.C..msf  to A.B.C_ / A.B.C_.msf, A.B.C / A.B.C.msf etc.
 3) Restart Thunderbird, and click the folder in order to display correct count

Comment 3

[WADA:World Anti-bad-Duping Agency]

To Andrew Schultz : Do you know result when Mac OS X? I guess that Mac OS creates file of "ending dot" normally too then this regression can occur on Mac OS too.   

Comment 4

[Andrew Schultz]

No, I don't have Mac OS X.

Comment 5

[Hiro]

In Mac OS X, it is made as follows. 
   Folder Name: A.B.C.
   File name made in Profile:
     A.B.Ce6bad49d
     A.B.Ce6bad49d.msf

Therefore, there is no problem. 

Mac OS X 10.3.9
version 2.0.0.4pre (20070428)

Comment 6

[Andrew Schultz]

Erm, that sounds exactly like linux... the problem is that old builds behaved differently.

Comment 7

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #5)
> Mac OS X 10.3.9 version 2.0.0.4pre (20070428)
Thanks for test with Tb 2.0 on Mac OS X.
Hiro, can you test with Tb 1,5.x or before? I'd like to know behavior before this regression when Mac OS X. 

Comment 8

[Hiro]

(In reply to comment #7)
> Hiro, can you test with Tb 1,5.x or before? I'd like to know behavior before
> this regression when Mac OS X. 
> 
In Tb 1.5.10 Release, the file was made as follows. 
(There is no problem in the access to this folder. )
  A.B.C.
  A.B.C..msf

And SeaMonkey1.1.1 results the same as comment#5. 
Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2) Gecko/20070221 SeaMonkey/1.1.1

Comment 9

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #8)
>   A.B.C.
>   A.B.C..msf
> Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2)
Thanks for quick test.

To Andrew Schultz and hiro :
When mails are held in "A.B.C."(created by 1.5 and mails are saved), I think Tb 2.0 can use this file as folder of "A.B.C." normally. And I think subfolder (A.B.C..sbd/subfolder-N) access problem only, if "rebuild index" is not involved. 
Is it true when Linux and Mac OS X? (sorry but I can test with Win only)

Comment 10

[Hiro]

>Is it true when Linux and Mac OS X? (sorry but I can test with Win only)
It is a case with OS X. 

It made it from Tb 1.5.10 as follows. 
    |
   +-A.B.C.
         |
         +--sub1
And the message save to "A.B.C." and "sub1". 

When this profile was used with 1.8branch Tb, the profile was as follows. 
-rw-------   1 sek  sek   750  5  5 06:47 A.B.C.
-rw-r--r--   1 sek  sek  2154  5  5 06:47 A.B.C..msf
drwxr-xr-x   4 sek  sek   136  5  5 06:47 A.B.C..sbd/
-rw-r--r--   1 sek  sek     0  5  5 06:49 A.B.Ce6bad49d
-rw-r--r--   1 sek  sek  1095  5  5 06:49 A.B.Ce6bad49d.msf

So, the message is not seen though there is "A.B.C." in the folder list. 
There is no change even if "Rebuild index" is executed. 
Of course, it cannot access subfolder. 

And  two "A.B.C." appears when 1.8branch Tb is re-started. 
In this case, it cannot access the message and the subfolder. 

If the work around of comment#2 is not executed, the message cannot be seen with 1.8branch Tb. 

Comment 11

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #10)
Thanks for clear test result. I now understand difference between "|-A.B.C." and "|-A.B.C. (the folder is empty)" in comment #0.
"Rebuild Index" process looks to be invoked during restart. It seems that I expected too much... 

I think there is two ways for ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER  "." issue:
 1)MS Win only "ending dot" for compatibility on Linux & Mac OS X.
   (When MS Win, no problem because folder of "ending dot" was impossible.)
   In this case, compatibilty between Linux/Mac and Win is lost,
   then several users will complain when they will move mail folders.
 2)Keep current patch for compatibility among OS's and for ease of maintenance.
   This means design change, then action for incompatibility is required.
   e.g. Explanation in release notes, automatic file name change logic.
And I feel 1) is better. 

Comment 12

[WADA:World Anti-bad-Duping Agency]

To David Bienvenu : What do you think about best way?

Comment 13

[WADA:World Anti-bad-Duping Agency]

Bug 303729 has added "trailing space" case(fixed on 2007-03-05).
> -#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER  ".~"
> +#define ILLEGAL_FOLDER_CHARS_AS_LAST_LETTER  ".~ "
(Sorry but I don't know whether included in tb 2.0.0.0 or not) 

To Andrew Schultz and hiro :
I guess "trailing space" has to be "MS Win Only" too. Is it right?

Comment 14

[Andrew Schultz]

Yes, space is a valid last character for linux (and I'd expect for Mac too).

Comment 15

[Hiro]

(In reply to comment #14)
> Yes, space is a valid last character for linux (and I'd expect for Mac too).
> 

Yes, it is the same also in Mac. 

Comment 16

[Peter Trenholme]

I just upgraded from Tb 1.5.10.10 to Tb 2.0.0.0 on a Fedora Linux system, and one of my folders (named "Receipts, Invoices, etc.") was shown in the folder list, under that name, twice. (I.e., a duplicated folder name.) Both folders were shown as empty.  Tihis was a sub-folder of the "Save" folder. When I when to the Mail subdirectory of the profile, in the Save.sbd directory, I saw two set of files: "Receipts, Invoices, etc7476509a.msf" and "Receipts, Invoices, etc..msf"

Deleting the "Receipts, Invoices, etc7476509a" files and renaming the "Receipts, Invoices, etc.." file and sbd directory to have a single dot fixed the problem. (Of course I lost the period at the end of the "etc.", but I can live with that.)

Still, the upgrade from 1.5 to 2.0 should do a better job of not loosing mail folders. Or, at a minimum, upgraders should be warned to rename their folders prior to the upgrade.

Comment 19

[Henrik Skupin [:whimboo][⌚️UTC+2]]

David, shouldn't this be a blocker for bug 378635?

Comment 21

[Elliott G]

I've tried to rename the folders and managed to delete ONE of the two new duplicate gibberish-name folders.  However, even after I have but ONE folder, and then RENAME it without any possible-offending characters, after I quit the program and then re-open it, the ORIGINAL bad folder returns, now with its ORIGINAL name: "!!Dana etc."  Note that the name I gave it starts with two exclamation points, has a space between the words, and ends with a period.  But, again, after I rename that folder to something without spaces or odd punctuation marks, it comes back after I reopen the program.

HELLLLLP!!!

Comment 23

[Henrik Skupin [:whimboo][⌚️UTC+2]]

(In reply to comment #21)
> But, again, after I rename that folder to something without spaces or odd
> punctuation marks, it comes back after I reopen the program.

This is not a user support forum but lets give a little help. You should rename this folder or MBOX file outside of Thunderbird. Close the application, rename the appropriate files/folders and delete the .msf file. After the next restart Thunderbird should come up with the new name and creates a new summary file.

Comment 24

[David Ascher (:davida)]

Given the age of 1.5, I think it's too late to consider this a blocking bug.

Comment 26

[bozz]

for what it's worth, I just experienced the "ending with period" subfolder problems in Mac OSX 10.4.11 whilst migrating from Netscape 7.2 to Thunderbird 2.0.0.19 (20081209). Ended up in a "duplicate report" [Bug 475169] because I wouldn't have thought to search the problem in anything prior to Tb 2.0.0.19...