378299 - Duplicate folders magically created in Local Folders

Status: RESOLVED INCOMPLETE

(Thunderbird :: General, defect)

Description

[Ian Graham]

Attachment: Graphic showing duplicated Folders in Local Folder list

Somehow Thunderbird has duplicated one of my Folders (see attached graphics) - the Folder that was originally in Local Folder/Personal/Subscriptions was duplicated / copied into Local Folder/Personal/jokes/Subscriptions. 
All messages were copied over also. 

Thunderbird version 1.5.0.10 (20070221) on WIndows XP SP2.

[... time thinking and looking around .... ]

Ahh, I now have a tentative diagnosis based on monitoring what happens in the ..Application Data\Thunderbird\Profiles\xxxx.default\Mail\Local Folders\ directories. 

If I move a mail folder (all Local Folders), for example

Personal/Subs --> Personal/jokes/Subs

Then the files 'Subs.msf' and 'Subs' are copied over to the target directory, but only the file 'Subs.msf' is deleted from the original location.

Now when I shut down and restart Thunderbird, TB automatically regenerates a "Subs.msf" file in the original source directory, and populates the mail Folder view with a 'Subs' folder in this original location.

I can't find any bug report for this -- but it seems so weird, someone else must've reported it. Or is there something weird in my TB instance to cause this?

Comment 1

[WADA:World Anti-bad-Duping Agency]

> Personal/Subs --> Personal/jokes/Subs
(Q1) Usual local mail folder? Or search folder(virtual folder)?
(Q2) Does Personal/Subs mean folder of Personal and subfolder of Subs?
     Or search folder named "Personal/Subs"?
If search folder and named Personal/Subs or Personal/jokes/Subs, see Bug 286523. 

Comment 2

[Ian Graham]

1. This is a standard local folder -- not a search or virtual folder.
2. I am using / to denote subfolders (unix notation) so Personal/Subs
   means Subs is a subfolder within Pesonal which is in turn a folder
   in Local Folders.

Comment 3

[Ian Graham]

So to further clarify, when I say
> Personal/Subs --> Personal/jokes/Subs
I mean that I am taking the Folder Subs, which was a subfolder of Personal, nad moving it to be a subfolder of jokes which is in turn a subfolder of Personal.

Comment 4

[Magnus Melin [:mkmelin]]

Is this reproducible for you? Have you tried thunderbird 2.0? 
(.msf files are just fast indexes, so regeneration is expected)

Comment 5

[Ian Graham]

Yes I understand about the .msf files being regenerated -- but if I move a Folder, I would expect the mailbox file to be deleted from the original location. In my case the 'move Folder' function is cloning the mailbox file, leaving two copies of it - one at the original location, and one at the location corresponding to where the Folder was moved to. 

However, I did a bunch more testing, and after being able to reproduce this problem several times over two days (TB restarts, computer reboots), the problem vanished just as suddenly as it appeared....

I am at a total loss.... 

I will mark as worksforme, and will reopen if it comes back ...

Comment 6

[WADA:World Anti-bad-Duping Agency]

Possible scenario. (/ denotes folder hierarchy as you did).
  1. Open mail file for Personal/Subs by text editor
     File of ...\Personal.sbd\Subs is locked by text editor. 
  2. Move Personal/Subs --> Personal/jokes/Subs
  3. Move completes without error.
     However, ...\Personal.sbd\Subs can not be deleted due to file lock,
     then ...\Personal.sbd\Subs remains (Subs.msf is deleted).
This phenomenon can be easily be re-created.

Comment 7

[Ian Graham]

Yes, that makes sense -- but why would the file be locked? And why would the problem seem to go away?   

Comment 8

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #7)
> why would the file be locked?
By text editor? If so, text editor is for update file, then "write lock" is obtained thru "write/update open" by text editor. 
In your environment, "read open" by other program such as backup software is a suspect who inhibits delete by Thunderbird.

> And why would the problem seem to go away?   
After step 3 in comment #6.
 4. Close text editor => write lock is released.
 5. Shutdown & restart Thunderbrd => ...\Personal.sbd\Subs.msf is recreated
    => Personal/Subs appears again
 6. Delete Personal/Subs or move Personal/Subs to somewhere
    ...\Personal.sbd\Subs.msf is deleted because no one locks/opens it.
    => No problem since this action

What did you do during "I did a bunch more testing"?
Please be careful in testing - don't interfere with software's activity.  

Comment 9

[Ian Graham]

By 'testing' I mean I was monitoring, using the windows file / folder browser (or using cygwin's ls command) the presence or absence of the indicated files before and after I moved a folder.  

At no time did I use an editor or other tool to touch the files.  

As far as I know the Windows file / directory browser does not lock files.....

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #9)
> By 'testing' I mean I was monitoring, using the windows file / folder browser
>(snip)
> As far as I know the Windows file / directory browser does not lock files...

Isn't file viewer(or some softwares who opens file) involved in your testing? 
Please note that; "Read open" of a file acquires "read lock", then write / update / delete / move of the file becomes impossible, if the file is "read-open'ed but not-closed yet" by other program.
Isn't it your case?
  (I opened prefs.js by text editor many times while testing, then prefs.js)
  (update by Mozilla/SeaMonkey/Firefox/Thunderbird failed many many times. )
  (This is still occurring, because I'm not so careful. :-)                )

Comment 11

[Ian Graham]

By monitoring files I mean that I used commands like unix's 'ls' (to list files in a folder), or the Windows explorer (which is what I meant by "Windows file/ directory browser") to view a list of files in a given folder. 

I can guarantee that at no time did I use a tool that would open, edit, touch or otherwise manipulate the files, or lock the file. 

Comment 12

[Magnus Melin [:mkmelin]]

I think windows may lock a directory if you have cd:ed to that dir in command prompt / cygwin bash. 

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #11)
> I can guarantee that at no time did I use a tool that would open, edit, touch
> or otherwise manipulate the files, or lock the file. 
I see.
Lets talk about other case than lock of directory mentioned by Magnus Melin.
Thunderbird himself(herself?) can be a suspect.

Thunderbird will never release lock of ".msf" file once the mail folder is opened, but closes and releases lock for mail folder file after use.
Please observe this by next steps before think about suspect.
 1. Start Thunderbird, and open x and x.msf files by text editor.
    => Text editor can open both files because they are not opend at this step.
 2. Click folder x and see a mail, and open x and x.msf files by text editor.
    => Unable to open both x and x.msf with write/update.
 3. Click other folder (folder of different account is better) and see a mail,
    and open x and x.msf files by text editor.
    => Unable to open x.msf but able to open x with write/update.

Since Thunderbird has multi-task capability, other process/task such as Automatic mail download, Filtering, Junk purge runs concurrently.
If such process/task hung or incorrectly worked,
   ( by bug many times, sometimes by inadequate setups by user, )
   ( sometimes by not-so-well="Tb dislikes/hates" operation     )
file lock may be kept by the other process/task in Tb. In such case, phenomenon you experienced can occur.

Above tends to occur during "bunch more testing". And above is not able to recreate usually because tester usually doesn't remember exact operations during test. And even if exact operations are remembered, completely same condition before test is unable to re-generate many many times.
Please be very careful when test software.

Comment 14

[Ian Graham]

I was very careful to use 'ls' or windows explorer to view only at the level 'above' that of the folder/directory that was moved. Thus even if ls or windows  explorer locked the folder being viewed, this does not affect folders below that. 

It is possible that TB has problems with it's own internal semaphores for file locks - but I can't diagnose or easily reproduce that. I will note, though, that the problems happened long after I open a session with my mail server, and that I do not receive a big volume of mail, so it is unlikely that TB file locking caused the  problem. 

Comment 15

[WADA:World Anti-bad-Duping Agency]

> I was very careful
Oh, i's good. I'm not so careful as I wrote, so I saw many prefs-N.js'es :-) 

Another possibility: Similar problem to Bug 368112.
See Bug 368112 Comment #6,  Bug 368112 Comment #9, Bug 368112 Comment #10.

Folder move process consists of multiple steps;
 Copy of Subs, Rebuild of Sub.msf, Delete of old .msf, Delete of old Subs,... 
During mail folder move, especially during move by drag&drop, similar condition to Bug 368112 maybe occur. If such condition occurred, folder move process may be interrupted. In this case, Thunderbird maybe fail to delete old Subs or skip delete of old Subs.

Anyway, I'm not suspecting your fault. "Open by text editor or some other" is only an example of inhibitor of TB's file deletion. If you'll find clear condition/recreation procedure, please open new bug with sufficient data for developers to analyze problem.

Comment 16

[WADA:World Anti-bad-Duping Agency]

WORRKSFORME means "something was wrong in Tb, but looks to be already resolved by unknown fix". But this bug's case is different ; Since problem occurred in your environment, something wrong apparently exists(or existed) in Tb or your environment, but it's very difficult(impossible for me) to re-create problem and analyze problem, although status and phenomenon after problem is very very clear.
For this kind of bug report, INCOMPLETE has been introduced, and I think INCOMPLETE is appropriate for this bug.

Changing to INCOMPLETE.