Closed Bug 439333 Opened 16 years ago Closed 5 years ago

Index (summary file) corruption while manually moving mail from inbox to folder

Categories

(Thunderbird :: Folder and Message Lists, defect)

Product:
Thunderbird
Component:
Folder and Message Lists
Platform:
x86
Windows XP
Type:
defect
Priority:
Not set
Severity:
critical

Tracking

(Not tracked)

Status:
RESOLVED WORKSFORME

People

(Reporter: mozilla, Unassigned)

References

(Depends on 1 open bug)

Details

(Whiteboard: [datalossy]) 

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.14) Gecko/20080404 Firefox/2.0.0.14
Build Identifier: 2.0.0.14 (20080421)

Sometimes while receiving mails if a one of them is manually moved to another folder the folder's summary file gets "corrupted" and the mail is hidden.

Need to "Rebuild Index" for the folder to see the mail again.

Reproducible: Sometimes

Steps to Reproduce:
1. Execute "Get all new messages" 
2. While executing, if a mail is received move it to another folder from inbox.
3. 
Actual Results:  
Sometimes the mail is hidden.

Expected Results:  
Mail hidden

Some settings I think could be involved:
1. "Compact folders when it will save ### KB" is on and set to 100 KB
2. "Allow anti-virus clients to quarantine individual incoming messages" is on.

I noticed the problem when moving mail from Inbox to folders which include non alphanumeric characters (ie. # / and +) and folder filenames seem to be encoded.
Examples: folder named ".NET" filename is 25564e66; folder named "C#" filename is C62d93f6b.
Could be related to bug: https://bugzilla.mozilla.org/show_bug.cgi?id=413519
Version: unspecified → 2.0
I have the similar story.
Avast is installed and "Allow anti-virus clients to quarantine individual incoming messages" is on. There was a message under suspicion, it was allowed to proceed with them, message came into Inbox and then manually moved to subfolder. One of the next messages, came into Inbox, after trying to move it to another subfolder - has gone.
We definitely have index corruption problems somewhere. I have a number of permanent saved searches defined which relate to messages coming in to one of my inboxes, and find myself having to rebuild the index on that Inbox multiple times per session. On the other inbox, I have no saved searches defined which affect it, and the problem is much less prominent (although I'm not sure if it's non-existent). Note that the searches do not _move_ messages.

I think this is the difference. I also have some filters set up which filter messages with certain spam headers into a spam folder, so perhaps it could be those causing the problem. Although I have those for both inboxes.

Gerv
Status: UNCONFIRMED → NEW
Ever confirmed: true
I have switched over seamonkey (from old mozilla 1.7.13) and when I check the mail the inbox ahows no mail, but if i wait awhile all the mail will appear, apparently it is re-indexing this happens during the same session if I go to the browser and then after an hour or so check mail again it is empty and then it re-indexes again.
I doubt antivirus is at fault in all your cases. see bug 383599 and bug 480938.

I've seen this also when drag and drop from a virtual folder was in progress and I received a new message before the message moves completed.
(In reply to comment #0)
> Some settings I think could be involved:
> 1. "Compact folders when it will save ### KB" is on and set to 100 KB

Check whether auto-compact is one of culprits or not first.
1. Change mail.purge.ask=false to mail.purge.ask=true (reset it via Config Editor)
2. When dialog before start of auto-compact appeared when you moved many mails,
   reply "No" to the dialog.
   Never check "[ ] Do this automatically from now on" at the dialog.
   If you cheked, do step 1 again, and restart Tb.
(Q1) Does your problem occur even when "No" to dialog?
3. If dialog appeared at step 2, execute File/"Compact Folders" for each account.
4. Because small threshold(defaulted 100KB), dialog will be issued frequently, and it's annoying. Increase the value for your daily use.
5. Repeat 2-4 in daily use.
(Q2) Does your problem still occur frequently?

> Examples: folder named ".NET" filename is 25564e66; folder named "C#" filename
is C62d93f6b.

It's result of hashing for illegal file name characters such as #, +. See bug 275770.
Gerv, Andy, ...

* are you imap or pop?
* do you have notifications for new messages "show an alert" turned on in options?
I'm IMAP. I do not have notifications turned on (either visual or audio).

Gerv
I've changed the mail purge settings as suggested.
I've changed the threshold to 16MB (On a 500GB drive, this is pretty small).

I've not yet seen the Purge dialog, but the problem persists.
I forgot to say, I'm using pop for all my accounts.
And I use an unique local folders root for all my mails.
(In reply to comment #9)
> I've not yet seen the Purge dialog, but the problem persists.

Maximiliano Podostroiec, did you restart Tb after change to mail.purge.ask=true?
(I forgot to write "restart Tb" in step 1. I wrote it in step 2 only...)
Yes I did. Several times since then.
(In reply to comment #8)
> I'm IMAP. (snip)
> Gerv

As bug opener says in comment #10, original problem of this bug is for problem on "local mail folder". Since "communication with server" is involved in IMAP folder case, and since "delete model" relates to IMAP problem, different analysis is required for IMAP case, even if same external symptom(rebuild index is required to see hidden mail) upon same situation(move mails while downloading).
See bug 414723 for "hidden or dup'ed mail" case of IMAP even when no "move mails while downloading".

To Gervase Markham: Open separate bug, attaching IMAP log and referencing to this bug, please.
(In reply to comment #0)
> Steps to Reproduce:
> 1. Execute "Get all new messages" 
> 2. While executing, if a mail is received move it to another folder from inbox.

Question about status of move target folder.
What operation did you do for "move it to another folder"?
(say move mails to XX folder from Inbox folder)
(1) XX is NOT clicked(XX folder is NOT opened) before move to XX.
  (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
  (1-2) "move to XX" is done via "Move" menu.
(2) XXX is clicked(XX folder is opened) before move to XX.
  (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
  (2-2) "move to XX" is done via "Move" menu.
Maximiliano Podostroiec, was error message issued to Error Console when you experienced problem?
(See Bug 493429 for error message when "outdated msf condition" exists.)
(In reply to comment #14)
> (In reply to comment #0)
> > Steps to Reproduce:
> > 1. Execute "Get all new messages" 
> > 2. While executing, if a mail is received move it to another folder from inbox.
> 
> Question about status of move target folder.
> What operation did you do for "move it to another folder"?
> (say move mails to XX folder from Inbox folder)
> (1) XX is NOT clicked(XX folder is NOT opened) before move to XX.
>   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
>   (1-2) "move to XX" is done via "Move" menu.
> (2) XXX is clicked(XX folder is opened) before move to XX.
>   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
>   (2-2) "move to XX" is done via "Move" menu.

1-1.
I manually drag the mail from inbox and drop it into the other folder without activating the other folder.

I can't be sure since I no longer user Message filters, but I think having a message filter move the mail caused the same.


(In reply to comment #15)
No, I dunno why but my Thunderbird's error console is completely clean always.
Depends on: 458570
(In reply to comment #9)
> I've not yet seen the Purge dialog, but the problem persists.

(In reply to comment #16)
> > (1) XX is NOT clicked(XX folder is NOT opened) before move to XX.
> >   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
> >   (1-2) "move to XX" is done via "Move" menu.
> > (2) XXX is clicked(XX folder is opened) before move to XX.
> >   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
> >   (2-2) "move to XX" is done via "Move" menu.
> 
> 1-1.
> I manually drag the mail from inbox and drop it into the other folder without
> activating the other folder.

"No explicit folder open of move target folder" case...
Outdated .msf condition exists?

Maximiliano Podostroiec, can you produce your problem frequently by next operation?
  1. Before move mails, click move target folder(open the folder explicitly).
     If you already selected mails, open separate Tb window, click move target
     folder at the new Tb window, and close the new Tb window.
  2. Execute move mails by Drag&Drop.
(In reply to comment #17)
> (In reply to comment #9)
> > I've not yet seen the Purge dialog, but the problem persists.
> 
> (In reply to comment #16)
> > > (1) XX is NOT clicked(XX folder is NOT opened) before move to XX.
> > >   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
> > >   (1-2) "move to XX" is done via "Move" menu.
> > > (2) XXX is clicked(XX folder is opened) before move to XX.
> > >   (1-1) "move to XX" is done by Drag&Drop of mails to XX of folder pane.
> > >   (2-2) "move to XX" is done via "Move" menu.
> > 
> > 1-1.
> > I manually drag the mail from inbox and drop it into the other folder without
> > activating the other folder.
> 
> "No explicit folder open of move target folder" case...
> Outdated .msf condition exists?
> 
> Maximiliano Podostroiec, can you produce your problem frequently by next
> operation?
>   1. Before move mails, click move target folder(open the folder explicitly).
>      If you already selected mails, open separate Tb window, click move target
>      folder at the new Tb window, and close the new Tb window.
>   2. Execute move mails by Drag&Drop.

I can't really understand your instructions.
WFM on

Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.5pre) Gecko/20100512 Lightning/1.0b2pre Lanikai/3.1pre ID:20100512033804

based on STR on comment #0.

(STR)
1. send myself 2 messages (the second very large) from account A to account B (they use different folders);
2. get all messages;
3. when message 1 is received (and messages 2 is still on dowloand), move with drag and drop message 1 to another account to other folder (inbox, sentbox, draft o custum folder):

No corruption observed.
(In reply to comment #19)
> WFM on
> 
> Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.5pre) Gecko/20100512
> Lightning/1.0b2pre Lanikai/3.1pre ID:20100512033804
> 
> based on STR on comment #0.
> 
> (STR)
> 1. send myself 2 messages (the second very large) from account A to account B
> (they use different folders);
> 2. get all messages;
> 3. when message 1 is received (and messages 2 is still on dowloand), move with
> drag and drop message 1 to another account to other folder (inbox, sentbox,
> draft o custum folder):
> 
> No corruption observed.

test on pop3 accounts
Maximiliano, gerv, et al, is this still broken for you?
(In reply to Wayne Mery (:wsmwk) from comment #21)
> Maximiliano, gerv, et al, is this still broken for you?

I no longer use Thunderbird anymore.
I no longer use Thunderbird in the way outlined in comment #3. I do see occasional index corruption, but nothing like on the scale it used to be. These days, it is most commonly associated with Undoing a move of a message from one folder to another.

Gerv
Severity: normal → major
Whiteboard: [datalossy]

If anyone still sees this issue please file a new bug report

Severity: major → critical
Status: NEW → RESOLVED
Closed: 5 years ago
Component: General → Folder and Message Lists
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.