Last Comment Bug 754301 - Messages lost when downloaded and inbox corrupted(==column change is lost) - POP3
: Messages lost when downloaded and inbox corrupted(==column change is lost) -...
Status: RESOLVED FIXED
: dataloss
Product: MailNews Core
Classification: Components
Component: Networking: POP (show other bugs)
: 12
: x86_64 Windows 7
: -- critical with 1 vote (vote)
: Thunderbird 16.0
Assigned To: David :Bienvenu
:
Mentors:
: 754508 (view as bug list)
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2012-05-11 08:30 PDT by quantum.projects
Modified: 2012-10-02 00:17 PDT (History)
11 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
+
fixed
fixed
fixed


Attachments
Message Filter Log (44.58 KB, text/plain)
2012-05-16 18:33 PDT, quantum.projects
no flags Details
The bug begins to happen (42.80 KB, application/pdf)
2012-05-24 16:23 PDT, quantum.projects
no flags Details
Messages being lost 01 (69.19 KB, application/pdf)
2012-05-24 16:23 PDT, quantum.projects
no flags Details
Messages being lost 02 (65.28 KB, application/pdf)
2012-05-24 16:25 PDT, quantum.projects
no flags Details
Messages being lost 03 (65.10 KB, application/pdf)
2012-05-24 16:26 PDT, quantum.projects
no flags Details
Messages lost (67.37 KB, application/pdf)
2012-05-24 16:28 PDT, quantum.projects
no flags Details
Inbox reset (45.87 KB, application/pdf)
2012-05-24 16:30 PDT, quantum.projects
no flags Details
Edited TB log file (185.72 KB, text/plain)
2012-05-24 16:32 PDT, quantum.projects
no flags Details
30.05.2012 Log file, first part. (15.17 KB, text/plain)
2012-05-30 17:41 PDT, quantum.projects
no flags Details
possible fix (1.04 KB, patch)
2012-05-31 17:32 PDT, David :Bienvenu
standard8: review+
standard8: approval‑comm‑aurora+
standard8: approval‑comm‑beta+
standard8: approval‑comm‑release+
Details | Diff | Review

Description quantum.projects 2012-05-11 08:30:14 PDT
User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0
Build ID: 20120420145725

Steps to reproduce:

Tried to retrieve messages from POP3 Gmail server.


Actual results:

Technical Details:

OS: Windows 7 64x, all recent updates installed.
Thunderbird: 12.0.1

From time to time, when asked to download messages from POP3 Gmail server, Thunderbird randomly corrupts the inbox folder and stops downloading messages. 

It no longer displays the "Messages received" pop up, no further messages are downloaded, inbox column layout is reset to default layout, indicating that the Inbox file may be corrupted. Repairing the .msf file did no good to restore the inbox.

If another synchronization with the POP3 server is attempted before any repair, no files will be downloaded, but Gmail will mark then as such, so, when the inbox file is deleted by the user and rebuilt by Thunderbird, messages received prior to this step will not be downloaded.

Message filters are enabled to automatically put messages in different folders.

This problem is happening even when the inbox is empty prior to server synchronization.

Automatic folder compaction is disabled, but it is done every time Thunderbird asks for it.

This bug is happening at least twice a week. It seems to have begun after upgrading Thunderbird to version 12.0.

Temporary workaround: delet inbox file and .msf. Let Thunderbird rebuild it. If Thunderbird tried to synchronize with Gmail POP3 server prior to this, messages received in between may have to be manually downloaded (I use to foward them to me again).


Expected results:

All messages should be properly downloaded. The inbox folder should not be corrupted.
Comment 1 WADA 2012-05-11 18:10:52 PDT
As you wrote "Build ID: 20120420145725", it looks Tb 12.0.0.
Did you use Tb 12.0.0 for a while with message filter which has action of "move to local mail folder" before upgrade to Tb 12.0.1(should be datd 20120428)?

Confusing(bug 730947 has status-thunderbird12:fixed) but patch of bug 730947 is landed on Tb 12.0.1 instead of Tb 12.0.0. As seen in Release Notes of Tb 12.0.1, poblem relevant to POP3 filter move is resolved by Tb 12.0.1.
Fixing of that bug won't remove bad data in .msf generated by that bug automatically. So, if Tb 12.0.0 was used, bad .msf data may remain, and manual recovery("Repair Folder" for example) is needed in such case.  
Please read both bug 736539 and bug 730947.

Do you see your problem even after upgrade to Tb 12.0.1 and "manual recovery operation from bug 730947"=="manual clean up of bad data in .msf file by that bug"?
Comment 2 quantum.projects 2012-05-13 19:20:03 PDT
I did indeed use TB 12.0.0 before upgrading to TB 12.0.1, with "move to local folder" filters enabled. However, I have already deleted both inbox files (one without any extension, and the .msf file), and the bug still keeps happening once in a while.

The only folder that gets affected by this bug is inbox. My other folders are ok.
Comment 3 WADA 2012-05-13 21:03:05 PDT
(In reply to quantum.projects from comment #0)
> Thunderbird randomly corrupts the inbox folder
> and stops downloading messages. 

What kind of phenomenon do you call by "corrupts"?

> It no longer displays the "Messages received" pop up,
> no further messages are downloaded, 

No automatic new mail check and download only?
Nothing is downloaded by Get Msgs even though new mail exist in Gmail's POP3 Mbox at server?

Does other manual operation such as "Copy/Move mails from a folder to other folder", "Run Filters on Folder" work when your problem occured? 

When your problem happened, can mails be downloaded manually(by Get Msgs) after next operation?
- Go Work Offline mode (File/Offline, check Work Offline)
- Then go back to Work online mode ((File/Offline, uncheck Work Offline))

> inbox column layout is reset to default layout,

When reset to default?
After restart, it's OK, but after a while, when your problem occurs, reset to default when you open Inbox again?

> indicating that the Inbox file may be corrupted.

What is the "indication"?
Comment 4 quantum.projects 2012-05-14 17:12:54 PDT
(In reply to WADA from comment #3)
> What kind of phenomenon do you call by "corrupts"?

It seems to corrupt, because the inbox no longer receives new messages, and all columns are reset to default. For instance, I always remove "Junk" as a column, because I don't want it displayed. When this problem happens, the "Junk" column appears again. Also, I set the "Received" to display. When the problem happens, it no longer displays this column.

> > It no longer displays the "Messages received" pop up,
> > no further messages are downloaded, 

> No automatic new mail check and download only?
> Nothing is downloaded by Get Msgs even though new mail exist in Gmail's POP3
> Mbox at server?

Yes. Not even asking TB to get messages makes it download them. If I ask it to Get new Messages, on the lower bar, where it usually displays "Downloading XX of XX from xxxx@xxxxx", it will tell you that new messages are being downloaded, but they won't be placed anywhere.

> Does other manual operation such as "Copy/Move mails from a folder to other
> folder", "Run Filters on Folder" work when your problem occured? 

On any other folder, yes. On inbox, I haven't tried, because it no longer receives new messages.

> When your problem happened, can mails be downloaded manually(by Get Msgs)
> after next operation?
> - Go Work Offline mode (File/Offline, check Work Offline)
> - Then go back to Work online mode ((File/Offline, uncheck Work Offline))

I can ask TB to download, and, as I said above, it will tell you that new messages are being downloaded, but they are not. The last time this happened to me I had 48 new mail in Gmail, and I've got only 2 downloaded.

> > inbox column layout is reset to default layout,
> 
> When reset to default?
> After restart, it's OK, but after a while, when your problem occurs, reset
> to default when you open Inbox again?

It restarts to default when the problem is happening. In fact, I only noticed the bug when I saw that the displayed columns were reset to default. I asked TB to download new messages, it told me that 48 new messages were available. I minimized the window. When I returned to look it, the inbox columns were reset to default, and I had only 2 mails (sorted in two different folders, as expected due to "move to folder rules"). All other mail were gone.
> > indicating that the Inbox file may be corrupted.
> 
> What is the "indication"?

Same answer as last question.
Comment 5 :aceman 2012-05-15 00:27:30 PDT
Mayb this be similar to bug 754508?
Comment 6 quantum.projects 2012-05-15 08:04:48 PDT
It happened again today.

When I tried to open the inbox folder in TB while the bug was happening, it stated that the "Inbox cannot be opened because another operation is using it" or something like that.

TB downloaded all messages received till 8:55 am. It skipped four messages received later than that. None of those four messages had attachments or were from people unknown to me.

From all those messages, some were successfully put into the correct folders, and others remained at inbox because they weren't subject to any message filter rule.

I manually repaired all .msf files for all folders, in TB, through right-click > properties > repair folder, but those four lost messages didn't appear anywhere.

I forwarded to myself each of those four messages individually through Gmail web client, and asked TB to synchronize again. All messages were downloaded successfully.

TB stopped to download at one message that would be moved to another folder through "move to folder" filter. This message rule was created while I was using older versions of TB. The message was sent to "undisclosed recipients", but it is from a trusted source and was small in size. The message rule that would move it would not check the "To:" field of the message, just the subject

I manually openned the inbox file in notepad, and those four messages weren't there. I also openned the file corresponding to the folder where that first lost message would be moved to, and the message wasn't there either.

I also edited the Junk file, and it was not there either.

Answering to aceman, yes, it is very similar to bug 754508. I had 18 messages to download, and most of them would be moved to different folders through messages rules.

Again, the column layout for inbox was changed back to default, overriding my configurations.

I'm inclined to downgrade TB to version 11.0, in which I had not this bug. I also believe this is a very critical bug, perhaps involving a conflict in fetch message timeout and message rule processing time, that can cause a lot of damage to enterprise users (like me).

Strangely, I have not yet experienced this bug in a Windows 32 XP, Service Pack 2 machine, with TB 12.0.1 and an account which has similar message rules and similar amount of messages to download with each fetch operation.
Comment 7 :aceman 2012-05-15 08:10:58 PDT
Can you see in Tools->Addons to see if you have the Test pilot extension enabled? If yes, try to disable it.
Comment 8 quantum.projects 2012-05-15 08:14:33 PDT
Additional information: it seemed that it took longer to TB to download each message. Usually, it downloads 18 messages really fast, even when those messages have more than 1mb in size. However, while the bug was happening, it took it almost a full minute to download 18 messages with no attachments.
Comment 9 quantum.projects 2012-05-15 08:26:55 PDT
Yes, I did have test pilot enabled. I'll disable it and see if this bug happens again.

I'll give it a week to happen again. If it doesn't, I'll let you know.
Comment 10 :aceman 2012-05-15 08:31:56 PDT
That one could help with the resetting of the columns.
Comment 11 quantum.projects 2012-05-15 08:37:06 PDT
Ok. From bug 754508, which steps should I try to pinpoint what is causing this bug?
Is there a log function to TB that allows me to log everything it is doing, in excrutiating detail?
Comment 12 quantum.projects 2012-05-15 08:38:06 PDT
(In reply to :aceman from comment #10)
> That one could help with the resetting of the columns.

In fact, it would be better if it doesn't solve this column problem, because I know a bug has happened exactly because of it.
Comment 13 quantum.projects 2012-05-15 08:46:05 PDT
How can I post here my filterlog? Do I have to zip it and post it as an attachment?
Comment 14 Fred Stluka 2012-05-15 08:55:37 PDT
(In reply to :aceman from comment #7)
> Can you see in Tools->Addons to see if you have the Test pilot extension
> enabled? If yes, try to disable it.

I am the person who reported related bug:
    https://bugzilla.mozilla.org/show_bug.cgi?id=754508

Is Test Pilot likely to be the culprit.  I don't know anything about it.  Does
it do something relevant?  In any case, I have now also disabled Test Pilot, 
which was previously enabled.

Hard to tell yet whether it has any effect.  I've only seen the bug
described in 754508 once in the past several days since I moved from 
12.0.1 to 11.0.

--Fred Stluka
Comment 15 :aceman 2012-05-15 09:47:33 PDT
(In reply to quantum.projects from comment #12)
> (In reply to :aceman from comment #10)
> > That one could help with the resetting of the columns.
> In fact, it would be better if it doesn't solve this column problem, because
> I know a bug has happened exactly because of it.
The columns problem could be unrelated so I wanted to exclude it. But if you can see a relation then no problem, keep it as an indicator.

(In reply to quantum.projects from comment #11)
> Is there a log function to TB that allows me to log everything it is doing,
> in excrutiating detail?
https://wiki.mozilla.org/MailNews:Logging
Comment 16 :aceman 2012-05-15 09:51:05 PDT
(In reply to quantum.projects from comment #13)
> How can I post here my filterlog? Do I have to zip it and post it as an
> attachment?

You can add it as attachment here. Could you see any mention of the messages that were "lost" in the log?
Comment 17 quantum.projects 2012-05-16 18:33:38 PDT
Created attachment 624619 [details]
Message Filter Log
Comment 18 quantum.projects 2012-05-16 18:48:02 PDT
(In reply to :aceman from comment #16)
> (In reply to quantum.projects from comment #13)
> > How can I post here my filterlog? Do I have to zip it and post it as an
> > attachment?
> 
> You can add it as attachment here. Could you see any mention of the messages
> that were "lost" in the log?

No, they were not mentioned in the log. I've seen some strange entries with strange dates, but many of them date back to days when I had not had the bug, so I believe this is unrelated.

Sorry to ask again, but I didn't find those two environment variables mentioned on the wikipage you shared. Are they not in Options > Advanced > Config Editor?
Comment 19 quantum.projects 2012-05-16 19:35:53 PDT
Forget about my last question, I just found out how to do it.

All right, I've set the batch to create a log every time I start TB, with level 5 POP3 and SMTP logging enabled, and timestamp. If the bug ever happens again I'll put the entire day's log here (luckily I don't have TB on all time).
Comment 20 :aceman 2012-05-16 23:55:24 PDT
I think the log will be very long so you should exclude SMTP (we don't need it). And also it will contain the contents of the messages so you may need to censor it a bit :) (like leave commands TB sent to the server but remove lines with the msg content).
Comment 21 quantum.projects 2012-05-18 16:34:05 PDT
Ok, will do.
Comment 22 quantum.projects 2012-05-24 16:23:00 PDT
Created attachment 627008 [details]
The bug begins to happen
Comment 23 quantum.projects 2012-05-24 16:23:38 PDT
Created attachment 627009 [details]
Messages being lost 01
Comment 24 quantum.projects 2012-05-24 16:25:29 PDT
Created attachment 627013 [details]
Messages being lost 02
Comment 25 quantum.projects 2012-05-24 16:26:57 PDT
Created attachment 627014 [details]
Messages being lost 03
Comment 26 quantum.projects 2012-05-24 16:28:27 PDT
Created attachment 627015 [details]
Messages lost
Comment 27 quantum.projects 2012-05-24 16:30:09 PDT
Created attachment 627017 [details]
Inbox reset
Comment 28 quantum.projects 2012-05-24 16:32:13 PDT
Created attachment 627019 [details]
Edited TB log file
Comment 29 quantum.projects 2012-05-24 16:39:11 PDT
All right, got the bug right on the leap! I've got a log, I've even got a tiny slideshow of this bug bugging me.

Steps:

1. Started TB through batch file;
2. Asked to retrive messages, by pressing F5. It tells me that I've got 35 new messages;
3. Bug is happening, Inbox folder is unavailable. Notice that I've set a different column layout to Inbox (attachment: TB Error);
4. It states that 9 messages were downloaded, but actually only 3 or so were received (attachment: Losing Messages 01);
5. It states that 25 messages were downloaded, but again, you may see that the number of unread messages is unchanged from the first screen capture. (attachment: Losing Messages 02);
6. It keeps losing further messages until the last one of the batch is downloaded. Still, only 3 unread messages. (attachment: Losing Messages 03)
7. No popup appeared to tell me how many messages were downloaded (attachment: Messages Lost).
7. Inbox column layout reset to default (attachment: Inbox reset), about 5 messages successfully downloaded (one or two were marked as read, by message filter rules), +- 32 messages lost. 

Luckily, the lost messages are stored in my Gmail, marked as read.

Log file confirms: a bug has happened. TB started to receive only gibberish, and did not distinguish between different messages.

I've kept the log entries of the last successfully message downloaded, and then foward. The entire log was 100 mb in size, so I edited it, kept many entries related to the bug, but discarded most of them. Log is still here, if needed. In the log I've uploaded, I've left the entries that followed the bug event, and added a few comments of my own (they start and end with ***).

I don't know if it'll help, but the next message that would be downloaded was subject of a message filter with a "move to folder" rule. The previous message (which was successfully downloaded) also was subject to a different filter with "move to folder" rule.
Comment 30 WADA 2012-05-24 17:49:00 PDT
(In reply to quantum.projects from comment #22)
> Created attachment 627008 [details]
> The bug begins to happen

Following are seen in screen shot.
(A) Dialog is shown by Tb.
> ! Unable to open the folder Inbox because it is used by some other operations.
>   Please wait for that operation to finish and then select the folder again.
>     [ OK ]
(B) Thread pane is blanked out.
(C) New message count of Inbox(folder pane) is ZERO.
(D) Thread pane's column selection is NOT reset or changed to default or 
    something, because Received column and Tag column is shown.   
 
(Q1) When was the dialog shown and was thread pane blanked out?
     (i)   when you switched to Inbox(clicked Inbox) frm other folder.
     (ii)  while Inbox is being viewed, dialog is suddenly shown,
           and thread pane was blanked out.
     (iii) when you tried to do "Repair Folder" of Inbox
     (iv)  other

(In reply to quantum.projects from comment #23)
> Messages being lost 01
(In reply to quantum.projects from comment #24)
> Messages being lost 02
(In reply to quantum.projects from comment #25)
> Messages being lost 03
(In reply to quantum.projects from comment #26)
> Messages lost

These are screen shot of sub folder named "Familia under Inbox".
(Q2) Did you reply "OK" to dialog of (A) then clicked(opened) "Familia" folder?  
(Q3) What phenomenon do you call by "Message lost" with these screen shots?
  Which message is last message that was normally downloaded?
  Which message is last message that was normally downloaded, moved to Familia?
  Which message is first message that was downloaded but not moved to Familia?
  Which message is first message that was not downloaded?

(In reply to quantum.projects from comment #27)
> Created attachment 627017 [details]
> Inbox reset

(E) Thread column setting looks default columns for Inbox.
This is normal phenomenon, if Inbox.msf content is somehow deleted by other problems. 

(Q4) Did you do other special operation such as manual Compact, Repair Folder, Restart Tb? Or you merely did following?
- Reply OK to dialog of (A)
- Switch to "Familia" folder from "Inbox" folder
- Switch to "Inbox" folder agin

Dialog of (A) seems phnomenon after problem like following bugs or phenomenon in Tb 12 when similar condition happend.
> Bug 497804 If "filter move" is invoked while "compact folder" is running,
>            both "compact folder" and "filter move" silently fails,
>            and "blank thread pane/UI hang" occurs on "move target folder"
> Bug 498814 "Compact Folder" silently fails and deletes .msf,
>            if mail folder file is opened by other software (in the worst case
>            of Tb 2, generates null mail folder file or deletes mail folder file)
> Bug 498185 MailDB(.msf) is corrupted(Rebuild-Index is invoked),
>            if "Compact Folder" is invoked on "copy target folder" while "copy to folder
>            by message filter" is running. "Compact Folder" itself silently fails.
> Bug 682774 Move message triggers compaction; message lost

(Q5) What is your auto-compact related setting?
     (check via Tools/Options/Advanced/General, Config Editor, with filter=purge)
  mail.prompt_purge_threshhold
  mail.purge.ask
  mail.purge_threshhold_mb
Comment 31 WADA 2012-05-24 18:51:42 PDT
(In reply to quantum.projects from comment #22)
> The bug begins to happen

It is your first referring to following dialog in this bug.
(A) Dialog is shown by Tb.
> ! Unable to open the folder Inbox because it is used by some other operations.
>   Please wait for that operation to finish and then select the folder again.
>     [ OK ]
(Q6) Does it mean that the screen shot of the dialog is first occurrence of the dialog in your problem?
Comment 32 WADA 2012-05-24 19:18:24 PDT
(In reply to quantum.projects from comment #28)
> Created attachment 627019 [details]
> Edited TB log file

As for POP3 level activity in the log, Tb issued RETR(retrieve, download mail data and DELE(deleted from server's Mbox) or all 35 mails, and Tb sent QUIT and server returned "+OK Farewell" for it.

(Q7) Was the POP3 log obtained when your problem occurred?
     Or it's log for normal download?

UIDL(returned to UIDL command) is writeen in X-UIDL: header in local mail folder file(...\Inbox if Inbox, ...\Inbox.sbd\Familia if "Familia under Inbox".

(Q8) Which mail is lost at \Inbox or \Inbox.sbd\Familia?
Comment 33 quantum.projects 2012-05-24 20:13:54 PDT
> Following are seen in screen shot.
> (A) Dialog is shown by Tb.
> > ! Unable to open the folder Inbox because it is used by some other operations.
> >   Please wait for that operation to finish and then select the folder again.
> >     [ OK ]
> (B) Thread pane is blanked out.
> (C) New message count of Inbox(folder pane) is ZERO.
> (D) Thread pane's column selection is NOT reset or changed to default or 
>     something, because Received column and Tag column is shown.   
>  
> (Q1) When was the dialog shown and was thread pane blanked out?
>      (i)   when you switched to Inbox(clicked Inbox) frm other folder.
>      (ii)  while Inbox is being viewed, dialog is suddenly shown,
>            and thread pane was blanked out.
>      (iii) when you tried to do "Repair Folder" of Inbox
>      (iv)  other

A: option (i).

> (In reply to quantum.projects from comment #23)
> > Messages being lost 01
> (In reply to quantum.projects from comment #24)
> > Messages being lost 02
> (In reply to quantum.projects from comment #25)
> > Messages being lost 03
> (In reply to quantum.projects from comment #26)
> > Messages lost
> 
> These are screen shot of sub folder named "Familia under Inbox".
> (Q2) Did you reply "OK" to dialog of (A) then clicked(opened) "Familia"
> folder?

Yes, I opened it at random. I had received no message that would be placed there when the screen shots were taken.

> (Q3) What phenomenon do you call by "Message lost" with these screen shots?
>   Which message is last message that was normally downloaded?
>   Which message is last message that was normally downloaded, moved to
> Familia?
>   Which message is first message that was downloaded but not moved to
> Familia?
>   Which message is first message that was not downloaded?

As you can see, in the image sequence I've uploaded, the number of unread messages remain the same (3 under "Publicações"), but at the bottom, TB states that it is downloading message 9 of 35, 25 of 35 and 35 of 35, respectively. The last successfully downloaded message was a return receipt, which would be placed in the "Lidas" folder. The next message that would be downloaded (but which wasn't) would be placed in the "Weyers" folder. Both would be moved by message filters.

The folder "Familia" has no relation to the bug. I opened it at random.

> (In reply to quantum.projects from comment #27)
> > Created attachment 627017 [details]
> > Inbox reset
> 
> (E) Thread column setting looks default columns for Inbox.
> This is normal phenomenon, if Inbox.msf content is somehow deleted by other
> problems. 
> 
> (Q4) Did you do other special operation such as manual Compact, Repair
> Folder, Restart Tb? Or you merely did following?
> - Reply OK to dialog of (A)
> - Switch to "Familia" folder from "Inbox" folder
> - Switch to "Inbox" folder agin

Replied ok to dialog of (A), changed to Familia folder (unrelated to this instance of the bug, since I had received no message that would be placed there), and then after the last message was "retrieved", I switched back to Inbox. Then, I closed TB, opened it again. Tried to Repair Inbox Folder. Explored the mbox file, the lost messages weren't there. Opened the "Weyers" mbox file, the first message not downloaded wasn't there either (but it should, if it were correctly retrieved). Closed TB. Deleted the Inbox file and .msf and restarted TB.

> Dialog of (A) seems phnomenon after problem like following bugs or
> phenomenon in Tb 12 when similar condition happend.
> > Bug 497804 If "filter move" is invoked while "compact folder" is running,
> >            both "compact folder" and "filter move" silently fails,
> >            and "blank thread pane/UI hang" occurs on "move target folder"

Unless TB is compacting folders without asking (and it shouldn't, because I've set it to ask before compacting), I believe it is not related to this. Also, it didn't simply downloaded and failed, it didn't downloaded at all (see my next post).

> > Bug 498814 "Compact Folder" silently fails and deletes .msf,
> >            if mail folder file is opened by other software (in the worst case
> >            of Tb 2, generates null mail folder file or deletes mail folder file)

How could this happen? Antivirus, perhaps? But it would happen everytime, and nowadays it is happening only once a week or two. Also, if it only deleted the .msf file, rebuilding it or deleting it would solve the problem, but it did not in my case.

> > Bug 498185 MailDB(.msf) is corrupted(Rebuild-Index is invoked),
> >            if "Compact Folder" is invoked on "copy target folder" while "copy to folder
> >            by message filter" is running. "Compact Folder" itself silently fails.

Have no idea, but again, the .msf file wasn't corrupted, neither the Inbox file. See my next post.

> > Bug 682774 Move message triggers compaction; message lost

This bug happened while retrieving messages from POP3 Server, but Bug 682774 happened while moving a single message, triggering compaction. I don't know if they are somehow related.

> 
> (Q5) What is your auto-compact related setting?
>      (check via Tools/Options/Advanced/General, Config Editor, with
> filter=purge)
>   mail.prompt_purge_threshhold
>   mail.purge.ask
>   mail.purge_threshhold_mb

mail.prompt_purge_threshhold = true
mail.purge.ask = true
mail.purge_threshhold_mb = 20
Comment 34 quantum.projects 2012-05-24 20:25:49 PDT
(In reply to WADA from comment #32)
> (In reply to quantum.projects from comment #28)
> > Created attachment 627019 [details]
> > Edited TB log file
> 
> As for POP3 level activity in the log, Tb issued RETR(retrieve, download
> mail data and DELE(deleted from server's Mbox) or all 35 mails, and Tb sent
> QUIT and server returned "+OK Farewell" for it.

Agreed. It does explain why those messages are marked as downloaded by Gmail Pop3 server. However, I've got a 100mb log file with several entries like these ones:

RECV: PkYf0s1kKj1oxoOIGlLH4R5qcTBYPLxQf1pJfgfYnVSube5kc0G8L8REMOkkZqRbr/KUFNl0NO112012-05-24 
20:33:22.263000 UTC - 0[1f0f140]: RECV: GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: 73QSDcxoyXsQapaWt8Cj+Wt6yDZia2AO0V+hR0k8w/bHbO7/B3yUSNu06dqfGXBcjX7/SB+vEKN7
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: NnP0ha3Dm5+k9V8AAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJl
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: bHMgogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 2800
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfL2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: T9abg5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: ImYzsKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kV2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: yAdhb9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ52012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: x687HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQCcgdxNZAEAAFAHAAAcAAgBd29y2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: ZC9fcmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAALSVy07DMBBF90j8Q+Q9cROgPNSkG4TULRSJrZNMHiK2I3sC9O8ZWtVNoTIbs4k0N8rcoztj2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: Z7H8lH30DsZ2WmUsiWcsAlXqqlNNxl7Wjxe3LLIoVCV6rSBjG7BsmZ+fLZ6gF0gf2bYbbERdlM1Y2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: izjcc27LFqSwsR5A0ZtaGymQStPwQZRvogGezmZzbqY9WH7UM1pVGTOrivzXm4Gc/+6t67or4UGX2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: owSFJyx4C6ICQx2FaQCp57ZOYoJk/LR/chkSACkYOPhvS759eiGCMljc9DRFF8Ku9mVwEzICUJXS2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: OAXYKz6EJA3JUGuFa1H0k1E4yUcRFEKNsgBDB+0wCif5IJKQSZSjRS1faf/dPsQxdyrvEKR3Nech2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: aWqt8cdyOMkbSdBMvi2nt8SuTn0A1yFT+IDiGRBpMyandCL6QBK6wP/7vvQmcRXS3/6KYa/4MrgL2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: iXB6GdyR4Ef/wfwLAAD//wMAUEsDBBQABgAIAAAAIQAquM0ZeDUAAJD5AAARAAAAd29yZC9kb2N12012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: bWVudC54bWzsnU2PG0e2pvcDzH9IaGUD9amSZFvT4kW5JN+WYFuCpDZ6m0VmlVKXZNL5UVJr5d1g2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: 1vcu7mIGkGYWF25AwADu2RizEv+Jf8k874lIMpNMkkmKVZY8ArqtKlYyMzLixDnvec9H/OmfXg762012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: wUWUZnEyvHPtcO/gWhANu0kvHp7fufaXp9/sfnktyPJw2Av7yTC6c+1vUXbtnzr/+T/96cXtXtIt2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: BtEwD7jFMLt9wV+f5fno9v5+1n0WDcJsLxlFQ/54lqSDMOfX9Hx/EKb/Uox2u8lgFObxadyP87/t2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: Xz84uHXN3ya5c61Ih7f9LXYHcTdNsuQs11duJ2dncTfy/5TfSNs8133zrh+yPXE/jfqMIRlmz+JR2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: Vt5tsOndeMVn5U0ulr3ExaBfXvdi1OZpvTR8wXoM+m7YL5K0N0qTbpRlfHrX/XFyx8ODZc/2E6hb2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: TL7RZgj1Z5YjGYTxcHIbScfM+k8Wb4/F23fP3tetpi/CXHSQpdOk9zf9Owpe3EYWe4/vXDs4+ObW2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: 0Y3rR9fKjx6x0AcHN24dfX3j+uTDu9FZWPTz+csfVT6yOz9K7Z8n+d/6Ebe8CPt3rj2JBveyURhK2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: jJNr+50/7TMEd2Ha+FjdIu/cezlI9oIn6V5wl/8/KOJXQS8K7sZpFOdJ0AuDw3d/D34I0zA4Gb+92012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: iPr66CRB9LuhLnwSDsLBaRiHwW8//Wtw95s9PTa3hzNGDaHdVBwdHx0csnH8lK07Fal/0W+SYZ5x2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: k25259pxGod9TQRLYv/NBmG/fxKOMjc59p1ykj6N1AmorZafzE9zOr9lP8nppx31SUt90qeY9U822012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: 6pM1XQTWDm/dOj44MTjoMdioDtaK0zwv+jWcdkkQZJOhgMJKJOaQ6tGXN258cWTv46FWXAFRQny72012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: gcfxSTCsg8D5mzmsZ/hTntJtYGsXxD1KoyxKL6JrnaCGIudvMDcaKSTmz48t7xwcHHx1/cuj3aMb2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: e9cPDq/vfbl38MWePqzduP2M+0ksJ2X51l8140+18oeyoxpAI0T37oKmaBbaRmGWH2dx6AFu8Jdh2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: jI8bBd89WWWZtUxcPYh6wSMcjVi+cBg8Sbpx1Au5w0mCk5viRl4YtH+ahqdh/1kSfDf+ucczalM32012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: tybel1l/xNVl2wlGeIJJGDwv0vFbHmoD6XlnZJQysl4S4IAFWcSAh4yt4PWvYGTjt1fwkFGMx4VP2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: Nf5fia1QshOwQMcX0TDG6/o6jc9ZpjhNgm/lph0P8/E/hrEu2nv3a3Djyy92gsMDfhLDke4EX0MJ2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 2800
2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: BD9olXeCeJh10zgPNWUn3z96EGTJacBOffe/gxtHbJHreze+uL7PBjncPTq4pDd1blhluTfa+9uQ2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: s+YHX0SDAA00kuObmRsdMbVJ9u51WPBJgORNVByXRqPx2/M0PIuCH4so6D+LgkFyEa3QXO+xr5u82012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: V+3oB+xkueQpHjvbJQm+D7MumxwW4JIWsrKEnZ0gHKVX8Jzo8p/RLBUZExmmQVaEK5Z2Y8Es6YlZ2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: i9qFy4jYveP/QB/8Xq+/0/6t884o6rNPzkLtFm0HLIn2TRSUGjxEb58XcRpEL0dJxmV1pCBzaARE2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: aWcrvFf5kcMji+1x/S/GnLWyx9ed9Xze5UHGp3VZ+Ch1n3obnL06Eblkf75+wxtw+5tG7g25XqCP2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: jX4cDXtRGvUehefR12kU/gv2nv3aLGWHewvnedUc2F079xcJSJuvr3w96Zlhnia9Yl4Y2y/aDAe62012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: HERVNGUjgjYQdeTXoDr3eedw7zB4smg+PJ6tQNDO+K02GlIqtDGQ9ax9uf0Lvic5VHcRWqBE/zwn2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: AcC1PNzVa0SB0xn6KTkVarBV069n4asoDeYAk7tDMSj64czLz+G86jObZXmpJFcciQVbQeiuNv8b2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: DGEU9WIsoWaCpe3Go3IGpJjOzMBrtfMiByMBOpM0GEQCXyHGPOwHYSCTD+ednD6PDA/LxodBOv7Z2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: /SV+pV9jaYihv/di068tuPq9V9mX6sR30Cf9+FX5VpEBk/aT1llhTWuP6jksEXTjdPxreo4jgM3v2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: j38eaHrzhGlOAuJFkdj2YBgplDN+kzKZIwUNkmAFOqg9a41xNcveGcMocZof+Ixm/Th2c/PbHfeK2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: V1PZBIMKlYwQULe/Y1whLQJrERB3Qpzj8BwAm+NHIdnaWIC15KxAw/H7rLw4JdCPMrkf+An95DTh2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: XgN2Bzf1Jnz2S3WzXFvMHYz9kFE4mVjji+M37S9unihkUbt1ZpMgwQUCHP6RJOIsJh4dELs01RXx2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: zrYFWG3/ssFZEpc6S6sORptf+oWrWCKDhsiZYxNar9SKO8VbulGzPNQpj1WYtibFxP4vdWhz23Dh2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: WuQdM0K5bBCWKjdnL3wOFeBivrJeRqFhldDNBmSCOpRZZkqXL1DzvK4z+G6FXXLyOpSiMcMRRIM42012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: l8bCORAIKwW29cznnfYvugB3OB0qJSmjlgX8z5u1OIMu6aNbc9inrHkHmfJsnqToxyIeLUYHDYB02012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: YIRbuKlz5OW3FKQt4OzlZGV1vzRPwdOkF17EIC2mN8EsKEtn/CYIMVAIK3jCkJXJQy8qgm5YZGEw2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: fj0nzjs4jlGKZ4ao4Eti/Wbw7azAOJsWDsdvoDsjnNJu4h1qSdlgweWNHze/mYkvAmJgZ/Z75Qo02012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: LLHhywmCY1xFlhsCZUcIyBt/IwuKze6GcDsiIYRN0QEpOx37zPw4lDr7WPfSfxzp+R5IGQlMSEUg2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: Alm3H6YATURg9tUXzvhy7TZPi7e+8QJtYqQgIur0Rvu7raPHOt6t87xkCdRTFI6HwbL1TkZ2glNE2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV: ybZfqdjWGFR764wiZjeI50l5XpGTFPfKtCaJcyGyLLfKf+LYIhGuQq3iqY1j7SbpyFw3pwUZLXYN2012-05-24 20:33:22.419000 UTC - 0[1f0f140]: RECV:

This is clearly not a message, and I've got a log file with so many entries like these that, when opened in Libreoffice Writer, the log file was more than 5000 pages long.

After a very long list of identical entries like these, with no regular entry (with message body, for instance) in between, it tells the server to delete message 35, quit and server replies.

> (Q7) Was the POP3 log obtained when your problem occurred?
>      Or it's log for normal download?

When the problem occurred. As mentioned before, this log was generated when I opened the TB the first time today, and asked it to retrieve messages.
 
> UIDL(returned to UIDL command) is writeen in X-UIDL: header in local mail
> folder file(...\Inbox if Inbox, ...\Inbox.sbd\Familia if "Familia under
> Inbox".
> 
> (Q8) Which mail is lost at \Inbox or \Inbox.sbd\Familia?

See above.
Comment 35 quantum.projects 2012-05-24 20:33:58 PDT
I've added comments to the log file, pinpointing the relevant events. You may see that, before losing all further messages, some messages that were successfully deleted have some RECV: "gibberish" in between other RECV: with regular and legible mail content.
Comment 36 quantum.projects 2012-05-24 20:43:31 PDT
Also, as a test, I've done it so that those 35 messages were retrieved again by TB. I don't have a log file of it (forgot to use it), but those 35 messages were sorted accordingly, and placed in the proper folders, in 1/3 the time TB took to fetch the messages during the bugging event.
Comment 37 quantum.projects 2012-05-24 20:48:31 PDT
If you want to, I have the entire log file here. I can upload it without any modification.
Comment 38 quantum.projects 2012-05-24 20:54:28 PDT
(In reply to quantum.projects from comment #35)
> I've added comments to the log file, pinpointing the relevant events. You
> may see that, before losing all further messages, some messages that were
> successfully deleted have some RECV: "gibberish" in between other RECV: with
> regular and legible mail content.

Let me try to put this in a different way: some parts of the messages that were lost (specially of the first ones lost) is retrieved correctly, but in between some RECV: "gibberish" entries appear. It gets worse and worse, until no legible message entry appears, only 5000(perhaps even more)pages of RECV: gibberish entries.
Comment 39 WADA 2012-05-24 23:00:17 PDT
(In reply to quantum.projects from comment #34)
> However, I've got a 100mb log file with several entries like these ones:
> 
> RECV:
> PkYf0s1kKj1oxoOIGlLH4R5qcTBYPLxQf1pJfgfYnVSube5kc0G8L8REMOkkZqRbr/
> KUFNl0NO112012-05-24 
> 20:33:22.263000 UTC - 0[1f0f140]: RECV:
> GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC
> 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV:
> 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb
> (snip)

Log for base64 encoded data lines of an attachment part, isn't it?
> 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
>  RECV: GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC
> 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
>  RECV: 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb
> 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
>  RECV: 73QSDcxoyXsQapaWt8Cj+Wt6yDZia2AO0V+hR0k8w/bHbO7/B3yUSNu06dqfGXBcjX7/SB+vEKN7
>(snip)

"New line in log file" is one of CRLF, LF, CR which may depend on IMAP server and OS on which Tb runs.
Does your tool appropriately treat these new line characters as "line break"?
(notepad.exe of Win-XP shows only CRLF as line break.)
Comment 40 WADA 2012-05-24 23:35:19 PDT
(In reply to quantum.projects from comment #33)
> > > Bug 682774 Move message triggers compaction; message lost
> This bug happened while retrieving messages from POP3 Server, 
> but Bug 682774 happened while moving a single message, triggering compaction.
> I don't know if they are somehow related.

Your setting is following. i.e. auto-compact is enabled and "prompt before auto-compact" is shown by Tb.
> mail.prompt_purge_threshhold = true
> mail.purge.ask = true
> mail.purge_threshhold_mb = 20
And, you say following in comment #0.
> Message filters are enabled to automatically put messages in different folders.

When POP3 download and filter move, "truncate at Inobx file" is done for "mail which is written(downloaded) to Inbox file and moved to other folder". So, "size of the moved mail" may not be counted as "deleted mail size".
But I don't know it's true or false. Because "filter move" is also a "copy mail to other folder + delete mail at source folder", "filter move" may be a trigger of auto-compact.

Does your comment mean that no auto-comact happened or you always replied "Cancel" to prompt since restart of Tb, and that you never did manual Compact/Compact Folders since restart of Tb?
Comment 41 quantum.projects 2012-05-25 03:32:33 PDT
(In reply to WADA from comment #39)
> (In reply to quantum.projects from comment #34)
> > However, I've got a 100mb log file with several entries like these ones:
> > 
> > RECV:
> > PkYf0s1kKj1oxoOIGlLH4R5qcTBYPLxQf1pJfgfYnVSube5kc0G8L8REMOkkZqRbr/
> > KUFNl0NO112012-05-24 
> > 20:33:22.263000 UTC - 0[1f0f140]: RECV:
> > GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC
> > 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV:
> > 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb
> > (snip)
> 
> Log for base64 encoded data lines of an attachment part, isn't it?

It seems as so, but it cannot possibly be, because once I start getting lines like these, they spawn for thousands of pages (on Libreoffice), exclusively. If you want to, I believe I can create a 10mb snippet of the log so you can see that I received no normal message part while I got these lines.

> > 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
> >  RECV: GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC
> > 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
> >  RECV: 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb
> > 2012-05-24 20:33:22.263000 UTC - 0[1f0f140]:
> >  RECV: 73QSDcxoyXsQapaWt8Cj+Wt6yDZia2AO0V+hR0k8w/bHbO7/B3yUSNu06dqfGXBcjX7/SB+vEKN7
> >(snip)
> 
> "New line in log file" is one of CRLF, LF, CR which may depend on IMAP
> server and OS on which Tb runs.
> Does your tool appropriately treat these new line characters as "line break"?
> (notepad.exe of Win-XP shows only CRLF as line break.)

I used Notepad to open the log, copied the relevant entries to a Open Document Text file, and then saved as a text file again, before uploading.
Comment 42 quantum.projects 2012-05-25 04:19:26 PDT
I ask you in advance to forgive my bad English, because it seems that I'm not being able to appropriately convey what have happened.

In my log, TB is downloading messages regularly:

2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: style=3D'font-size:12.0pt;font-family:"Times New =
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: Roman","serif";color:#1F497D'>This message was sent from a law firm and =
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: may contain confidential or privileged attorney/client =
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: information.&#8221;</span><span =
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: style=3D'font-size:12.0pt;color:#1F497D'><o:p></o:p></span></p></div></di=2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: v></body></html>
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: ------=_NextPart_001_071B_01CD38F9.00952940--
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: 2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: ------=_NextPart_000_071A_01CD38F9.009529402012-05-24 20:33:22.248000 UTC - 0[1f0f140]: 
RECV: Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document;2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: 	name="=?iso-8859-1?Q?Contesta=E7=E3o_=5Fdemora_-_Cota=E7=E3o_de_materiais_-_753?=
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: 	=?iso-8859-1?Q?7.docx?="2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: Content-Transfer-Encoding: base64
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: Content-Disposition: attachment;2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: 	filename="=?iso-8859-1?Q?Contesta=E7=E3o_=5Fdemora_-_Cota=E7=E3o_de_materiais_-_753?=
2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: 	=?iso-8859-1?Q?7.docx?="2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV:

> (and, in the very next log line, it starts receiving unreadable lines like those below)

2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: UEsDBBQABgAIAAAAIQDoMIDy2wEAAGUJAAATAAgCW0NvbnRlbnRfVHlwZXNdLnhtbCCiBAIooAAC2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.248000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 1400
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAC02012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: Vk1v2zAMvQ/YfzB0HWKlPQzDEKeHtTtuBZYBuyoSnWjTFySmbf79KLs2hiS1hxi+GBAEPj7yPVJe2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: 3b1YUzxBTNq7it2US1aAk15pt6vYz83XxSdWJBROCeMdVOwIid2t379bbY4BUkHRLlVsjxg+c57k2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: HqxIpQ/g6Kb20QqkY9zxIOQfsQN+u1x+5NI7BIcLzBhsvfpOBKJWUDyKiN+EpTz82UfFa+/ReYRU2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: EhwrvrRxOXXFRAhGS4FEnD85dZJ04etaS1BeHiylKjNciF5CSlSaNWUP/SFD88sk5CGht7+s4RrB2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: PkYf0s1kKj1oxoOIGlLH4R5qcTBYPLxQf1pJfgfYnVSube5kc0G8L8REMOkkZqRbr/KUFNl0NO112012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: GGI1LMdARxtZe1WGYa5QtUe2Qruuq2/ayx3sFiL5YbKmZ/bqoUdJJDyaOQze4o6mB6dmmrAOeYgC2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: 6dVMFSd/ThYB8tQoUAsa9JPBetMCCRDJADMsmA55qPx+yUG8nVz+mQfzioP4n/mn77Vr8yM9GsCb2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: 73QSDcxoyXsQapaWt8Cj+Wt6yDZia2AO0V+hR0k8w/bHbO7/B3yUSNu06dqfGXBcjX7/SB+vEKN72012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: NnP0ha3Dm5+k9V8AAAD//wMAUEsDBBQABgAIAAAAIQAekRq38wAAAE4CAAALAAgCX3JlbHMvLnJl2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: bHMgogQCKKAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.263000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 2800
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAjJLbSgNBDIbvBd9hyH032woi0tneSKF3IusDhJnsAXcOzKTavr2jILpQ217m9OfL2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: T9abg5vUO6c8Bq9hWdWg2JtgR99reG23iwdQWchbmoJnDUfOsGlub9YvPJGUoTyMMaui4rOGQSQ+2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: ImYzsKNchci+VLqQHEkJU4+RzBv1jKu6vsf0VwOamabaWQ1pZ+9AtcdYNl/WDl03Gn4KZu/Yy4kV2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: yAdhb9kuYipsScZyjWop9SwabDDPJZ2RYqwKNuBpotX1RP9fi46FLAmhCYnP83x1nANaXg902aJ52012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: x687HyFZLBZ9e/tDg7MvaD4BAAD//wMAUEsDBBQABgAIAAAAIQCcgdxNZAEAAFAHAAAcAAgBd29y2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: ZC9fcmVscy9kb2N1bWVudC54bWwucmVscyCiBAEooAABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:33:22.388000 UTC - 0[1f0f140]: RECV: AAAAALSVy07DMBBF90j8Q

> TB keeps getting only lines like these for a long time. Below, some lines that TB was receiving at 20:36:58. Notice: there was no legible mail line received between the last log line above and those below)

2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF2012-05-24 20:36:58.109000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 4200
2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFf/ZCmVuZHN0cmVhbQpl2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: bmRvYmoKMTUgMCBvYmoKMjA1ODEyCmVuZG9iagoxNiAwIG9iago8PCAvVHlwZSAvUGFnZQogL1Bh2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: cmVudCAzIDAgUgovUmVzb3VyY2VzIDw8IC9Qcm9jU2V0IFsvUERGIC9JbWFnZUIgL0ltYWdlQyAv2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: SW1hZ2VJXS9YT2JqZWN0IDw8L0kxOCAxOCAwIFI+PiA+PgovTWVkaWFCb3ggWzAgMCA1OTUgODQy2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: XS9Sb3RhdGUgMC9Db250ZW50cyAxNyAwIFIKPj4KZW5kb2JqCjE3IDAgb2JqCjw8IC9MZW5ndGgg2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: NTcKPj4Kc3RyZWFtCnEKUQpxClcKMAowCjU5NQo4NDIKcmUKbgpxCjU5NQowCjAKODQyCjAKMCBj2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: bQovSTE4IERvClEKUQplbmRzdHJlYW0KZW5kb2JqCjE4IDAgb2JqCjw8IC9UeXBlIC9YT2JqZWN02012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: Ci9TdWJ0eXBlIC9JbWFnZQovTmFtZSAvSTE4Ci9GaWx0ZXIgL0RDVERlY29kZQovV2lkdGggMTI02012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: MAovSGVpZ2h0IDE3NTQKL0RlY29kZSBbMCAxXQovQml0c1BlckNvbXBvbmVudCA4Ci9Db2xvclNw2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: YWNlIC9EZXZpY2VHcmF5Ci9MZW5ndGggMTkgMCBSCj4+CnN0cmVhbQr/2P/gABBKRklGAAEBAQCW2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: AJYAAP/bAEMACAYGBwYFCAcHBwkJCAoMFA0MCwsMGRITDxQdGh8eHRocHCAkLicgIiwjHBwoNyks2012-05-24 20:36:58.140000 UTC - 0[1f0f140]: RECV: MDE0NDQfJzk9ODI8LjM0Mv/AAAsIBtoE2AEBEQD/xAAfAAABBQEBAQEBAQAAAAAAAAAAAQIDBAUG

> (Actually, I've found some regular lines when TB finished downloading message 20, as seen below)

2012-05-24 20:37:41.571000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:37:41.571000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA2012-05-24 20:37:41.571000 UTC - 0[1f0f140]: RECV: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=2012-05-24 20:37:41.571000 UTC - 0[1f0f140]: RECV: --20cf303b4063e71c3b04c0c86cf1--2012-05-24 20:37:41.571000 UTC - 0[1f0f140]: RECV: .2012-05-24 20:37:41.618000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:37:41.618000 UTC - 0[1f0f140]: POP3: Entering state: 20
2012-05-24 20:37:41.618000 UTC - 0[1f0f140]: SEND: DELE 20
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 25
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: POP3: Entering state: 3
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: RECV: +OK marked for deletion
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: POP3: Entering state: 21
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: POP3: Entering state: 15
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: POP3: Entering state: 18
2012-05-24 20:37:41.836000 UTC - 0[1f0f140]: SEND: RETR 212012-05-24 20:37:42.413000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 2800
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: POP3: Entering state: 3
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV: +OK message follows
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: POP3: Entering state: 19
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: Opening message stream: MSG_IncorporateBegin
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: Done opening message stream!
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV: Delivered-To: r.afonsoadv@gmail.com2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV: Received: by 10.76.11.102 with SMTP id p6csp45565oab;2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV:         Thu, 24 May 2012 06:53:45 -0700 (PDT)2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV: Received: by 10.220.242.6 with SMTP id lg6mr1474997vcb.18.1337867625345;2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV:         Thu, 24 May 2012 06:53:45 -0700 (PDT)
2012-05-24 20:37:42.413000 UTC - 0[1f0f140]: RECV: Return-Path: <oabdf@recortedigital.adv.br>

>(This message was not found in any of my mbox files).
>(Later, it starts to receive gibberish again, as seen below, then suddenly asks the server to delete message 35 and log out)

2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: MDAwMTM3NDIzMiAwMDAwMCBuDQowMDAxMzc0NDIwIDAwMDAwIG4NCjAwMDEzNzQ1MTYgMDAwMDAg2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: bg0KMDAwMTM3NTkzNCAwMDAwMCBuDQowMDAxNjMxNTkxIDAwMDAwIG4NCjAwMDE2MzE3NzkgMDAw2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: MDAgbg0KMDAwMTYzMTg3NSAwMDAwMCBuDQowMDAxNjMzMjkzIDAwMDAwIG4NCjAwMDE4OTE5NjEg2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: MDAwMDAgbg0KMDAwMTg5MjE1MSAwMDAwMCBuDQowMDAxODkyMjQ3IDAwMDAwIG4NCjAwMDE4OTM22012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: NjUgMDAwMDAgbg0KMDAwMjE2ODMwNCAwMDAwMCBuDQowMDAyMTY4NDkyIDAwMDAwIG4NCjAwMDIx2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: Njg1ODAgMDAwMDAgbg0KMDAwMjE2OTk5OCAwMDAwMCBuDQowMDAyNDMzMzQ1IDAwMDAwIG4NCjAw2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: MDI0MzM1MzUgMDAwMDAgbg0KMDAwMjQzMzYzMSAwMDAwMCBuDQowMDAyNDM1MDQ5IDAwMDAwIG4N2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: CjAwMDI3NDc1MjEgMDAwMDAgbg0KMDAwMjc0NzcxMSAwMDAwMCBuDQowMDAyNzQ3ODA3IDAwMDAw2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: IG4NCjAwMDI3NDkyMjUgMDAwMDAgbg0KMDAwMzAyMzU4NSAwMDAwMCBuDQowMDAzMDIzNzc1IDAw2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: MDAwIG4NCjAwMDMwMjM4NzEgMDAwMDAgbg0KMDAwMzAyNTI4OSAwMDAwMCBuDQowMDAzMjQ4MTIy2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: IDAwMDAwIG4NCjAwMDMyNDgzMTIgMDAwMDAgbg0KMDAwMzI0ODQwOCAwMDAwMCBuDQowMDAzMjQ52012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: ODI2IDAwMDAwIG4NCjAwMDM0NzE1MDQgMDAwMDAgbg0KMDAwMzQ3MTU2NCAwMDAwMCBuDQowMDAz2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: NDcxNjU1IDAwMDAwIG4NCjAwMDM0NzE3ODUgMDAwMDAgbg0KMDAwMzQ3NTE3NyAwMDAwMCBuDQp02012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: cmFpbGVyDQo8PC9TaXplIDYyPj4NCnN0YXJ0eHJlZg0KMTE2DQolJUVPRg0K2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: 2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: ------=_NextPart_000_0159_01CD39CC.4667ECE0--2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: 2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: .2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: RECV: (null)
2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: POP3: Entering state: 20
2012-05-24 20:40:23.593000 UTC - 0[1f0f140]: SEND: DELE 35
2012-05-24 20:40:23.858000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 25
2012-05-24 20:40:23.858000 UTC - 0[1f0f140]: POP3: Entering state: 3
2012-05-24 20:40:23.858000 UTC - 0[1f0f140]: RECV: +OK marked for deletion
2012-05-24 20:40:23.858000 UTC - 0[1f0f140]: POP3: Entering state: 21
2012-05-24 20:40:23.858000 UTC - 0[1f0f140]: POP3: Entering state: 15
2012-05-24 20:40:23.873000 UTC - 0[1f0f140]: Calling ReleaseFolderLock from EndMailDelivery
2012-05-24 20:40:23.873000 UTC - 0[1f0f140]: ReleaseFolderLock haveSemaphore = TRUE
2012-05-24 20:40:23.920000 UTC - 0[1f0f140]: POP3: Entering state: 22
2012-05-24 20:40:23.920000 UTC - 0[1f0f140]: SEND: QUIT2012-05-24 20:40:24.544000 UTC - 0[1f0f140]: Entering NET_ProcessPop3 15
2012-05-24 20:40:24.544000 UTC - 0[1f0f140]: POP3: Entering state: 3
2012-05-24 20:40:24.544000 UTC - 0[1f0f140]: RECV: +OK Farewell.
Comment 43 quantum.projects 2012-05-25 04:23:56 PDT
This is the Message Filters log entries for the bugging event:

Applied filter "Publicações" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Publica%C3%A7%C3%B5es

Applied filter "Confirmações de Leitura" to message from "Weyers Advocacia" <weyersadv@gmail.com> - Lida: PROJETO PRICE TRABALHISTA DF-2 at 23/05/2012 14:55:52 marked as read

Applied filter "Confirmações de Leitura" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Lidas

> It jumps from the message received at 14:55 to the ones received at 17:49, as seen below.

Applied filter "Confirmações de Leitura" to message from Escritório Barbacena <operamg@gmail.com> - Lida: 61316 - CÓPIA INTEGRAL - 2011.05.1.005795-7 at 24/05/2012 17:49:26 marked as read

Applied filter "Confirmações de Leitura" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Lidas

Applied filter "Confirmações de Leitura" to message from Escritório Barbacena <operamg@gmail.com> - Lida: 61316 - CÓPIA INTEGRAL - 2011.05.1.005795-7 at 24/05/2012 17:52:37 marked as read

Applied filter "Confirmações de Leitura" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Lidas

Applied filter "Publicações" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Publica%C3%A7%C3%B5es

Applied filter "Publicações" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Publica%C3%A7%C3%B5es

Applied filter "Publicações" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Publica%C3%A7%C3%B5es

> Below, the Message Filter log entries for all messages that were not downloaded, when I asked Gmail do enable pop for all mail, including those already downloaded.

Applied filter "Confirmações de Leitura" to message from "Weyers Advocacia" <weyersadv@gmail.com> - Lida: PROJETO PRICE TRABALHISTA DF-2 at 23/05/2012 14:55:52 marked as read

Applied filter "Confirmações de Leitura" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Lidas

> Now, it processess all mail correctly.

Applied filter "Correspondência Weyers" to message from "Weyers Advocacia" <weyersadv@gmail.com> - RES: PROJETO PRICE TRABALHISTA DF-2 at 23/05/2012 15:11:37 tagged

Applied filter "Correspondência Weyers" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Weyers

Applied filter "Correspondência Weyers" to message from "Maria Eliza Lima" <mariaeliza@weyers.com.br> - Protocolo de contestação - PRazo fatal hoje 23/05 - 1ª Vara Cível, Samambaia - DF - Julia Correia do Nascimento X Unimed Paulistana at 23/05/2012 15:30:33 tagged

Applied filter "Correspondência Weyers" to message from - at 31/12/1969 22:00:00 moved message id = to mailbox://r.afonsoadv%40gmail.com@pop.gmail.com/Inbox/Weyers

Applied filter "Correspondência Weyers" to message from "Maria Eliza Lima" <mariaeliza@weyers.com.br> - Protocolo de contestação - PRazo fatal hoje 23/05 - 1ª Vara Cível, Samambaia - DF - Julia Correia do Nascimento X Unimed Paulistana at 23/05/2012 15:31:34 tagged

>etc.
Comment 44 quantum.projects 2012-05-25 04:34:17 PDT
> When POP3 download and filter move, "truncate at Inobx file" is done for
> "mail which is written(downloaded) to Inbox file and moved to other folder".
> So, "size of the moved mail" may not be counted as "deleted mail size".
> But I don't know it's true or false. Because "filter move" is also a "copy
> mail to other folder + delete mail at source folder", "filter move" may be a
> trigger of auto-compact.
> 
> Does your comment mean that no auto-comact happened or you always replied
> "Cancel" to prompt since restart of Tb, and that you never did manual
> Compact/Compact Folders since restart of Tb?

Auto compact did not prompt me to compact, not before, during or after those lost messages were downloaded. I did try a manual compact before deleting the inbox mbox file and .msf file.
Comment 45 quantum.projects 2012-05-25 04:36:24 PDT
Well, that was a wealth of data, but I confess I'm confused. WADA, what do you think is causing this?
Comment 46 WADA 2012-05-25 15:22:48 PDT
It may be problem relivant to bug 710056 comment #110, because Compact is apparently irrelevant to your problem.
  While download is in progress with Inbox not-selected at folder pane,
  you opened(clicked) Inbox. Due to problem, "Open Inbox" inereferes downloading.
  Sometimes, because downloading is using Inbox, "Open Inbox" detects "folder is
  in use" and dialog is shown. But if downloading ended, no dialog is shown.
  If downloading is interefered, download process accesss wrong data.

It may be "buffer over run" like problem. Gmail's server is perhaps one of fastest server. If very fast network is used, data from Gmail's server arrives rapidly.
It may be problem when network is slow.
Can you get POP3 log with timestamp in your environment, for normal download of the 35-th mail and a few other mails only?
Comment 47 WADA 2012-05-25 15:49:51 PDT
FYI.
There are three kinds of issue around POP3 download after Tb 12.
  This bug(Bug 754301) : Open Inbox while downloading, then your problem happens.
  Bug 710056 : Thread column selection is reset to default.
               Many of them are for Inbox, and is problem when restart.
               It may be next.
                 Inbox is opened, and restart is forced by upgrade.
                 Automatic download is executed at restart, and Inbox is shown
                 at folder pane/thread pane==Open Inbox occurs at same time.
                 Auto-compact may be relivant, but it doesn't look always.
               Second case is similar to yours although relivant process=Compact.
                 Empty Trash(auto-compact is invoked), then manually open Inbox.
  Bug 757331, or bug 752866 or bug 755317 or bug 714364 or bug 751562 :
    Message filter rules are disabled when "move to folder" action fires.
    It looks only when fisrt download by restart.
    This issue may be contention at "move target folder", and auto-compact seems
    relivant to problem.
Comment 48 WADA 2012-05-26 12:31:29 PDT
FYI.
bug 754508 comment #32 and #33 reports similar funny filter log and refers to "file size greater than 2GB on MS Win".
Comment 49 quantum.projects 2012-05-26 13:59:18 PDT
(In reply to WADA from comment #46)
> It may be problem relivant to bug 710056 comment #110, because Compact is
> apparently irrelevant to your problem.
>   While download is in progress with Inbox not-selected at folder pane,
>   you opened(clicked) Inbox. Due to problem, "Open Inbox" inereferes
> downloading.
>   Sometimes, because downloading is using Inbox, "Open Inbox" detects
> "folder is
>   in use" and dialog is shown. But if downloading ended, no dialog is shown.
>   If downloading is interefered, download process accesss wrong data.
> 
> It may be "buffer over run" like problem. Gmail's server is perhaps one of
> fastest server. If very fast network is used, data from Gmail's server
> arrives rapidly.
> It may be problem when network is slow.
> Can you get POP3 log with timestamp in your environment, for normal download
> of the 35-th mail and a few other mails only?

I'll get the log file for normal download of many messages tomorrow in the afternoon. I won't be able to reproduce the specific download of those same messages that were being downloaded when the bug last happened to me. 

On the other side, I'm not sure the bug is related to openning the Inbox folder while messages are being downloaded, because I do it all the time and it doesn't necessarily cause the bug to happen.

In my non-expert opinion, I believe the bug may be caused because:

a) There may be no flag variable to tell TB that the folders are undergoing compaction when it tries a connection to the server to download messages (and vice-versa), which could cause parallel modifications of the same file, corrupting it.

b) It could be an error on one of the algorithms that translates the bits received from the server into messages. Perhaps TB is skipping a field of the message, and then mistranslates anything it receives afterwards.

c)It could also be that TB is not processing correctly messages that contain attachments. Every time the bug happened to me, the first lost message had some kind of attachment.

Well, I'll try to help pinpoint the bug the best I can. The log will be here tomorrow night.

As a precaution, I've changed auto-compact threshold to 1 mb, instead of 20mb. I've had other problems, when using past versions of TB, because I used to set the threshold too high. If moved messages do not count to the auto-compact threshold, I might be having a huge inbox file (gigabytes in size), and then the problem is not related to the POP3 itself, but to the gargantuan size of the Inbox file.
Comment 50 quantum.projects 2012-05-26 14:03:40 PDT
(In reply to WADA from comment #46)
> It may be problem relivant to bug 710056 comment #110, because Compact is
> apparently irrelevant to your problem.
>   While download is in progress with Inbox not-selected at folder pane,
>   you opened(clicked) Inbox. Due to problem, "Open Inbox" inereferes
> downloading.
>   Sometimes, because downloading is using Inbox, "Open Inbox" detects
> "folder is
>   in use" and dialog is shown. But if downloading ended, no dialog is shown.
>   If downloading is interefered, download process accesss wrong data.
> 
> It may be "buffer over run" like problem. Gmail's server is perhaps one of
> fastest server. If very fast network is used, data from Gmail's server
> arrives rapidly.
> It may be problem when network is slow.
> Can you get POP3 log with timestamp in your environment, for normal download
> of the 35-th mail and a few other mails only?

I'll get the log file for normal download of many messages tomorrow in the afternoon. I won't be able to reproduce the specific download of those same messages that were being downloaded when the bug last happened to me. 

On the other side, I'm not sure the bug is related to openning the Inbox folder while messages are being downloaded, because I do it all the time and it doesn't necessarily cause the bug to happen.

In my non-expert opinion, I believe the bug may be caused because:

a) There may be no flag variable to tell TB that the folders are undergoing compaction when it tries a connection to the server to download messages (and vice-versa), which could cause parallel modifications of the same file, corrupting it.

b) It could be an error on one of the algorithms that translates the bits received from the server into messages. Perhaps TB is skipping a field of the message, and then mistranslates anything it receives afterwards.

c)It could also be that TB is not processing correctly messages that contain attachments. Every time the bug happened to me, the first lost message had some kind of attachment.

Well, I'll try to help pinpoint the bug the best I can. The log will be here tomorrow night.

As a precaution, I've changed auto-compact threshold to 1 mb, instead of 20mb. I've had other problems, when using past versions of TB, because I used to set the threshold too high. If moved messages do not count to the auto-compact threshold, I might be having a huge inbox file (gigabytes in size), and then the problem is not related to the POP3 itself, but to the gargantuan size of the Inbox file.
Comment 51 WADA 2012-05-26 14:34:38 PDT
(In reply to quantum.projects from comment #50)
> a) There may be no flag variable to tell TB that the folders are undergoing
> compaction when it tries a connection to the server to download messages
> (and vice-versa), which could cause parallel modifications of the same file,
> corrupting it.

As for between "Compacting" and "Repair Folder or Manual Get Msgs", a locking mechanism already exists(a MsgDB level locking).
  If compacting is in progress on POP3 Inbox,
  both "Repair Folder" and "manul Get Msgs" is rejected with message like
  "other operation is in progress" which is similar to dialog you saw.

> c)It could also be that TB is not processing correctly messages that contain
> attachments. Every time the bug happened to me, the first lost message had
> some kind of attachment.

As for POP3 download, "with attachment" or "without attachment" is irrelevant, because download of POP3 mail is merely a combination of "RETR command" and "mail data stream from server" only. "With attachment" or "without attachment" is relevant only to mail data size which should be sent by server/received by mail client.

> As a precaution, I've changed auto-compact threshold to 1 mb, instead of
> 20mb. I've had other problems, when using past versions of TB, because I
> used to set the threshold too high. If moved messages do not count to the
> auto-compact threshold, I might be having a huge inbox file (gigabytes in
> size), and then the problem is not related to the POP3 itself, but to the
> gargantuan size of the Inbox file.

For auto-compact, too small threshold value merely forces (a) too frequent dialog if mal.purge.ask=true, or (b) too frequent useless compacting(merely 1MB gain in file size) with long compacting time(if 1GB file, non-deleted 1023 MB should be copied) if mal.purge.ask=false.
Threshold value is "sum of deleted mail size". See bug 758803.
And, Tb 12 still has bug 725810 and Tb 12 has bug 750569 in auto-compact.
Are you looking bug 725810?
Comment 52 WADA 2012-05-27 00:19:20 PDT
Other concerns - qurantine option and Global Inbox.

For quarantine option.
If rebuild-index was in progress on Inbox, or if Compact was in progress, Download action with mailnews.downloadToTempFile=true was not executed with "other process is running" like message.
If rebuild-index was on progress on filter move target folder, or if Compact was in progress on filter move target folder, filter move silently failed and move was not executed(filter log says nothing about move failute. already known issue).
If mail is downloaded to temp file, and if move to Inbox or move to filter move target folder somehow fails, mail may be lost.

Do you enable quarantine option? (mailnews.downloadToTempFile=true)

For Global Inbox.
If download is done at hidden Inbox of deffering account, and if rebuild-index or Compact is in progress at Global Inbox folder, downloaded mail remains in hidden Inbox.

Do you use Global Inbox?
Comment 53 quantum.projects 2012-05-27 15:12:41 PDT
Yes, I do use quarantine options, and no, I do not use Global Inbox. 

Should I disable quarantine for now and see if it prevents the bug from happening?

>If rebuild-index was in progress on Inbox, or if Compact was in progress, Download action with mailnews.downloadToTempFile=true was not executed with "other process is running" like message.
If rebuild-index was on progress on filter move target folder, or if Compact was in progress on filter move target folder, filter move silently failed and move was not executed(filter log says nothing about move failute. already known issue).
If mail is downloaded to temp file, and if move to Inbox or move to filter move target folder somehow fails, mail may be lost.

It sounds like a very likely candidate to be the cause of the bug, since every time the bug happened to me, the first lost message would also be the target of a move-to-folder filter rule.

I didn't know TB would download messages to a temporary file before putting the messages into the proper Inbox folder.
Comment 54 quantum.projects 2012-05-27 15:15:02 PDT
Anyway, it doesn't explain why TB takes so long to "download" the messages when the bug is happening. It seemed like a loop to me.
Comment 55 quantum.projects 2012-05-28 10:48:22 PDT
New info: today the bug happened to me again, and I restrained myself from openning the Inbox file during its occurrance. The log file is much smaller now (<1mb) and all messages seems to have been received by TB, but, as before, some messages weren't placed anywhere.

When the retrieval of messages ended, I got the pop up telling that 21 messages had been retrieved, but it had no messages listed in it (the correct behavior is to display a header for the first five messages, if I'm not mistaken). When I clicked on the Inbox File, TB stated that the inbox index was being rebuilt, and then its content was displayed.

I've checked the inbox file and some other mbox files, the lost messages aren't there.

What should I do now? Should I disable quarantine to see if it is the cause of the bug?
Comment 56 quantum.projects 2012-05-28 11:12:29 PDT
I've tried to:

1) Retrieve those lost messages again, without deleting the inbox file or repairing it. It was no use, the very same messages got lost again.

2) Repair the Inbox folder before synchronizing again through right-click > properties > repair folder. This time, TB downloaded all messages *** except for the most recently received one ***, and again, the popup did show up, but with no information about the five first downloaded messages.

3) I've deleted only the Inbox .msf file and tried again. Again, the last received message was not put in any folder, but this time a popup with the header of the last four messages received was displayed.

4) Deleted the Inbox mbox file and the .msf, after compacting to save previously downloaded messages. The last message wasn't put into any folder again (new strange behavior). The inbox column layout was reset again to default.

This time, the lost message had no attachment. It was destined to "undisclosed recipients" and would be a target for a move-to-folder rule. Previous mail from the same sender and with the same characteristics have been correctly received before.
Comment 57 quantum.projects 2012-05-28 11:15:08 PDT
I'll disable quarantine and see if it solves the problem. Maybe Quarantine and Message filters conflict with each other.
Comment 58 quantum.projects 2012-05-28 11:18:02 PDT
Actually, disabling junk control. If it doesn't help, I'll disable Quarantine.
Comment 59 quantum.projects 2012-05-29 18:16:32 PDT
Is it safe to disable quarantine? Do I risk losing more than those messages I'm trying to download? The bug happened on my machine again, it seems it is getting more and more frequent.
Comment 60 quantum.projects 2012-05-29 18:46:14 PDT
I've noticed just now that the .msf files of all other folders were corrupted, but I'm not sure if it is a direct consequence of the bug, or if it is related to me setting Gmail to download again all messages received (after moving unwanted messages to trash, and then moving them back again to Inbox).

Geez, this is getting worse by the minute. I can't keep losing messages like this.
Comment 61 quantum.projects 2012-05-30 07:44:32 PDT
I might have discovered a possible cause for the bug.

Reviewing my "move-to-folder" filters, I've noticed that some of them hadn't had a "stop filter execution" clause. 

In every instance of the bug, I believe the first lost message was subject to one of these no-stop-filter-execution message filters.

Every one of my filters have "match any" clauses, and, usually, those first lost messages would be moved to the expected folder based on more than one of the 'any' clauses.

So, I don't know if it's possible at all, but maybe those lost messages were subject to the same message filter again? Or could it be that, after being moved to another folder, TB checked them towards the next message filter on the stack and failed, because the message had been already moved?

I'll inform if the bug stops happening after these changes I've made.
Comment 62 Joe Sabash [:JoeS1] 2012-05-30 16:26:39 PDT
David, any ideas here for triage.
Comment 63 David :Bienvenu 2012-05-30 16:47:01 PDT
adding MSGDB:5 to the logging might be helpful.

Move filter actions have an automatic stop execution after the move action, so you shouldn't need an explicit one.

It's safe to disable quarantining if you don't have an anti-virus program that will delete your inbox.

For some users that had similar issues, disabling quarantining helped. One of them also had multiple accounts using the same local directory (in account settings, the local directory was set to the same directory as other pop3 servers), though they were only receiving mail on one of them.
Comment 64 David :Bienvenu 2012-05-30 17:05:35 PDT
Oh, and quantum.projects@yahoo.com, if you want to e-mail me your prefs.js file (to dbienvenu@mozilla.com), and turn on MSGDB:5 logging, and send me a log of when a failure happens, I'd be happy to look at it. Given that you are losing your extra columns, it's pretty clear that there's an invalid db thing happening.
Comment 65 quantum.projects 2012-05-30 17:41:32 PDT
Created attachment 628550 [details]
30.05.2012 Log file, first part.

Yeah, adding "stop filter execution" clauses didn't help. The bug happened to me again, the third time this week. This time I closed TB before the bug could cause more damage.

I do use an antivirus (Kaspersky Internet Security), but it haven't caused any problem before.

I've attached the first part of my log here. The full log is 63 mb in size.

The first lost message (this time, the very first message that was being downloaded) had attachment.

I've included MSGDB:5, now I have a 441mb log here. What should I be looking for?
Comment 66 David :Bienvenu 2012-05-30 17:45:34 PDT
(In reply to quantum.projects from comment #65)
> Created attachment 628550 [details]
> 30.05.2012 Log file, first part.
> 
> Yeah, adding "stop filter execution" clauses didn't help. The bug happened
> to me again, the third time this week. This time I closed TB before the bug
> could cause more damage.
> 
> I do use an antivirus (Kaspersky Internet Security), but it haven't caused
> any problem before.
> 
> I've attached the first part of my log here. The full log is 63 mb in size.
> 
> The first lost message (this time, the very first message that was being
> downloaded) had attachment.
> 
> I've included MSGDB:5, now I have a 441mb log here. What should I be looking
> for?

I'm curious if you see any log entries where we're closing the inbox db in the middle of a pop3 download - it would say "closing " along with the path to the inbox.msf file
Comment 67 quantum.projects 2012-05-30 17:51:10 PDT
Yes, it stated something like Abortmessage..., close something, semaphore=FALSE, but I don't have the log anymore.

I'm trying to re-download the lost messages again, let me see if the bug happens twice today.
Comment 68 quantum.projects 2012-05-30 17:52:05 PDT
(In reply to David :Bienvenu from comment #64)
> Oh, and quantum.projects@yahoo.com, if you want to e-mail me your prefs.js
> file (to dbienvenu@mozilla.com), and turn on MSGDB:5 logging, and send me a
> log of when a failure happens, I'd be happy to look at it. Given that you
> are losing your extra columns, it's pretty clear that there's an invalid db
> thing happening.

I can do that, but how can I send you the log? It is more than 400mb.
Comment 69 quantum.projects 2012-05-30 18:00:43 PDT
Well, the bug just happened again, but I'm not sure if I've lost messages this time, probably I had, but no important one.

Full log ready to deploy anywhere, just tell me how. Prefs mailed to David, as asked.
Comment 70 quantum.projects 2012-05-30 18:24:16 PDT
The first lines of my log seem very strange:

>2012-05-31 00:54:04.748000 UTC - 0[b0f140]: nsMsgDatabase::Open(C:\Arquivos Pessoais\Thunderbird\r.afonsoadv@gmail.com\Inbox.msf, FALSE, 9207ce0, TRUE)
>2012-05-31 00:54:04.758000 UTC - 0[b0f140]: 0 open DB's

Shouldn't it be 1 open DB's?
Comment 71 Ludovic Hirlimann [:Usul] 2012-05-31 00:06:13 PDT
(In reply to quantum.projects from comment #69)
> Well, the bug just happened again, but I'm not sure if I've lost messages
> this time, probably I had, but no important one.
> 
> Full log ready to deploy anywhere, just tell me how. Prefs mailed to David,
> as asked.

you could use a service like dropbox (http://db.tt/xaGolYd) and send the link to david. Or email it to david using Thunderbird Beta (we support sending very big files with it).
Comment 72 quantum.projects 2012-05-31 06:19:27 PDT
Log shared with David, and my prefs.js have been already emailed to him.
Comment 73 David :Bienvenu 2012-05-31 17:18:28 PDT
I think I figured out why the db might be getting closed during the get new mail process - if the inbox isn't open, this code:

  if (msgDBService)
    rv = msgDBService->OpenFolderDB(downloadFolder, false,
                                    getter_AddRefs(m_mailDB));

can cause the ref count on m_mailDB to go to 0 if m_mailDB is not null, through the magic of getter_AddRefs. Perhaps we don't want to make this call if m_mailDB is not null. I'll see what the pre TB 12 code did. I don't know why we would then fail to open the db, but one mystery at a time...
Comment 74 David :Bienvenu 2012-05-31 17:26:26 PDT
Pre TB 12, we got the db once at the start of delivery in nsPop3Sink::BeginDelivery. I can tweak nsParseNewMailState::Init not to re-open the db if we already have a db.
Comment 75 David :Bienvenu 2012-05-31 17:32:41 PDT
Created attachment 629009 [details] [diff] [review]
possible fix

I'll generate a try server build with this change to see if it helps...I'll post a link later tonight.
Comment 76 quantum.projects 2012-05-31 19:17:48 PDT
Nice to see that TB is coded in my favorite programming language (CPP).

So, are the first lines of my log (which, after openning the inbox, states that there are 0 DB openned)indeed showing that a problem has happened?

Also, if I understood this correctly, there was a change in the way TB 12 retrieves messages and opens db's, compared to Pre TB 12?
Comment 77 quantum.projects 2012-05-31 19:25:22 PDT
And, if my dumb guesses are of any help, could it be that an antivirus scanning the db when TB tries to open it makes the openning call fail? I don't know, like some sort of OS flag that prevents the file from being openned by two programs simultaneously?
Comment 78 David :Bienvenu 2012-05-31 21:45:14 PDT
try builds here - http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-e6fe2bb93d63/ 

There were a lot of changes in tb 12 to handle pluggable stores. I'm not sure why the db open call would fail, but I'd be really interested in hearing if these try builds help.
Comment 79 :aceman 2012-06-01 00:09:20 PDT
(In reply to quantum.projects from comment #76)
> Nice to see that TB is coded in my favorite programming language (CPP).
The backend (platform) parts are in C++. There are large parts in XUL, Javascript, CSS (mostly TB and Seamonkey UI).

Maybe you could help with some C++ tasks :)
Comment 80 David :Bienvenu 2012-06-01 11:49:38 PDT
Comment on attachment 629009 [details] [diff] [review]
possible fix

judging from a few of the logs of people having this issue, and looking at the way tb 11 worked, I think this has a good chance of helping. And it avoids an extra call to OpenFolderDB :-)
Comment 81 quantum.projects 2012-06-01 12:06:38 PDT
I'm still a novice in programming (I'm a lawyer by profession), still learning the basics, but it would be interesting to try doing some debugging in my spare time, at first. Obviously, I have a long way to go and it wouldn't be wise to blindly trust in my programming skills, but I'll be more than glad to help with anything I can.

Do you have any novice-level tasks pending?

I'll try the build you've shared. If it helps, I'll say.
Comment 82 quantum.projects 2012-06-02 06:51:42 PDT
The build isn't there, David. All there is in the win64 folder is the report of the build, stating that it failed.
Comment 83 David :Bienvenu 2012-06-02 09:04:10 PDT
our win32 builds are backed up because of some other issue. I'm told that eventually the builds will finish.
Comment 84 quantum.projects 2012-06-03 17:02:44 PDT
I'll keep checking, thanks!
Comment 85 David :Bienvenu 2012-06-06 13:29:44 PDT
win32 try server build here:

http://ftp.mozilla.org/pub/mozilla.org/thunderbird/try-builds/bienvenu@nventure.com-27f3e6591ede/try-comm-central-win32/

this is a tb 13 build with a possible fix for this, along with an SMTP fix for sending messages when the server drops the conneciton on quit.
Comment 86 quantum.projects 2012-06-07 08:01:21 PDT
Why is it installing as "Earlybird"? Isn't it going to merge with my Thunderbird account?
Comment 87 quantum.projects 2012-06-07 08:06:28 PDT
Forget what I asked. I'll try this build and report if the bug was solved by it.
Comment 88 quantum.projects 2012-06-11 08:31:33 PDT
It seems that the fix worked. It's been five days and the bug haven't happened again, while, before the fix, it was happening every day or two. 

David, thank you for the fix! I think it should be merged with the trunk, so it is made available for future releases of Thunderbird.
Comment 89 David :Bienvenu 2012-06-11 08:34:08 PDT
(In reply to quantum.projects from comment #88)
> It seems that the fix worked. It's been five days and the bug haven't
> happened again, while, before the fix, it was happening every day or two. 
> 
> David, thank you for the fix! I think it should be merged with the trunk, so
> it is made available for future releases of Thunderbird.

Thx very much for your feedback; it's extremely helpful. I'll push to get this change into tb 14.
Comment 90 David :Bienvenu 2012-06-11 08:35:02 PDT
In fact, given the dataloss issue, I think this would be worth getting into 13.0.1, if we do one, though that would require a little baking on the trunk first.
Comment 91 :aceman 2012-06-11 08:39:45 PDT
The fix is not yet merged into the tree (only in a try build) so the bug is open. But thanks for confirmation quantum.projects.
Comment 92 Mark Banner (:standard8) 2012-06-12 11:20:38 PDT
Comment on attachment 629009 [details] [diff] [review]
possible fix

[Triage Comment]
This looks fine to me, although I'm assuming we're not likely to have a case where we get a reference to a db that is closed, so r+a=me.
Comment 93 Mark Banner (:standard8) 2012-06-12 11:24:31 PDT
Checked into trunk, I'll land on branches in a few mins:

https://hg.mozilla.org/comm-central/rev/45d7d3c4c616
Comment 95 Mark Banner (:standard8) 2012-06-12 13:01:20 PDT
Landed for 13.0.1:

https://hg.mozilla.org/releases/comm-release/rev/287215a8d103
Comment 96 David :Bienvenu 2012-06-18 07:24:27 PDT
*** Bug 754508 has been marked as a duplicate of this bug. ***
Comment 97 Fred Stluka 2012-06-18 09:44:48 PDT
I'm recently being prompted to upgrade to TB 13.0.1.

Is the proposed fix for this bug included in 13.0.1?

Thanks!
--Fred Stluka
Comment 98 Mark Banner (:standard8) 2012-06-18 10:02:09 PDT
(In reply to Fred Stluka from comment #97)
> I'm recently being prompted to upgrade to TB 13.0.1.
> 
> Is the proposed fix for this bug included in 13.0.1?

See:

(In reply to Mark Banner (:standard8) from comment #95)
> Landed for 13.0.1:
> 
> https://hg.mozilla.org/releases/comm-release/rev/287215a8d103

Note You need to log in before you can comment on or make changes to this bug.