682774 - Move message triggers compaction; message lost

Status: RESOLVED WORKSFORME

(Thunderbird :: Folder and Message Lists, defect)

Description

[Ian Ford]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0) Gecko/20100101 Firefox/6.0
Build ID: 20110811165603

Steps to reproduce:

Moved message that was currently being viewed by dragging to another folder.


Actual results:

The message disappeared from the current folder. A compaction started. The current folder reverted to no message selected (even though there are messages in the view). The message did NOT get moved into the dragged-to folder; in other words it disappeared permanently. When I then selected another message in the current folder, the subject of that message shown in the message list did NOT match the subject in the message pane; it showed the deleted message subject in the list. (Indicating something like the folder index was not syncd with the actual folder contents.)


Expected results:

The message should have appeared in the dragged-to folder. The message should have disappeared from the current folder, with no problem in the subject lines of the remaining messages. The compaction work should have been done without any effect on the display.

Comment 1

[Ian Ford]

Without knowing the source code, this bug suggests that the software layer that exposes the folders as indexed messages is not a complete and threadsafe layer. Somehow multiple actions are interfering with each other, or are bypassing architectural layers. In 6.0, the auto compact feature has several new and critical problems (which could be traced to the same root problem), such as de-selecting the current message.

Comment 2

[WADA:World Anti-bad-Duping Agency]

> "Move message triggers compaction" in your bug summary

This is design of auto-compact. What value do you specify as "trigger of auto-compact"? (Tools/Options/Advanced/General, Config Editor) 
> mail.purge_threshhold_mb
> mail.purge_threshhold

> The current folder reverted to no message selected

Next after Compact is known issue.
  - Sort order at a column is lost(probably reset to "ascending"==default)
"Loss of current mail selection" is perhaps a variant of phenomenon of "reset of something after Compact".

> "message lost" in your bug summary

Move mail between what kind of mail folders?
(a) Local mail folder(POP3 or Local Folders) -> Local mail folder
(b) Local mail folder -> IMAP mail folder
(c) IMAP mail folder -> Local mail folder
(d) IMAP mail folder -> IMAP mail folder
Does your "message lost" occur on any folder combination?
Was your "move mail" "move of single mail only"? Or "move of multiple mails"?

> The compaction work should have been done without any effect on the display.

As seen in Dependency tree for Bug 498274, there are known problems around Compact.

Comment 3

[Ian Ford]

Responses to questions in comment 2.

My settings:
mail.purge_threshhold_mb=1
mail.purge_threshhold=1000

Type of folders causing problem
(a) Local mail folder(POP3 or Local Folders) -> Local mail folder

Number of messages moved:
Single message move

Folder combinations:
It has happened a few times so I suspect it is any combination of source and target folders. However I'm only "testing" against real mail and no longer want to run tests on this issue, which causes the loss of the mail.

Comment 4

[halfdog]

I think I observed similar behavior also 2 times with Seamonkey-2.13a1.en-US.linux-i686-20120607 on linux, both times folder compacting caused the dataloss, not moving (moving can just trigger it). In https://bugzilla.mozilla.org/show_bug.cgi?id=740709 it was stated, that TB12 has some corruption problem, I do not know if it should affect my seamonkey release also:

The two events had that characteristics:

* Box data corruption occurred days before observed data loss (when looking into backups)
* Before data loss, some messages were corrupted, e.g. when clicking on it, the from/to headers were missing, no text in the message view pane
* The corruption is persistent between restarts (messages stay corrupted).
* When compacting, the message list contains only e.g. 2 messages (12 were lost), while the folder overview says 14 messages.
* After another restart, seamonkey corrects the number of messages from 14 to 2, hence also index is updated to lost data

Dataloss occurred only after compacting of folders, those folders with high throughput (e.g. inbox with lkml message) were more likely to suffer loss due to more frequent compaction intervall

@Mozilla-devs: due to severity of this issue: could you please create one central tracking issue, where all those diffuse data loss issues can be attached?

Comment 5

[Wayne Mery (:wsmwk)]

halfdog, can you easily reproduce this?  doing a "manual" compact?  And, did this only start happening to you recently?

Comment 6

[halfdog]

The compacting was easily triggerable, but does not occur any more.

Comment 7

[Wayne Mery (:wsmwk)]

(In reply to halfdog from comment #6)
> The compacting was easily triggerable, but does not occur any more.

WFM based on comment 6.
Thanks for the feedback