1203934 - Corruption/data loss when moving collapsed message thread between folders

Status: RESOLVED WORKSFORME

(Thunderbird :: Folder and Message Lists, defect)

Description

[Chris Newton]

User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0
Build ID: 20150826023504

Steps to reproduce:

(Thunderbird 38.2.0 on Windows 7 64-bit.)

1. During an ongoing mail exchange, receive a new message in the Inbox, which was otherwise empty.

2.  Send a reply to that message.

(In my case, the reply message is automatically saved under Inbox as well, via the Copy Sent to Current extension. At this point, the Inbox contains both messages and everything appears to be working normally.)

3. With the display threaded, collapse that thread so two messages are now showing as a single non-expanded line in the message list pane.

4. Drag that line to another folder where earlier mails from the same discussion are already stored.

For obvious reasons I can't reproduce the exact bug, but I have now seen apparently related data loss twice within a few days.


Actual results:

From a UI point of view:

In one instance, only the second message (my reply) was added to the target folder. The first message appears to have vanished without trace.

The thread also appears broken in this case, i.e., existing messages that were already in the target folder show up in one thread, and then the new message that did copy in starts a second thread in the list.

In a second instance a few days later, both messages did appear in the target folder, but the first message was completely empty when opened.

More specifically in this case, the correct details did appear in the threaded folder view for that message. Selecting each message in the thread in turn would show the expected information in the message pane as well, until that first copied-in message, at which point the message pane would show no message body and would still show the header details and an attachment from the previous message in the thread. Similarly selecting the messages in the thread in turn, starting from the end and working backward, showed a blank message body and didn't change the headers from the last message shown.


From a files-on-disk point of view:

In the first instance, the missing message is completely absent from the relevant folder's file in the profile directory.

In the second instance, the second message (the one that did show up as expected in the folder/message panes) has been appended to the relevant folder's file in the profile directory twice. The first message is again completely absent from the data file.

The data for the two entries in the file is identical in the two cases (as compared with a developer's diff tool) except that the first copy has the line

X-Mozilla-Status: 0019

while the second copy begins with

X-Mozilla-Status: 0001

Unfortunately I had already tried dragging the two messages back to the Inbox and then back to the other folder again before I thought to check the message file in that case, so I can't confirm where in the process this duplication happened.

I also tried saving the empty message from the UI, which gave a completely empty (0 bytes) .eml file.

It's not immediately obvious to me where the correct metadata for the missing message is coming from to show it in the threaded folder view. If Thunderbird maintains that information in some sort of separate index then that index does appear to have been updated correctly in the second instance of the bug, though not the first.


Expected results:

In each instance, both messages should have been appended to the existing thread in the target folder.

Comment 1

[Chris Newton]

In an amusing little irony, without thinking that the entire target folder might now be corrupt rather than just those two discussion threads, I then dragged the mail from Bugzilla that I used to set up this account into the same folder.

That mail from Bugzilla now appears in the threaded folder view right in the middle of the thread from the second instance of the bug that I described above, instead of in its own separate thread where you'd expect it to be. The empty message that previously had correct metadata showing in the folder pane has now completely disappeared.

If anyone has any suggestions for how to preserve the other data in that folder and prevent any further corruption, that would be appreciated as well...

Comment 2

[Wayne Mery (:wsmwk)]

Chris does this still reproduce for you when using a current versoin?

Comment 3

[Chris Newton]

I'm not sure I can contribute much useful new information, as unfortunately this one wasn't 100% reproducible even when I first reported it. I saw it a couple of times within a short period, so I thought a bug report might prompt someone familiar with the code to spot something like a race condition or locking problem that hadn't previously been identified, but without a confirmed root cause I'm not sure there is much to be done here.

For what it's worth, I can confirm that I haven't seen any further instances of the issue since the original bug report, but for obvious reasons I've also been avoiding the action I described that seemed to be the common trigger for the corruption. I've definitely forgotten a few times and no harm seems to have resulted, but again, it didn't happen consistently before either.

Sorry I can't be more help.

Comment 4

[Wayne Mery (:wsmwk)]

(In reply to Chris Newton from comment #3)
> I'm not sure I can contribute much useful new information, as unfortunately
> this one wasn't 100% reproducible even when I first reported it. I saw it a
> couple of times within a short period, so I thought a bug report might
> prompt someone familiar with the code to spot something like a race
> condition or locking problem that hadn't previously been identified, but
> without a confirmed root cause I'm not sure there is much to be done here.

investigation by pure code inspection is rarely productive or a good first course of action, unless the potential area of code is quite narrow.
 
> For what it's worth, I can confirm that I haven't seen any further instances
> of the issue since the original bug report, but for obvious reasons I've
> also been avoiding the action 

Perfectly understable you'd want to avoid it. 

Your comments are helpful. Consistent with your current experience, let's assume the problem is gone. Please reopen the bug if you see it again.