791925 - Tags are lost by deletion of .msf, if tags are added at IMAP folder and mail is moved from IMAP folder to local mail folder, unless overflow of X-Mozilla-Keys: happens and Compact writes all tags in expanded X-Mozilla-Keys: header.

Status: NEW

(MailNews Core :: Networking: IMAP, defect)

Description

[WADA:World Anti-bad-Duping Agency]

Tags are lost by deletion of .msf, if tags are added at IMAP folder and mail is moved from IMAP folder to local mail folder, unless overflow of X-Mozilla-Keys: happens and Compact writes all tags in expanded X-Mozilla-Keys: header.

This is new pheomeno after fix of bug 691770(fixed in Tb 10, adds X-Mozilla-Keys: header upon copy from IMAP to local in order to make room for newly added tags after the copy), and this bug is for a case which was not covered by bug 691770. 

To force write of tags(which were added while mail was held in IMAP folder) in X-Mozilla-Keys: header, one of following is curretly needed.
(A) Remove the tags(added while mail is held in IMAP folder), then add the tags again.
(B) Delete X-Mozilla-Keys: header in mail data, Repair Folder, copy some mails to the local mail folder and delete them, Compact.
(C) Add many tags to the mail in order to force overflow of X-Mozilla-Keys: header. When some tags are not written in X-Mozilla-Keys: header, copy some mails to the local mail folder and delete them, Compact.

Comment 2

[Kent James (:rkent)]

The issue that I have with this is that we don't really support "deletion of .msf" as a completely safe operation. We went to great lengths a few years ago to preserve the information in the .msf file in operations that clobber it (folder rename for example) so that local information could be stored there. That involves creating a backup copy of the .msf, and migrating the information to the new .msf  There is also other data stored there in addition to tags that we don't want to lose. There are plenty of other files that people could delete that would also completely mess up the operation of Thunderbird, so why should we allow deletion of these files? Yes years ago that was the fix, but that is no longer the case. I suggest that people who are doing this as a "fix" stop doing it.

Now I don't mind trying to take heroic efforts to preserve the information in the db file whenever possible, but the "dataloss" occurred when the user, outside of the program, decided to delete the .msf file.

I'm not quite sure how to express this in the bug metadata, but I'll take my best shot.

Comment 3

[WADA:World Anti-bad-Duping Agency]

(A) "Tag is not written in X-Mozilla-Keys:" condition occurs when;
(A-1) This bug's case : mail copy/move from IMAP to local.
(A-2) Overflow of X-Mozilla-Keys: (no room to write tag in X-Mozilla-Keys:)
(B) Not-writen-yet tags are written to X-Mozilla-Keys: header when;
(B-1) If (A-1), Overflow of X-Mozilla-Keys: happens and Compact happens.
(B-2) If (A-2), Compact happens(overflow already happened).
(C) Delete of .msf occurs by;
(C-1) Problem like Bug 498814.
(C-2) User's intentional .msf file delettion.

Problem of this bug is:
Because of (A-1), and because of (B-1), if (C-1) or (C-2) happens before both "overlow of X-Mozilla-Keys:" and "Compact of the local mail folder" occurs, "Tags added while mail was kept in IMAP folder" is lost.

For (A-2)/(B-2), we consider "loss of tag by delete .msf" as "current restriction in design/implementation", because there is no way to avoid "loss of tag by delete .msf"(never impossible, but many changes are required).
However, in this bug's case, main cause is (A-1) and (B-1).

We can easily reject "loss of data by delete of .msf" as INVALID or WONTFIX, because (C-1) is simply a bug which should be resolved by the bug and because (C-2) is not legal operation.
But I beleve we need to reduce "loss of data by manual delete of .msf" cases as many as possible if it's not so tough work like new feature implementation or big design change.

Comment 4

[Kent James (:rkent)]

"But I beleve we need to reduce 'loss of data by manual delete of .msf' cases as many as possible if it's not so tough work like new feature implementation or big design change."

Yes I agree with that at some level, as long as we realize that we do not claim you can delete .msf files with no dataloss.

Comment 5

[Ludovic Hirlimann [:Usul]]

Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.