377986 - X-Mozilla-Keys: header(for tag) is better to be added if not-exists, because old mails doesn't have X-Mozilla-Keys: header then tag information is lost when "delete .msf & restart mailer"

Status: RESOLVED WONTFIX

(MailNews Core :: Database, enhancement)

Description

[WADA:World Anti-bad-Duping Agency]

Old mails doesn't have X-Mozilla-Keys: header( for tag. This means tag information of old mails is lost when manual .msf recovery, "delete .msf & restart mailer".
Although creation of batch type script to add blank X-Mozilla-Keys: header is very very easy for usual programmers, it's difficult for general users.
X-Mozilla-Keys: header is better to be added at somewhere if not-exist.
Candidates are ;
  tag addition/change, mail copy, mail move, compact folder
  ("rebuild index" is too dangerous, I believe.)
"compact folder" is best place?

Above is applicable to local mail folder only. Sorry but I don't know about IMAP.

Comment 1

[David :Bienvenu]

as far as I know, we do add it when you compact a folder...

Comment 2

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #1)
> as far as I know, we do add it when you compact a folder...
You are absolutely right. My SeaMonkey 1.1.1 kindly added the header by compact.
I checked with an old test folder that I never did "compact folder".
I'm very sorry for wasting your valuable time, and thanks for quick answer.

Closing as INVALID.

Comment 3

[WADA:World Anti-bad-Duping Agency]

Just a memorandum for me.

When all of X-Mozilla-Status:/X-Mozilla-Status2:/X-Account-Key: doesn't exist,
Thunderbird trunk(2007/4/13) added X-Mozilla-Status: and X-Mozilla-Status2: when first compact folder, and added X-Account-Key: when second compact folder.
Adding X-Mozilla-Status:/X-Mozilla-Status2 and adding X-Account-Key: seems to be independent, because implemented by different enhancement.

We have to be careful when test with manually created test mail, because problem such as Bug 377998 is exposed if X-Account-Key: doesn't exists in mail data.

Comment 4

[WADA:World Anti-bad-Duping Agency]

Reason why I opened this bug was that I saw some mails without X-Mozilla-Keys: in some local mail folders which is rarely compacted, and I was convinced that they were old mails when comment #1. But I was aware now that it was wrong.
It was Bug 383846. 
I usually filter my sent mails by "Run filter on folder" and move to a local folder which will rarely be compacted(I rarely delete needed mails), then the sent and moved mails stay "no X-Mozilla-Keys:" status by Bug 383846.
I should have checked deeply why "no X-Mozilla-Keys:".

Changing to DUP of Bug 383846.

Comment 5

[WADA:World Anti-bad-Duping Agency]

Loss of tag after re-build index were reported in some bugs due to lack of X-Mozilla-Keys: header. This can occur on other mails than Sent mail. (e.g. Imported mail). So I think initial request was not INVALID and is not DUP.

Changing to WONTFIX (because "Compact" always has the functionality), for ease of pointing Comment #3 for workaround procedure in other bug.

Comment 6

[WADA:World Anti-bad-Duping Agency]

(In addition to comment #3)

Because of fact of Comment #3, and because that compact is executed only when deleted mail exists, next procedure is required to generate all headers manually, if no X-Mozilla-Status:/X-Mozilla-Status2: doesn't exists.
(Read "X-Account-Key:" in Comment #3 "X-Mozilla-Keys:", please.)

(1) First Compact (for X-Mozilla-Status:/X-Mozilla-Status2:)
(1-1) Sort folder by "Order Received Column"
     (largest number in the column == highest offset in mail folder file)
(1-2) Move mail with highest "Order Received column" value to other mail folder,
      then move back to the mail folder
(1-3) Compact folder
(2) Click other mail folder and click the folder again
    (re-open, in order to avoid unwanted/unknown problem)
(3) Second Compact (for X-Mozilla-Keys:)
(3-1) Sort folder by "Order Received Column"
(3-2) Move mail with highest "Order Received column" value to other mail folder,
      then move back to the mail folder
(3-3) Compact folder

Comment 7

[WADA:World Anti-bad-Duping Agency]

See Bug 392510 Comment #3 for cases for which manual creation is needed.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Known situations that Tag is lost by Rebuild-Index because Tag is not written to X-Mozilla-Keys: header yet.

(Case-1) Local mail folder, X-Mozilla-Keys: header is not written yet
(1-A) POP3, Downloaded mail before Tb 2.0, and "Compact Folder" by Tb 2.0
      or later is not done yet (Tb 1.5 or before doesn't use X-Mozilla-Keys:)
(1-B) POP3, Sent mail by Tb before Tb 2.0.0.5, and "Compact Folder" by
      Tb 2.0 or later is not done yet
      (Tb before Tb 2.0.0.5 doesn't write X-Mozilla-Keys: for Sent mail)
(1-C) Locally copied mail from IMAP, and "Compact Folder" by Tb 2.0 or later
      is not done yet
      (Tb 2.0.0.x doesn't write X-Mozilla-Keys: for copied mail from IMAP)
(1-D) Imported mail

(Case-2) Local mail folder, X-Mozilla-Keys: exists but Tag overflow exists.
         ("Compact folder" is not done yet after Tag overflow)
 Length of X-Mozilla-Keys: is around 100 bytes, then overflow can occur.
 On any local mail folder, overflowed Tags will lost when Rebuild Index,
 if overflowed Tags are not written to additional X-Mozilla-Keys: header.

(Case-3) IMAP folder (X-Mozilla-Keys: header is not used)
 FLAG(keyword) support depends on IMAP server.
 - If IMAP server doesn't support FLAG(keyword), all Tags will be lost.
 - If IMAP server supports FLAG(keyword) for Tags partially, or limited number
      of FLAG(keyword) only, partial loss of Tag will occur.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Memo for me.
(Some comments on compact utility in Bug 315069)
> Bug 315069 Comment #6 From  David :Bienvenu   2005-11-04 07:29:10 PST
>(snip)
>  Compacting folders is the only way to get those x-mozilla-status lines into he messages. 
>  And compacting won't do anything unless at least one message is deleted.
> Bug 315069 Comment #7 From Giacomo Fonderico 2005-11-04 08:55:20 PST
>(snip)
>  Do you think is a good idea to submit a bug for "compact folder" utility?
> Bug 315069 Comment #8 From David :Bienvenu 2005-11-04 09:03:53 PST
>  We've already got a command File | compact folders that will compact all the
>  folders in a given account. The auto-compact feature will compact folders in
>  all accounts (tools | options | advanced | offline and disk space -
>  automatically compact folders when it will save xx KB. So we're unlikely to add
>  a separate utility. We'd rather spend resources making compaction more
>  automatic, and/or support mail formats that don't require compaction.

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In addition to comment #8)
> (1-C) Locally copied mail from IMAP, (snip)
Bug 378973 was found for this case.