855954 - Mail data is truncated by copy mail from IMAP Offline-Use=On folder of maildirstore to Mbox of non-IMAP/maildirstore, because size of "From -", "X-Mozilla-Status:/-Status2:" is not added to messageSize

Status: VERIFIED FIXED

(MailNews Core :: Database, defect)

Description

[WADA:World Anti-bad-Duping Agency]

+++ This bug was initially created as a clone of Bug #855950 +++
This bug is spin-off of step (3) in bug 855950 comment #0.

Mail data is truncated by mail copy from IMAP Offline-Use=On follder of maidirstore to Mbox of non-IMAP/maidirstore, because size of "From -", "X-Mozilla-Status:/-Status2:" is not added to messageSize.

[Build Id] Thunderbird 17.0.2 o Win-XP.
 
[Steps to reproduce]
Copy mail from IMAP Offline-Use=On follder of maidirstore(IMAP)
to Mbox of non-IMAP/maidirstore(type=none, Local Folders).
- Offline flag of copy source mail = On
  storeToken = nnnnnnnn
- messageKey is normally incremented by 1.
- storeToken of copy source mail is used for file name.
- when same mail is copied again, file name of nnnnnnnn-1 is used.
- mail data is normally/fully copied to nnnnnnnn under "cur".
  However, messageSize of source mail is copied and used
  even though "From - ...", X-Mozilla-Status:/-Status2:, is added.
  So, "cut at messageSize" occurs in mail view and message source,
  even though entire mail data is copied to file of nnnnnnnn.
- statusOffset=0 is kept as 0.

Comment 1

[WADA:World Anti-bad-Duping Agency]

Additional problem.
  X-Mozilla-Keys: is not added upon Copy.
Without Compact, how can X-Mozilla-Keys: be written as message header?

Comment 2

[WADA:World Anti-bad-Duping Agency]

Additional observation.
- Mail data itself is not lost in cur file.
  so all data is shown by Repair Folder.
- When copy between maildirstore/local, maildirstore/IMAP/Offline-Use=On,
  copy is done by "move cur file", so no data loss occurs, and is 
  recoverable by Repair Folder.
  If copy to berkleystore/local, berkleystore/IMAP, data loss occurs,
  because messageSize is used for Copy.

Comment 3

[Steve Kelem]

This affects me too. I have my mail downloaded from google. A few messages get truncated. When this happens, the message summary line shows the Subject, From, Date, but Size=0.2K, and the message content is completely empty. Typing control-U, I get 5 lines:
From - Wed Dec 04 09:02:36 2013
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
I see the same thing in the mailbox file: no headers at all!
If I look at the source in Gmail, the message body and headers ARE there.

Comment 4

[Steve Kelem]

Oh yeah, I'm running Thunderbird 24.1.1 on Ubuntu 13.04.

Comment 5

[Wayne Mery (:wsmwk)]

sounds like this can result in dataloss

Comment 6

[Kent James (:rkent)]

In the latest trunk maildir implementation, cross server copies of emails are accomplished with a different code path than previously, so it is likely that this bug no longer exists in the current code. Would be good to test this of course.

Comment 7

[WADA:World Anti-bad-Duping Agency]

As Ketn says, many problems are already resolved by current code.
   Files are correctly created by copy/move, directory is correctly created by rename although ".sbd is not deleted so garbage incresaes" still remains,
   X-Mozilla-Keys: is added as expected by copy/move, and so on.
I'll check this bug with recent code.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Tis bug was not observed in Thunderbird Version 39.0a1 (trunk nightly on 3/15).