Open Bug 856385 Opened 7 years ago Updated 2 years ago

When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy to IMAP folder

Categories

(MailNews Core :: Database, defect)

x86
Windows XP
defect
Not set

Tracking

(Not tracked)

People

(Reporter: World, Unassigned)

Details

[Build ID]
> Mozilla/5.0 (Windows NT 5.1; rv:22.0) Gecko/20100101 Thunderbird/22.0a1

When two mails has different Subject: but same Messae-ID:, subject of second mail is set in subject and mime2DecodedSubject of msgDBHdr of mail-1 by mail copy between maildirstore/IMAP folder.

[Speps to reproduce]

(1) Two mails, different Subject:, same Message-ID:
  mail-1 : Subject: d-001
           Message-ID: <512971D6.7090603@gmail.com>
  mail-2 : Subject: d-003
           Message-ID: <512971D6.7090603@gmail.com>
  Note:
  This is mail to see Gmail's behaviour in duplicate mail detection. 

(2) Copy two mails at same time
      from maildirstore/IMAP/Offline-Use=Off folder
      to   maildirstore/IMAP/Offline-Use=On  folder
    Because Copy source folder is Offline-use=Off,
    each mail data is fetched from server, written in Target/tmp/xxxx,
    and moved to Target/cur/xxxx, with different xxxx for each mail.

(3) At thread pane, Subject of both mails is shown as "d-003".
    Header display of mail-1 in Header pane shows "d-001",
    because offline-store file is correctly created and referred. 

Test is done by Gmail IMAP, so X-GM-MSGID is saved in StringProperty. 
  mail-1 / originally d-001  : X-GM-MSGID = 1427815128571927523
  mail-2 / d-003             : X-GM-MSGID = 1427821591278214115
Different X-GM-MSGID, so different mail in Gmail is apparent. 
Mail data in final TargertFolder/cur/nnnn is as follows.
  (mail-1, file size=625)              (mail-2, file size=638)
  (file name = 1364697053992000)       (file name =  1364697054008000)
  From - Sun Mar 31 11:30:53 2013      From - Sun Mar 31 11:30:54 2013
  Message-ID: <512971D6.7090603@ ... > Same Message-ID as mail-1
  Subject: d-001                       Subject: d-003
                Different Date: header
                Some other headers are different
msgDBHdr_Of_Mail_1.offlineMessageSize=625 and msgDBHdr_Of_Mail_2.    offlineMessageSize=638 is correctly set.

Following is dump of msgDBHdr data of two message when copy from Same-ID to Offline-Use-On-02.

(Copy Source Folder = Offline-Use-Off/Same-ID)
> Mbox=Offline-Use-Off/Same-ID :  prettiestName = Same-ID, expungedBytes = 0, sizeOnDisk = 1091, All_isSpecialFolder = ( Mail,Elided,ImapBox,ImapPersonal )
> messageKey = 1, messageOffset = 1, messageSize = 539, offlineMessageSize = 0, date = 1361670614000000, dateInSeconds = 1361670614, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 512971D6.7090603@gmail.com, mime2DecodedSubject = d-001, All_StringProperty = { flags = 1, statusOfset = 0, sender = yatter king <yatter.king@gmail.com>, recipients = z@z.z.z, subject = d-001, message-id = 512971D6.7090603@gmail.com, date = 512971d6, dateReceived = 512971d6, priority = 1, msgCharSet = ISO-8859-1, size = 21b, threadParent = ffffffff, msgThreadId = fffffffe, ProtoThreadFlags = 0, X-GM-MSGID = 1427815128571927523, X-GM-THRID = 1427815128571927523, X-GM-LABELS = Junk, sender_name = 85|yatter king, keywords = , label = 0 }
> messageKey = 2, messageOffset = 2, messageSize = 552, offlineMessageSize = 0, date = 1361676777000000, dateInSeconds = 1361676777, StringProperty_pendingRemoval = , flags = 1, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = false, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 512971D6.7090603@gmail.com, mime2DecodedSubject = d-003, All_StringProperty = { flags = 1, statusOfset = 0, sender = yatter king <yatter.king@gmail.com>, recipients = z@z.z.z, subject = d-003, message-id = 512971D6.7090603@gmail.com, date = 512989e9, dateReceived = 512989e9, priority = 1, msgCharSet = ISO-8859-1, size = 228, threadParent = ffffffff, msgThreadId = 2, ProtoThreadFlags = 0, X-GM-MSGID = 1427821591278214115, X-GM-THRID = 1427821591278214115, X-GM-LABELS = [Imap]/Drafts Junk, sender_name = 85|yatter king, keywords = , label = 0 }

(Copy Target Folder = Offline-Use-On-02)
> Mbox=Offline-Use-On-02 :  prettiestName = Offline-Use-On-02, expungedBytes = 5467681, sizeOnDisk = 1091, All_isSpecialFolder = ( Mail,Elided,ImapBox,ImapPersonal,Offline,CheckNew )
> messageKey = 557, messageOffset = 0, messageSize = 539, offlineMessageSize = 625, date = 1361670614000000, dateInSeconds = 1361670614, StringProperty_pendingRemoval = , flags = 129, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = true, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 512971D6.7090603@gmail.com, mime2DecodedSubject = d-003, All_StringProperty = { flags = 81, statusOfset = 0, sender = yatter king <yatter.king@gmail.com>, recipients = z@z.z.z, subject = d-003, message-id = 512971D6.7090603@gmail.com, date = 512971d6, dateReceived = 512971d6, msgCharSet = ISO-8859-1, X-GM-MSGID = 1427815128571927523, X-GM-THRID = 1427815128571927523, X-GM-LABELS = Offline-Use-Off/Same-ID Junk, sender_name = 85|yatter king, priority = 1, size = 21b, threadParent = ffffffff, msgThreadId = 22d, ProtoThreadFlags = 0, msgOffset = 0, storeToken = 1364697053992000, offlineMsgSize = 271, numLines = f }
> messageKey = 558, messageOffset = 0, messageSize = 552, offlineMessageSize = 638, date = 1361676777000000, dateInSeconds = 1361676777, StringProperty_pendingRemoval = , flags = 129, flag_Detail = { FeedMsg = false, IMAPDeleted = false, MDNReportSent = false, Read = true, Replied = false, Marked = false, Expunged = false, HasRe = false, Elided = false, Offline = true, Watched = false, SenderAuthed = false, Partial = false, Queued = false, Forwarded = false, Priorities = false, New = false, Ignored = false, MDNReportNeeded = false, Template = false, Attachment = false, Labels = false, RuntimeOnly = false }, messageId = 512971D6.7090603@gmail.com, mime2DecodedSubject = d-003, All_StringProperty = { flags = 81, statusOfset = 0, sender = yatter king <yatter.king@gmail.com>, recipients = z@z.z.z, subject = d-003, message-id = 512971D6.7090603@gmail.com, date = 512989e9, dateReceived = 512989e9, priority = 1, msgCharSet = ISO-8859-1, size = 228, threadParent = ffffffff, msgThreadId = 22e, ProtoThreadFlags = 0, X-GM-MSGID = 1427821591278214115, X-GM-THRID = 1427821591278214115, X-GM-LABELS = Offline-Use-Off/Same-ID [Imap]/Drafts Junk, sender_name = 85|yatter king, msgOffset = 0, storeToken = 1364697054008000, offlineMsgSize = 27e, numLines = f }
Blocks: 856387
No longer blocks: 856387
Thread pane display is corrected by "Repair Folder", because message headers are reparsed by "Repair Folder".
Summary: When two mails has different Subject: but same Messae-ID:, subject of second mail is set in subject and mime2DecodedSubject of msgDBHdr of mail-1 by mail copy between maildirstore/IMAP folder → When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder
Summary: When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder → When two mails has different Subject: but same Messae-ID:, Date: etc., subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder
Summary: When two mails has different Subject: but same Messae-ID:, Date: etc., subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder → When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder
Pheomenon is peraps;
- Because of Copy, msgDBHdr is copyed from Source folder.
- Because Source folder is IMAP Offline-Use=Off,
  mail is normally fetched to maildir/local folder's Target/tmp/nnnn.
- After normal end of "uid aa:bb fetch body[]" command,
  Tb looks feched data of last mail(mail #2) only.(bug 856519) 
- Because same Message-ID:, Tb considers "fetch of mail #1 is OK",
  and msgDBHsr.suject of mail #1 is set from last mail data for mail #2.
  Then, Target/tmp/nnnn-mail-#1 is normally moved to /tmp/nnnn-mail-#1.
  Thus, phenomenon of bug 856519 doesn't occur.
- Because problem is essentially same as bug 856387,
  last /tmp/nnnn-mail-#2 is normally moved to /tmp/nnnn-mail-#2.
Depends on: 856387
Depends on: 856519
No longer depends on: 856387
Blocks: 859011
No longer blocks: maildirblockers
This bug was not mildir related problem.
Problem was observed by BerkleyStore too.
- Crafted mails : Same message-Id:, and different Subject.
- Copy many crafted mails to IMAP folder.
After copy to IMAP(uid copy or append), Tb requests "uid fetch xx:yy" for copied mails.
When multiple mails are fetched by single command, order of UID returned from server is not always ascending order.
So, when UID=1/Subject-1/Message-ID=1 and  UID=2/Subject-2/Message-ID=2 are returned in order of UID=2, UID=1, Tb seems to confuse if Message-ID is same.

This was observed by test f or Bug 1106225.
Due to that bug, "Move MM mails" is repeated DD times, so MM*DD mails are appended to an IMAP mbox by single click of a button.
In this case, many mails has same Subject and same Message-Id(duplicated mails) and many mails has different Subject but same Message-Id(crafted mails), so problem is easily observed.

Removing "maildir" from bug summary and dependency.
No longer blocks: 859011
No longer depends on: 856519
Summary: When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy between maildirstore/IMAP folder → When two mails has different Subject: but same Messae-ID:, subject of mail-2 is set in msgDBHdr.subject/msgDBHdr.mime2DecodedSubject of mail-1 by mail copy to IMAP folder
You need to log in before you can comment on or make changes to this bug.