341548 - date/time shown in msg list differs from one shown in msg header pane (After a patch for Bug 166254, trunk always works as if mailnews.use_received_date=true, then "mail date" is always obtained from Recieived: header instead of traditional Date: header)

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Message Display, defect)

Description

[Nelson Bolyard (seldom reads bugmail)]

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9a1) Gecko/20060605 SeaMonkey/1.5a

I received an email that was delayed in transit by about a day.
The date/time shown in the message list pane differs by over 18 hours
from the date/time shown in the message header pane.  
Both dates/times should be the same, and should match the date/time
in the message's Date: header line.  

The date/time shown in the message list pane appears as

  Subject                              Sender       Date         S...  Size
  Crypto code in Mozilla outside NSS   Wan-Teh ...  08:20        Read   9KB

The Message header pane appears as 

  Subject:   Crypto code in Mozilla outside NSS
     From:   Wan-Teh Chang <xxxxxxxxxxxxxxxxxx>
     Date:   2006-06-13 13:49
       To:   Mozilla NSS Developers <xxxxxxxxxxxxxxxxx>

That Date does indeed match the date in the message header.
I will attach the full message with headers.

Comment 1

[Nelson Bolyard (seldom reads bugmail)]

Attachment: email message that shows two dates in SM 1.5a

This is the email message (complete, with headers) which SeaMonkey 1.5a 
shows with two different dates.

Comment 2

[Nelson Bolyard (seldom reads bugmail)]

Perhaps more important than the fact that the date appears wrong in the 
message list pane is the fact that the message appears in the wrong order
in the message list pane.  Other messages (replies to it) that were sent 
after it appear before it in the message list pane because its date/time is 
wrong in that pane.

Comment 3

[Nelson Bolyard (seldom reads bugmail)]

I sometimes receive multiple copies of an email, where my email address was
listed as "To:", and my email address is also a mailing list that was "Cc'ed".
It's a single message, with a single message ID, but I get two copies of it.

In Seamonkey's email, these two copies of the same message show different 
time stamps in the email message list pane.   This is clearly wrong.

Examples:

Subject                                  Sender        Date              ...
Re: PayPal Security Flaw demoed with...  Bob Lord      2006-06-16 15:09  Read
Re: PayPal Security Flaw demoed with...  Bob Lord      2006-06-16 15:11  Read
Re: PayPal Security Flaw demoed with...  Daniel Veditz 2006-06-16 18:30  Read
Re: PayPal Security Flaw demoed with...  Daniel Veditz 2006-06-16 18:32  Read

Comment 4

[David :Bienvenu]

you've looked at the Date: header in each message in the message source and they're the same?

Comment 5

[Nelson Bolyard (seldom reads bugmail)]

Attachment: headers of the 4 messages from comment 3

Here are the full headers from the 4 messages cited above.
For each pair, the Date: header is the same in both copies.

In the first pair, neither of the two dates shown in the message list pane
matches the date given in the message's Date: header.  The message list pane 
is apparently taking its dates from some other header alltogether.

Comment 6

[Nelson Bolyard (seldom reads bugmail)]

After examining each of the attached messages, and comparing them to the
dates displayed in the message list pane, it appears that the date being
displayed is being consistently taken from this header:

X-OB-Received: from unknown (192.168.9.229)
  by 192.168.9.132; 16 Jun 2006 22:09:43 -0000

I do not know the source of that header, but IMO it is the wrong source
for that date.  The Date: header is the right source.

Comment 7

[Nelson Bolyard (seldom reads bugmail)]

(In reply to comment #6)

> I do not know the source of that header, but IMO it is the wrong source
> for that date.  The Date: header is the right source.

That wasn't very clear.  I meant to write:

I do not know the source of that header, but IMO that header is not the 
place from which our mailnews client should be getting the date to display
in the message list pane.  The Date: header is the right source.

Comment 8

[David :Bienvenu]

see bug 166254 for a fix.

Comment 9

[Nelson Bolyard (seldom reads bugmail)]

Or perhaps this is a duplicate of 166254?

Comment 10

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #9)
> Or perhaps this is a duplicate of 166254?

No, not DUP of Bug 166254.
Following is short story, story of long term though.

Changing of "Date column" meaning from "Date: header" to "Received: header" is a result of David's first patch of Bug 166254 for testing.
> Bug 166254 Comment #22 From David Bienvenu 2005-10-16 20:33 PDT
> Created an attachment 199777 [details] [diff] [review]
> proposed fix
> OK, use the first received: header. And if there's no Date: or Received: hdr,
> use PR_Now()

After first patch, David proposed second patch.
> Bug 166254 Comment #66 From David Bienvenu  2005-12-05 14:15 PDT
> Created an attachment 205084 [details] [diff] [review]
>  only used received: hdr
>    if no Date: header,
>    or user sets hidden pref
> (This second patch will introduce mailnews.use_received_date(default=false))

After this second patch, 
(A) When user doesn't set mailnews.use_received_date=true,
      date in Received: header is used, only if no Date: header.
      This will resolve problem of Bug 73565.
      This doesn't change meaning of current "Date column". 
(B) When user sets mailnews.use_received_date=true,
      patch wiil change meaning of "Date column" to "date in Received: header".
      This will resolve problem of Bug 73565, Bug 32216 and some other bugs
      which complaint about old date or future date set by spammer.
      This will partially satisfy request of Bug 166254, new "Received: column"
      request, although user has to accept loss of "Date: header column" by
      setting mailnews.use_received_date=true.   

Because only first patch is active on trunk, (B) is effective always(no check for mailnews.use_received_date is made). However, super reviewer of second patch still rejects approval of David's second patch.
(Super review for second patch was requested in 2005/12, but no super review for more then 9 months. I can't say other than "rejection" in this case.)

(end of short story)

By the way, because only first patch is active on trunk, (1) "Date column" display becomes date in Received: header on trunk(problem you are looking).
This means that (2) date value of mail(probably held in ".msf") also becomes date in Recieved: header.
As a result of (1), I needn't see future date at date column of mail list pane, and as a result of (2), spam of future Date: header is deleted shortly by usual Junk Purge setting such as "Delete after 14 days".
So I think, if users set mailnews.use_received_date=true intentionaly, many users can accept loss of "column for Date: header" like me.

Comment 11

[Nelson Bolyard (seldom reads bugmail)]

Wada, Thanks for your explanation.

I reported this bug against SeaMonkey because that is what I use.
Is it actually a bug common to both SeaMonkey and Thunderbird?

Comment 12

[Nelson Bolyard (seldom reads bugmail)]

As I understand it, this bug actually has TWO undesirable effects:
a) the wrong date appears in the Date column of the message list pane,
AND
b) the messages are sorted in the wrong order in the message list pane
when sort-by-date is selected; that is, the messages are sorted by the 
date that appears in the message list pane, which is the date received,
even though the list is supposedly sorted by date, not by "order received".

Comment 13

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #12)
> a) the wrong date appears in the Date column of the message list pane

Sorry for my misdirection by my Comment #0.
Please read my Bug 166254 Comment #90.
 - The patch doesn't change meaning of "Date column in mail list pane". 
 - The patch simply changes original header for "mail date(I call it) in .msf" ;
    A) Before patch       : "mail date" is obtained from "Date: header"
    B) After first patch  : "mail date" is obtained from "Received: header"
    C) After second patch : user can choose A) or B) by setting hidden pref  
And please be aware that ;
 1. Column names is "Date", is NOT "Date:".
 2. Column names for value from From:,To: etc. are "Sender"/"Recipient".   
 3. In contrast to 1/2, mail header box in message pane displays
    real mail header names (From:, To:, Date:, Subject: and so on)  

As you say, we, including me, are convinced that "Date column in mail list pane" means "Date: header" for long long time, but it was not accurate.
Please note that "wrong date" will not be displayed in message list pane even after patch, the "mail date" is always displayed correctly. 

Request of multiple time-stamp related columns for sorting, one for "Date: header value" and one for "Received: header value", is different requirement from David's objective of his second patch.
See Bug 166254 Comment #93 and my Bug 166254 Comment #94 for it, please.

By the way, super review of the second patch has done at last! 

Comment 14

[WADA:World Anti-bad-Duping Agency]

David has landed the second patch on 2007-04-17. Congratulations!

Comment 15

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #12)
> a) the wrong date appears in the Date column of the message list pane,
Additional comment to your "wrong date".
As far as I know, "mail date" in .msf is set on mail receive time only. So if you have old mails, old mails have "mail date" from Date: header and new mails have "mail date" from Received: header, or vice versa when user changed mailnews.use_received_date setting from true to false.
This discrepancy will be resolved by .msf recreation, "delete .msf and restart Thunderbird". "Rebuild Index" from property will have same effect, I think.

Comment 16

[Nelson Bolyard (seldom reads bugmail)]

OK, I'm confused.  I just downloaded and tried a "nightly" trunk build 
that contains David Bienvenu's second patch for bug 166254.
I haven't set the new mailnews.use_received_date preference yet, but
have verified that it defaulted to false.  So, I have made no change 
other than trying newer code.  

Today, when I look at the list of messages in my "mozilla-security" 
folder (the same list previous shown above in comment 3), the dates 
shown there are now the dates from the Date: header.  I see 

Subject                                  Sender        Date              ...
Re: PayPal Security Flaw demoed with...  Bob Lord      2006-06-16 15:10  Read
Re: PayPal Security Flaw demoed with...  Bob Lord      2006-06-16 15:10  Read
Re: PayPal Security Flaw demoed with...  Daniel Veditz 2006-06-16 18:30  Read
Re: PayPal Security Flaw demoed with...  Daniel Veditz 2006-06-16 18:30  Read

I'm happy to see this.   But I thought that it was going to be necessary 
to set the new mailnews.use_received_date preference to TRUE to get this
result.  But in another folder, I still see the received date in the message
list.  So, given that my mailnews.use_received_date preference is false,
why did the contents of the message list (shown above) change?

Comment 17

[Nelson Bolyard (seldom reads bugmail)]

I just rebuilt the summary file for my "mozilla" folder, which was another 
folder that was showing messages in the wrong order because of using 
received dates.  Now it's show the messages in the right order, and using
the Date: dates in the date column of the message list.  

The good news.  This is exactly what I want!!
But I was expecting to need to set mailnews.use_received_date preference to 
TRUE to get this result, and I haven't done that yet.  (I guess I won't! :)

Comment 18

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #17)
> (I guess I won't! :)
Don't worry about. I'm a one who loves sort by "received date", and I'll surely use it because I have already line of user_pref("mailnews.use_received_date", true); in my user.js. :)

Comment 19

[WADA:World Anti-bad-Duping Agency]

Modifying summary, in order to explain your initial problem clearly(caused by first patch on trunk), and to close this bug as FIXED(fixed by second patch).

Comment 20

[WADA:World Anti-bad-Duping Agency]

This bug is caused by first patch for Bug 166254, and is resolved by landing of second patch for Bug 166254 on 2007-04-17.
mailnews.use_received_date=false/true works as expected now, and mailnews.use_received_date=false is defaulted.

=> FIXED by second patch for Bug 166254

Comment 21

[WADA:World Anti-bad-Duping Agency]

Changing to VERIFIED according to Comment #17.

Comment 22

[Nelson Bolyard (seldom reads bugmail)]

OK, I'm not confused any more.  I had understood the sense of the new 
preference to be the opposite of what it really is.  But now I have it right.
So, I agree, it's fixed.  Hallelujah!  Thanks to all who helped!