Sorting by Received Column is Wrong - it sorts by Date instead (last/top-most Received: header doen't have optional day-of-week field of timestamp)

RESOLVED WORKSFORME

Status

RESOLVED WORKSFORME
9 years ago
5 years ago

People

(Reporter: gjunk, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Whiteboard: resolved_by_Tb_3.0.3)

Attachments

(2 attachments)

(Reporter)

Description

9 years ago
User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2a1pre) Gecko/20090601 Minefield/3.6a1pre
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1pre) Gecko/20090601 Lightning/1.0pre Shredder/3.0b3pre ID:20090601030621

I get mail (usually spam) which has date/time in the Date field which is very different than the time received by the last server (as shown by date/time in Recevied Header).

Click on Received column to sort by received date - and the sort is incorrect - it still sorts by Date field. 

Reproducible: Always

Steps to Reproduce:
1. Get message in INBOX which gas a Date field older than Last received
2. Sort by Received
3.
Actual Results:  
Messages ar sorted by Date instead 

Expected Results:  
Sorted by date/time in moist recent Recevieved Header
Gene can you attach such message to the bug ?
Component: General → Mail Window Front End
QA Contact: general → front-end
(Reporter)

Comment 2

9 years ago
I will try send one later if you really need one (cant at the moment) - but to see this all you need to do is take any message where the headers are essentially correct - by this I mean the most recent Received and the Date field are approx correct. Choose one that is not the most recently received message - then save it as a file - edit the Date field and set the Date to the future (say Dec 2009).

Put it back and it will rise to be the most recently received - even tho the received date says otherwise. 

You can just as easily change the Date to the past of course. The received sort uses the Date header instead of the received one.

Mmm .. i seem to no longer be able to find where to import an mbox file into TB - I could swear i've used this ... I'm sure you can find a way to get into a mail folder tho. The import tool wants something called a 'communicator' file .. a bit ancient that is ... ;-)
IMAP? POP3?
If IMAP, DUP of bug 402594. (Too hard to reach bug 402549 by search?)
If POP3 or local mail folder, rebuild-index may be required for already downloaded mails.
(In reply to comment #2)
> I will try send one later if you really need one (cant at the moment) - but to
> see this all you need to do is take any message where the headers are


Please do. And check wada's bugs to make sure yours is different.

Updated

9 years ago
Duplicate of this bug: 514328

Comment 6

9 years ago
(In reply to comment #5)
> *** Bug 514328 has been marked as a duplicate of this bug. ***

I created the above duplicate bug, because I missed this bug report when I first searched.

However, in my report I give an example of a message header that produces the problem (in TB3b3), if that is of any help.
Torrance can you answer wada's questions ?

Comment 8

9 years ago
I am using the default Gmail setup, so IMAP. I haven't tested to confirm that the problem does not exist in a POP account.

And even though I'm not using POP, I rebuilt the index anyway, but to no avail.

Comment 9

9 years ago
I can confirm this behaviour with Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.5) Gecko/20091204 Thunderbird/3.0

Using a POP3 inbox, all my messages have the same Received and Date column entries.

Here are the important headers of a mail I received yesterday

From - Fri Jan 08 11:18:11 2010
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:
Return-Path: <xxxx@xxxxx-xxxxxxx.xx>
X-Flags: 0000
Received: (qmail invoked by alias); 08 Jan 2010 09:53:06 -0000
Received: from xxxxxx.xxxxxxxxxxxx.xx (EHLO xxxxxx.xxxxxxxxxxxx.xx) [999.999.999.999]
  by xxx.xxx.xxx (mx115) with SMTP; 08 Jan 2010 10:53:06 +0100
Received: from xx-xxx-xx.xxxxx.xx (xx-xxx-xx.xxxxx.xx [99.999.999.999])
	by xx.xxxxxxxxxxxx.xx (node=mxeu5) with ESMTP (Nemesis)
	id 0LyUCI-1Nwb5g0vnq-015uLR for xxxxxxxxxxxxxxxxxx@xxxxxxxxxxxx.xx; Fri, 08 Jan 2010 10:53:06 +0100
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; t=1262944386; l=570509;
	s=domk; d=xxxxx-xxxxxxx.xx;
	h=List-Unsubscribe:Date:Reply-To:MIME-Version:Content-Type:To:Subject:
	From:X-RZG-CLASS-ID:X-RZG-AUTH;
	bh=WaXn2k5etQ3gus/7clcB6PymFHM=;
	b=ZNfDIhdhSVxwh/JNw7K+L3qJDH5+ks1lIR9qNfYAEQxN3TaP68Fs3OSp2lP4aQ8E6f2
	rjf57t8+bxjcrrNCPpZjno38YgpXCPdjHVQddZfnrWtsW3okLwCGxCZQAqtja+YQ/AMQ+
	Vto24bW8G4JdXyuyVt+enmvLTpZuUSU3mro=
X-RZG-AUTH: :ImkTZmCleviQVPztV+pBHRWt/IqjsjF8J0xflLlyZxdCbE/3Bcwc1YctlsaUOiM=
X-RZG-CLASS-ID: mo00
Received: from xxxxx1
	(xxxxxxxxxxxxxxxxxxxx.xxxxx.xxxxxxxx.xxx [99.99.999.999])
	by xxxx.xxxxxx.xx (fruni mo22) (RZmta 22.6)
	with ESMTP id R018a3m088rR4X for <xxxxxxxxxxxxxxxxxx@xxxxxxxxxxxx.xx>;
	Fri, 8 Jan 2010 10:52:33 +0100 (MET)
From: <xxxx@xxxxx-xxxxxxx.xx>
Subject: xxxxx xxxxxxx - Newsletter 01. - 2010
To: xxxxxxxxxxxxxxxxxx@xxxxxxxxxxxx.xx
Content-Type: multipart/related;
	boundary="----=_NextPart_687_7666_76568706.F6A66662"
MIME-Version: 1.0
Reply-To: xxxx@xxxxx-xxxxxxx.xx
Date: Sun, 8 Aug 2010 10:52:27 +0200

Expected behaviour: Date column = 8 Aug 2010, Received column = 8 Jan 2010
Actual behaviour:   Date column = 8 Aug 2010, Received column = 8 Aug 2010
Four Received headers.
> Received: (qmail invoked by alias); 08 Jan 2010 09:53:06 -0000
> Received: from (snip)    with SMTP; 08 Jan 2010 10:53:06 +0100
> Received: from (snip)             ; Fri, 08 Jan 2010 10:53:06 +0100 DKIM-Signature: (snip)
> Received: from xxxxx1 (snip)      ; Fri, 8 Jan 2010 10:52:33 +0100 (MET)

First/second one has wrong or obsolete timestamp format.
Tb doesn't fall back to next valid Received: header?
Or all Received: headers has wrong timestamp format?

Can you check with next header as top(last) Received: header or second header? 
> Received: from a@x.y.z by x.y.z with ESMTP id 52AC333CE5D
>  for <b@x.y.z; Fri, 08 Jan 2010 10:53:06 +0000 (GMT)
Copy the mail in new mail folder, edit mail folder file, restart Tb.

Comment 11

9 years ago
That's it!

Your timestamp gets recognized correctly as Received. It seems, Thunderbird is choking on the missing day of the week.
(The last Received: header is inserted by my mail provider. This explains why my whole Inbox does show the same time for Date and Received.)

I see two issues here:

1. Thunderbird should fall back to the next valid Received: header.

2. Thunderbird should be more tolerant regarding the time format. My mail provider is GMX. According to their website, they have 8.52 million unique users per month from Germany. Not all of them using Thunderbird with POP3 of course, but the number of Thunderbird users affected by this issue should be quite high.
(In reply to comment #11)
> I see two issues here:
> 1. Thunderbird should fall back to the next valid Received: header.

AFAIK, MS Outlook Express 6 falls back to next, if corrupted timestamp or no timestamp in Received: header.
Tb may believe that nearest server is most accurate. But in your case, nearest server is worst one on timestamp in Received: header.

> 2. Thunderbird should be more tolerant regarding the time format.

Timestamp format is invalid or obsolete? If obsolete, it may be reasonable request, but I thunk the format is invalid.

There is apparently third/biggest problem.
3. Your nearest server sends Received: which is not compliant with current RFC.
As error is lack of "day_of_week," only, I think it's merely a result of incorrect setup of Qmail by your provider.
Summary: Sorting by Received Column is Wrong - it sorts by Date instead → Sorting by Received Column is Wrong - it sorts by Date instead (last/top-most Received: header has malformed timestamp)
Status: UNCONFIRMED → NEW
Ever confirmed: true

Comment 13

9 years ago
>> 2. Thunderbird should be more tolerant regarding the time format.
> Timestamp format is invalid or obsolete?

No, the Received: date by my provider is correct. Thunderbird should be fixed.

RCF 2821 specifies "date-time" and refers the reader to RFC 2822. There it is defined as
  date-time = [ day-of-week "," ] date FWS time [CFWS]

So, according to the specification, day-of-week is optional and Thunderbird should recognise notations like "08 Jan 2010 09:53:06 -0000" correctly.
Summary: Sorting by Received Column is Wrong - it sorts by Date instead (last/top-most Received: header has malformed timestamp) → Sorting by Received Column is Wrong - it sorts by Date instead (last/top-most Received: header doen't have optional day-of-week field of timestamp)
FYI.
Tb processes omitted [ day-of-week "," ] in Date: header properly. Why common routine is not used for timestamp of Date: and Received:?

Comment 15

9 years ago
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3

Apparently the "Order Received" column is sorting the messages correctly. So how is this calculated if TB is choking on parsing the Received: field?

Comment 16

9 years ago
It seems to be fixed now with Mozilla/5.0 (Windows; U; Windows NT 6.0; de; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3

The example I posted above works now. However, for the mails received a while a ago, I had to delete the .msf file in order to get correct results.
(In reply to comment #16)
> I had to delete the .msf file in order to get correct results.

Does it mean "Folder Properties/Rebuild Index didn't work as expected"?

Comment 18

9 years ago
> Does it mean "Folder Properties/Rebuild Index didn't work as expected"?

I have to admit that I did not try that. Firstly, I was not aware of that option. Secondly, that option has a wrong translation into German (translated as "restore/undelete index", Bug 555172).
(In reply to comment #16)
> rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3
> The example I posted above works now.

I could confirm it, by pasting example you posted to mail data in local "Inbox" file(as file size is increased by me, rebuild-index is internally executed during restart of Tb automattically).
Someone looks to have resolved problem.

Closing as worksforme.



  .
Status: NEW → RESOLVED
Last Resolved: 9 years ago
Resolution: --- → WORKSFORME
Whiteboard: resolved_by_Tb_3.0.3

Comment 20

9 years ago
Created attachment 435302 [details]
"Received:" vs "Date:" in OE6 and TB3.0.3

Is this really "resolved"? As you can see from the screenshot, the "Received" date shown in TB is the same as the message's "Date" field. This is wrong since the actual "Received" date as displayed correctly by OE is earlier than the "Date" of the message. I'm running TB 3.0.3

Comment 21

9 years ago
Created attachment 435305 [details]
Message header

This is the header source of the message. The "Received" column is displaying the wrong field.

Comment 22

9 years ago
To be clear: I've verified Martin's claim that this has been fixed *ONLY* if you click on the "Rebuild Index" on the Local Inbox. This does not work for IMAP folders. In addition, having to click "Rebuild Index" to get TB to display the correct date should be considered a "work around" and not an actual "Fix".

Comment 23

9 years ago
Sorry, it's been a long day... didn't notice Bug 402594. My apologies.
You need to log in before you can comment on or make changes to this bug.