Closed Bug 190337 Opened 22 years ago Closed 12 years ago

[RFE] Order received lost

Categories

(MailNews Core :: Database, enhancement)

Product:
MailNews Core
Component:
Database
Type:
enhancement
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED DUPLICATE of bug 166254

People

(Reporter: jon.roland, Assigned: Bienvenu)

References

Details

User-Agent:       Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.1) Gecko/20020826
Build Identifier: Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.1) Gecko/20020826

When messages transferred from one folder to another, the order in which the
message was received is lost.

Reproducible: Always

Steps to Reproduce:
1. Set Sort By options for source folder.
2. Transfer multiple messages from source folder to target folder.
3. View target folder by order received.

Actual Results:  
Messages transferred to target folder not in same or correct order as they were
in source folder.

Expected Results:  
Maintain order received based on system time of user's system at time messages
received.

Messages not timestamped at moment they are received, so order of receipt is
represented only by the order in which the messages are appended to the mailbox
file. While this is a longstanding practice, for commercial and legal purposes
it can be important to timestamp messages and retain the timestamp as messages
are moved and archived. A timestamp line needs to be added to messages if they
are not to be stored as separate files with their own creation timestamps. The
timestamp should be encrypted to impede manual editing.

As an aside, Mozilla should include an NTP client function to maintain system
time synced to Universal Coordinated Time, to be used for timestamping messages.
it's order received in the folder that we're sorting by.
Status: UNCONFIRMED → RESOLVED
Closed: 22 years ago
Resolution: --- → INVALID
I am well aware of how it is order of storage in the folder that is currently
being sorted by. I consider that a bug, in that a timestamp is needed. Perhaps
it should be considered an RFE.

If Mozilla is to be used in commerce and legal matters, a timestamp of receipt
is critical. The lack of an ability to sort on actual date and time received
rather than just order stored, when that order can be easily disturbed, should
not be summarily dismissed as RESOLVED INVALID.
Status: RESOLVED → UNCONFIRMED
Resolution: INVALID → ---
Summary: Order received lost → [RFE] Order received lost
One consequence of not having timestamps for the receipt of messages is that
filters and other functions based on age of messages, such as movement from the
Junk folder to the Trash folder, doesn't work for messages that are misdated, as
much junk email is not. For example, I have junk email dated next year that will
sit in the Junk folder until next year unless I remove them manually.

On the other hand, when I try to use a filter to move messages from before a
certain date to the archive folder, it picks up messages misdated, such as some
dated 1969 or 1980, even though they are actually recent and not among those I
want to archive at this time.

Not all senders have system clocks accurately set. Many are way off.

I propose that a header line Rcpt Dt: be added to every incoming message with
the system date and time used to set the value (hopefully the user's system date
is set correctly, thus my proposed RFE #194929), and that sort and filter
functions be able to sort or select on the value.
Jon Roland, is there some reason why the last Received header's date couldn't be 
used?  Do you really need a whole other header being added?  This date is far 
less likely to be forged or mistaken.

If you're looking for verification suitable for legal matters, this feature 
you're asking for isn't going to cut it; you're going to need a certifying 
authority and encrypted timestamp.  It's a trivial matter to change text in the 
Inbox.

And if the receiver's clock is inaccurate, all that new header data is going to 
be off, too, just as the sender's clock can bollix up the Date header.  Your 
server's clock is far more likely to be accurate.

All of which is leading to say: I think what you want can more or less be 
handled by Bug 166254 (or the bug it's duplicating, if any).
The last Received header's date could be used as one option, but it may not
always be supported in a uniform manner by all mail servers, and will not show
when the message was actually retrieved. The option of being able to have a Rcpt
Dt: line added from the receiver's system clock is under the control of the
user's email client and will work on all platforms that support a system
timestamp. Obviously, maintaining an accurate local system time is the user's
responsibility: I normally set up a network with an NTP server that keeps all
systems in sync with each other and with the rest of the Internet. These are
secure Unix/Linux systems in which only those with root privileges could alter
the header information on message files. (One can also tee the messages or their
headers into a secure log file to maintain such integrity.)

Imagine the problem users have when they move messages around from folder to
folder and then try to figure out the order received of the messages to make
sense of who read and responded to what and in what order, and business
decisions depend on making that determination. No problem if each message is a
separate file with its own system timestamp, but when messages are concatenated
into a single large file that information is lost, and it is needed.

I strongly urge action on this request. You will appreciate it a lot better
after you have it and realize how much easier it makes doing a lot of things.

There is no substitite for having critical information under the control of the
client.
I had a further thought that we might add a line called Retrieved: with not only
the timestamp but other kinds of information on the message arising from actual
retrieval by the client, such as a checksum, MIME data, number of attachments,
etc. It could be a basis for message integrity security.
The further thought occurs that the purpose of this enhancement might be
accomplished by the client program adding a Received: line with the data for the
retrieval to the client host, as though it were a server, thus conforming to the
standard Received lines produced by other nodes along the chain. That leaves the
question of whether there is any other information gathered by the retrieval
operation that should be preserved in the message header. That may be worth some
discussion.
OK, I'm confirming this bug.  See the related Bug 123786 and Bug 166254.

One question about this scheme: how does it apply for IMAP mail accounts, where 
the messages are generally not stored on the client machine but on the server 
(kinda like a newsgroup)?  The client cannot add headers to the message on the 
server (I don't think...), so subsequent d/l's of the message would imply 
varying "received" dates.  Maybe for IMAP, the server's Received header needs to 
be used.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Severity: normal → enhancement
For IMAP or other mail accounts where the last resident location of the message
is the server then obviously the last Received date is the last date available,
since the client can't change it or add a line, but most users these days
download the messages and erase them on the server, which makes the client a
kind of server, at least a receive-only server. For purposes of verifying
receipt for these there is the return receipt, which it should be possible to
make non-optional, so that the sender could prevent the recipient from reading
the message without acknowledging receipt, and the return receipt would provide
a timestamp.

I expect the future will be for all mail clients to also be mail servers, which
I have suggested in other RFEs. This is the Unix paradigm and that is the
paradigm that it seems will ultimately prevail, if for no other reason than that
it makes no sense not to include all functions on everything.
See bug 200802, about using 'received' date to time automatic deletion of junk
mail, rather than the Date: header.
Can someone explain the new "Order Received" field values that are now part of
V1.4? What do the values represent, how are they generated, and how can they be
used to solve problems discussed in this bug? I have not been able to find any
reference to the field in any of the usually suspected places.
Blocks: 166254
In mbox format mailboxes, the From_ header can be used to timestamp receipt into
the mailbox.  In fact, mozilla mailbox files already appear to have this header
populated.  All that may be needed here is to connect that header to the UI (and
possibly preserve its value when transferring messages from one mailbox to another).
It does appear the first header line "From - " contains the timestamp of
receipt, and might serve the purpose of this RFE. What would it take to add it
to the fields on which an index is maintained (converted to yyyymmddhhmmss
order, of course), for sorting and searching, and to not change it when moving
messages from one folder to another? That would seem to be fairly easy to do.
Are there any other uses of this field that might conflict with this use?

In doing a select on equality or inequality it should be possible to specify the
time as well as the date, and a range, so one could, for example, use the Filter
to move all messages received after 2003/08/15 02:12:16 and before 2003/08/15
10:14:22.

While people are working on this, I hope that they might also enable selecting,
sorting, and filtering on Label and Flag, if not as one of the initial options,
then using the Customize functionality. See Bug 217034.
A further change that would be needed would be to make the "From - " field the
basis for the present selection and sorting on "Date received" wherever it now
appears.

As far as I know there is currently no "date received" function in Mozilla, only
"order received" which is different.  My suggestion would be rename "order
received" to "message index" or even just "#" as in Mail.app, and add a "date
received" (possibly using the From_ header).

(By the way, it is "From " or From_ header, not "From -".  The space after From
is part of the header, and the "-" is the envelope sender, which mozilla doesn't
use.)
An additional change that should be made if Date Received is defined on the
"From " header field would be to include the field in a View | Headers | All
display.

Actually, this should probably be done in any case.
Any further action on this? It is still a priority for me, as it is a constant
chore.
Product: MailNews → Core
(In reply to comment #17)
> Any further action on this? It is still a priority for me, as it is a constant
> chore.

This has been a thorn in the side of Thunderbird for years. According to https://bugzilla.mozilla.org/show_bug.cgi?id=216033 which is related, it has been annoying people and preventing widespread rollouts since 2003.

3 years later, and this problem is still not resolved.

There is a proposed solution, which reads the date from the received headers rather than trusting the sending mail client, so why is this not already applied? Is there some political reason that is holding this back?
Full ACK. - I already proposed introducing a separate column for this so that those who prefer the other way do still have that.

We have here found the workaround to configure exim to modify the sent date from the e-mail client on receive. - This is possible of course only if you use an appropriate mail server where you can influence this and the server is hosted by yourself. This situation in a lot of cases is not given.
Isn't this resoled with bug 166254 - we now have the received date column?
OS: Windows 98 → All
Hardware: PC → All
QA Contact: esther → database
From which version on should be that column? - The actual Thunderbird release (2.0.0.9) hasn't an additional column.
In the nightly builds - 3.0a1pre.
Product: Core → MailNews Core
Status: NEW → RESOLVED
Closed: 22 years ago → 12 years ago
Resolution: --- → DUPLICATE
You need to log in before you can comment on or make changes to this bug.