939519 - When saving/exporting messages as .eml files, should use date/time of internal "Date:" MIME header for "Modified" timestamp property of the file (not the date/time of exporting)

Status: NEW

(MailNews Core :: Backend, enhancement)

Description

[Thomas D. (:thomas8)]

+++ This bug was initially created as a clone of Bug #766073 +++

(Dissected from bug 766073)

STR:

1. Select an old msg, e.g. having this MIME header in msg source:
Date: Sun, 09 Sep 1999 09:00:00 +0100
2. Ctrl+S, save as .eml (or drag & drop into OS file system)
3. From OS file manager, open .eml file properties and look at "Modified" date/timestamp

Actual result:

- "Modified" timestamp shows the date and time when you saved/exported the msg (today's date):

SavedMessage.eml file properties (in OS file manager):
Created:  Sunday, November 17, 2013, 11:00:00 AM
Modified: Sunday, November 17, 2013, 11:00:00 AM

Expected result:

- "Modified" timestamp of the .eml file should show the original date of the message, i.e. date/time as contained in the original "Date:" header of the MIME message source (according to bug 766073, comment 0, that's also what OE6 does)

SavedMessage.eml file properties (in OS file manager):
Created:  Sunday, November 17, 2013, 11:00:00 AM
Modified: Thursday, September 09, 1999, 9:00:00 AM

From bug 766073, comment 0:
> The file Modified Date should be EXACTLY the one specified in the "Date: "
> [header] line in the MIME source, AT THE SECOND, and ACCORDING TO THE TIME
> OFFSET specified in the MIME source following http://tools.ietf.org
> /html/rfc5322#section-3.3 RFC5322 §"3.3.  Date and Time Specification".

Assuming that X-Mozilla-Headers (like tags) don't count as a content-relevant modification, and we don't maintain a timestamp for that anyway, I'd agree it looks reasonable, useful, perhaps "more correct" to use the internal MIME Date: header for the file's "Modified" date property.

"Modified" file property usually reflects when the *content* of the file was last modified (vs. "Created" which reflects when the file was created on that particular file system). For email messages, the relevant content is the original message which has an internal timestamp aka Date header.

This would allow users to determine the true age of saved messages from their file manager, and to group/sort/compare the saved messages (.eml files) by their "real" age of content (as found in date header) via their "Modified" timestamp properties, which is helpful especially for multiple saved messages in list view. It might also be helpful to identify duplicate .eml files.

Otoh, the date/time when the message was /exported/ from TB is rather random and thus probably not very important to users, and is still somewhat preserved in the "Created" timestamp of the file.

Comment 1

[Thomas D. (:thomas8)]

Per anecdotal evidence, also seen on getsatisfaction:
https://getsatisfaction.com/mozilla_messaging/topics/drag_drop_mails_in_eml_format_from_thunderbird_to_a_random_folder_on_my_pc#reply_3135748

> Yes!! this [drag & drop of multiple messages into file system] would be extremely useful. I also would
> like to see the time stamp in the file attributes correspond to the message's time of reception,
> instead of the time of creation of the eml file.
> anybody know a hack to achieve this?
> thank you!!

Comment 2

[Magnus Melin [:mkmelin]]

Attachment: proposed fix

Comment 3

[Magnus Melin [:mkmelin]]

Fixes the save as case. I don't think it fixes d'n'd, but can't drag to desktop on linux yet, so i can't test it.

Comment 4

[:Irving Reid (No longer working on Firefox)]

Comment on attachment 8343914 [details] [diff] [review]
proposed fix

Review of attachment 8343914 [details] [diff] [review]:
-----------------------------------------------------------------

Well, the code looks fine, and it works with 'Save' from the menu on Mac OS X, but my trunk/debug build crashes when I try to drag/drop so I can't test that.

The funny thing is, I disagree with the fundamental premise of this bug (and the vehement complaint in bug 766073) - I believe that the file-system level time stamp should represent when that file was modified *in the file system*, not an application-specific idea of when the semantic contents of the file were modified.

That said, I don't feel strongly enough about that to override a consensus that we *do* want this behaviour.

I'm waffling between r+, and giving f+ and requesting support (and tests) for drag/drop before we land. If you want to compromise on having a follow-up bug for drag/drop support, we could go with that...

Comment 5

[Kent James (:rkent)]

Sorry to have been so slow to look at this.

I'll have to say, agreeing with Irving, that I would not normally expect the last modified date of a file to reflect application-specific concepts like the value of the Date header. Although that might be useful to some people who are trying to use a file folder as a kind of messaging application to sort by Date, that usage seems to me to be rather specific to a small subset of users. For most users, who would expected normal file behavior, that would be viewed as unusual behavior. Also that cannot be maintained consistently, as any other application that touched that file would change the modified date. Also, upon creation, the modified date would be older than the creation date, which also seems weird to me.

If this was my call, I would simply say no, as when there is no generally agreed answer, I would prefer to keep the existing behavior unchanged. But I don't believe this really should be my call, as it is more of a user experience issue.

I'll record this by giving a feedback-. If others with more user experience perspective want to overrule this and request that this move forward, I would be happy to do a review of the code.

Comment 6

[Magnus Melin [:mkmelin]]

The things that would be pro this is
 - we have the precedence from Microsoft products (apparently)
 - general usefulness if you export messages and roughly remember when it was sent
 - there is still the created time of the file, so there is fairly little other use of the modified time

The testing of this got stuck as we need to set the modified timestamp when the changes have been done = in the listener we normally use to see the test was done, and we can check things. So making that testing possible would appear to to need other significant code changes.

Comment 7

[Michel Merlin]

Inside the File System, Created is for Envelope, Last Modified is for Contents
~--~--~--~--~--~--~- ~--~--~--~--~--~--~- ~--~--~--~--~--~--~- ~--~--~--~--~--
This is NOT your or my choice, it is how it has worked for decades for everyone, in UNIX, Windows, etc:

- At the right moment a *record* (a file) is *created*, it gets the corresponding "Created" date
- At the right moment a *Content* is *modified*, it gets the corresponding "Last Modified" date
- When a file is copied, the Content is unchanged, hence the LMD (Last Modified Date) neither. And the "Created" date remains unchanged if the record is existing (hence NOT "created"), else gets a new "Created" date.
- As a consequence, and as Windows users have observed for decades, the "Created" date of a file can be older, or equal, or newer, than its "Last Modified" date.

More detailed discussion in Bug 766073 https://bugzilla.mozilla.org/show_bug.cgi?id=766073#c8 cmt8 "Created and Last Modified dates are for Envelope and Contents" and cmt10 "Created, Modified, Accessed dates in Windows Explorer, EXIF and MIME (sent) dates".

Versailles, Wed 16 Apr 2014 17:20:30 +0200

Comment 8

[Michel Merlin]

Message contents vs envelope in Thunderbird, Outlook Express or File System
~--~--~--~--~--~--~--~--~- ~--~--~--~--~--~--~--~--~- ~--~--~--~--~--~--~--
The recall in my previous comment reduces the question in this thread (Bug 766073 + Bug 939519) to this: is TB (or OE) part of the FS? Does it matter? what matters isn't it the actual info conveyed by the email message? Let's see:

In OE, messages are contained in "folders"; e.g. the "Newsletter AZNot Apr 2014" message is in the "Inbox" folder.

In WE (Windows Explorer), OE's "folders" are seen, but as *files* rather than as "folders"; and messages are NOT directly visible, only by dragging a msg from OE's message pane into WE do you get that message in the form of an EML file; there is no way AFAIK to get OE's "folders" as WE folders.

This makes the user want that file handling in WE and OE get unified; not so hard to do, unfortunately MS strategic plans have become incompatible with OE's further development. Anyway these mechanics don't change the ground: dragging a msg from OE (or TB) message pane into WE can be seen, either as creating a new file inside Windows FS (if OE is seen as an external part of the FS), or as copying info from an internal part of Windows (OE msg pane) into another one (WE), whence the 2 different ways of handling the "Last Modified" Date: either keeping the one in the msg (the MIME "Date"), or creating a new one at the moment the dragging is released.

My opinion is that after handling email in Windows for long enough, most users think the EML file should take the MIME "Date" as its "Last Modified" date. A way to please everyone would be IMO to write a small Windows addon or applet, as detailed in Bug 766073 https://bugzilla.mozilla.org/show_bug.cgi?id=766073#c11 cmt11 "An addon to WINDOWS to redate files to their internal Date". Thanks to anyone for their POV.

Versailles, Wed 16 Apr 2014 17:21:20 +0200

Comment 9

[Kent James (:rkent)]

Magnus, how can we get more opinions on this? Maybe bwinton?

Comment 10

[Magnus Melin [:mkmelin]]

Comment on attachment 8343914 [details] [diff] [review]
proposed fix

Review of attachment 8343914 [details] [diff] [review]:
-----------------------------------------------------------------

Let's ask. Blake, what do you think? Pro arguments are in comment 6, the main argument against appear to be that people may want the file modification date to always be just be modification date of the physical file. The file creation date is still the physical file date, what's discussed is the last modification date.

Comment 11

[Blake Winton (:bwinton) (:☕️)]

Comment on attachment 8343914 [details] [diff] [review]
proposed fix

So, I have _never_ seen a modification date prior to a creation date, and indeed if I did, I would strongly suspect that someone had a bug somewhere…

I could probably accept a patch that made the creation date the date from the message (since that's kind of when the content was created, as opposed to when it was written out, which would be the modification date), but that's sort of the opposite of what's being asked for, and so if Michel would prefer the current state to my proposal, I would also be happy to leave things the way they are.

Comment 12

[Magnus Melin [:mkmelin]]

There is no method (at least no mozilla method) for setting the created date. Maybe it's a hack even for windows to do so, as microsoft also used the modified date for this purpose.

Comment 13

[Thomas D. (:thomas8)]

(In reply to Blake Winton (:bwinton) from comment #11)
> Comment on attachment 8343914 [details] [diff] [review]
> proposed fix
> 
> So, I have _never_ seen a modification date prior to a creation date, and
> indeed if I did, I would strongly suspect that someone had a bug somewhere…

So you _never_ copied a file in Windows Explorer? Touching the Created date while preserving the (then older) Modification Date is default behaviour there...

Comment 14

[Blake Winton (:bwinton) (:☕️)]

I wasn't looking at creation dates much in Windows, and when I was, I wasn't looking at modification dates…

Comment 15

[Michel Merlin]

I posted a test for Created (physical) vs Modified (Contents)
~--~--~--~--~--~- ~--~--~--~--~--~--~- ~--~--~--~--~--~--~--~
(In reply to Magnus Melin from comment #10)
> people may want the file modification date
> to always be just be modification date of the physical file.
(In reply to Blake Winton (:bwinton) from comment #11)
> So, I have _never_ seen a modification date prior to a creation date, and
> indeed if I did, I would strongly suspect that someone had a bug somewhere...
May I suggest you, Magnus and Blake, to read my Comment 7 "Created is for Envelope, Last Modified is for Contents", carefully, particularly its 1st line ("This is NOT your or my choice, it is how it has worked for decades for everyone, in UNIX, Windows, etc"), and to execute by yourselves the test in its link (Bug 766073 https://bugzilla.mozilla.org/show_bug.cgi?id=766073#c10 cmt10). Then you probably will join "Thomas D." Comment 13 and me.

Versailles, Thu 17 Apr 2014 02:01:00 +0200

Comment 16

[Thomas D. (:thomas8)]

(In reply to Blake Winton (:bwinton) from comment #14)
> I wasn't looking at creation dates much in Windows, and when I was, I wasn't
> looking at modification dates…

Blake, thank you for looking into this. You may want to have a look at M$'s official "Description of NTFS date and time stamps for files and folders", which describes in detail how the Created and Modified Attributes are handled for various scenarios of moving or copying files.

http://support.microsoft.com/kb/299648

> "In all examples, the modified date and time of a file does not change unless a property of the
> file has changed. The created date and time of the file changes depending on whether the file was
> copied or moved."

I think this confirms what Magnus, Michel and I laid out in favour of this RFE:
Created = Date of the file's "envelope" creation
Modified = Date of last change to file's "contents" (or "properties of the file")

Note that e.g. changing the read-only attribute, or renaming file does NOT touch the Modified datestamp (considered envelope change), but changing the file's Title property does (considered content change).

Some more points and questions in favour of this RFE:

1) What's the point of recording the /same/ date (when user archived messages from TB into the file system) in /two/ properties, Created /and/ Modified? So per this RFE, those other users can still just use the Created datestamp, and we can use Modified datestamp for knowing the actual age of contents (as Magnus suggests in comment 6).

2) What's the main intention of saving messages outside TB as .eml files? (I suppose it's archiving). How do people usually sort their archives?

3) What's the point of knowing the date of archiving at all? Compare: When in 2014, user archives a message dated 2010 within TB, we create a folder "2010" (based on content date) and not "2014" (date of archiving)!? (If we shouldn't create "2014" folder for messages dated 2010, then we probably shouldn't make the Modified file date the date of archiving, either...)

4) Due to the default behaviour of date stamps in Windows (as described in link above), a common way of verifying the identity of files is this: Files having same size AND same Modified datestamp are usually exactly identical. Suppose you're archiving a thread of messages with MIME date of 2010 into .eml files on your OS. Say you copy the first 5 msgs of that thread in 2013, and the next 5 in 2014. If we take date of archiving as Modified file date, messages of the same thread will look as if they are 1 year apart - how does that help? Or for whatever reason you end up copying the same messages into their OS file system more than once. If we only record the date of archiving (current behaviour), it will be very confusing to have several versions of the very same message which look different (per their Modified datestamp) but are actually the same.

In a nutshell, I see a number of tangible advantages for the behaviour proposed in this bug (at least for Windows systems), whereas I'm not sure if there's any convincing argument in favour of the current behaviour.

Comment 17

[Thomas D. (:thomas8)]

(In reply to Thomas D. from comment #16)
> In a nutshell, I see a number of tangible advantages for the behaviour
> proposed in this bug (at least for Windows systems), whereas I'm not sure if
> there's any convincing argument in favour of the current behaviour.

Assuming that users copy/export messages into their file system as .eml files for the purpose of *archiving*, the main advantage of this RFE is obviously laid out in comment 0:

(In reply to Thomas D. from comment #0)
> This would allow users to determine the true age of saved messages from
> their file manager, and to group/sort/compare the saved messages (.eml
> files) by their "real" age of content (as found in date header) via their
> "Modified" timestamp properties, which is helpful especially for multiple
> saved messages in list view. It might also be helpful to identify duplicate
> .eml files.
> 
> Otoh, the date/time when the message was /exported/ from TB is rather random
> and thus probably not very important to users, and is still somewhat
> preserved in the "Created" timestamp of the file.