216613 - when importing some mails' dates are set to year 2101

Status: RESOLVED FIXED

(MailNews Core :: Import, defect)

Description

[greg]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.5a) Gecko/20030728 Mozilla Firebird/0.6.1

See below for the source of a mail that had this problem.
Oddly enough, this mail also has a missing subject in the list view (although it
appears in the details pane)

Reproducible: Didn't try

Steps to Reproduce:
1.
2.
3.




Here's the source of a mail that caused the problem: (modified to preserve
anonymousity(is that a word?))
------

Return-Path: <[e-mail]>
Delivered-To: [e-mail]
Received: (qmail 6065 invoked by uid 503); 23 Jan 2003 13:59:17 -0000
Received: (QMFILT: 1.0); 23 Jan 2003 13:59:17 -0000
Received: from f73.sea2.hotmail.com (HELO hotmail.com) (207.68.165.73)
  by mx1.ovh.net with SMTP; 23 Jan 2003 13:59:17 -0000
Received: from mail pickup service by hotmail.com with Microsoft SMTPSVC;
	 Thu, 23 Jan 2003 05:59:24 -0800
Received: from 81.6.12.63 by sea2fd.sea2.hotmail.msn.com with HTTP;
	Thu, 23 Jan 2003 13:59:21 GMT
X-Originating-IP: [1.1.1.1]
From: "Some mate" <[e-mail]>
To: [e-mail]
Cc: [e-mail]
Bcc: 
Subject: Names of all the venues... from San. It's the last time... I promise. :)
Date: Thu, 23 Jan 2003 14:59:21 +0100
Mime-Version: 1.0
Content-Type: text/plain; format=flowed
Message-ID: <[e-mail]>
X-OriginalArrivalTime: 23 Jan 2003 13:59:24.0857 (UTC) FILETIME=[A30D0A90:01C2C2E7]

blablablablablablbbla

Comment 1

[Erik Putrycz]

Here is another mail that causes me the same issue.
Is there a chance to fix the date of the messages afterwards the import ?

From - Mon Jan 1 00:00:00 1965
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>
Delivered-To: [e-mail]
Received: (qmail 11967 invoked from network); 18 May 2002 09:35:33 -0000
Received: from www.nativeinstruments.de (195.179.166.9)
  by mrelay1-1.pro.proxad.net with SMTP; 18 May 2002 09:35:33 -0000
Received: by www.nativeinstruments.de (Postfix, from userid 502)
	id 1DE2F7C8A; Sat, 18 May 2002 11:35:32 +0200 (CEST)
To: [e-mail]
Subject: NI Product Registration
MIME-Version: 1.0

Content-Type: text/plain; charset=iso-8859-1

From: [e-mail]
Message-Id: <[e-mail]>
Date: Sat, 18 May 2002 11:35:32 +0200 (CEST)


Dear NI user,

blablabla

Comment 2

[Wayne Woods]

*** Bug 219331 has been marked as a duplicate of this bug. ***

Comment 3

[Wayne Woods]

Hi Greg and Erik,

Are you still reading this bug? I think this is fixed in the latest trunk
nightlies. Can you please confirm? It sounds identical to bug 219331, where I
reported my problems, because I didn't see this earlier bug.

cheers,

Wayne

Comment 4

[Horst]

Using Thunderbird version 0.7.3 (20040803).
I imported an older mail folder. 
Only Mails sent before 5/9/2001 seem to be affected. Subjects seem to be okay.

Examle Code 

From - Mon Jan 1 00:00:00 1965
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>
Received: from linux01.wekanet-team.de ([213.61.180.191]) by
mailin05.sul.t-online.com with smtp id 14309X-17DeISc; Mon, 4 Dec 2000 19:15:51
+0100
Received: (qmail 12739 invoked by uid 16); 4 Dec 2000 17:14:58 -0000
Received: from localhost (sendmail-bs@127.0.0.1)  by localhost with SMTP; 4 Dec
2000 17:14:58 -0000
X-Priority: 3
X-MSMail-Priority: Normal
Received: by wekanet-team.de (bulk_mailer v1.8); Mon, 4 Dec 2000 18:14:21 +0100
X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200
From: "PC Magazin aktuell" <[e-mail]>
To: <[e-mail]>
Subject: PC Magazin News vom 04.12.2000
Message-ID: <[e-mail]>

++ PC Magazin News +++ PC Magazin News +++ PC Magazin News ++

Dieser Newsletter hat 31322 Abonnenten (Stand: KW 48/2000)

....

Comment 5

[Wayne Woods]

By "5/9/2001", did you mean international date format (d/m/y = September, 2001)
or the American one (m/d/y = May, 2001)? The oldest mails I had were from July,
2001 and they imported fine. So if you meant the former, then I can confirm that
older mails worked for me. If the latter, then I don't have mails old enough to
test.

The few mails I had that showed up with weird dates were because the original
mail (in Outlook Express) had a bad "sent" date. Apparently OE displays the date
received and Thunderbird displays the date sent.

Maybe this bug was fixed subsequent to the official 0.7.3 release? Is there any
chance that you could try out the latest-0.8 or latest-trunk (all in
http://ftp.mozilla.org/pub/mozilla.org/thunderbird/nightly/)? That would confirm
it :)

cheers :)

Comment 6

[u157000]

Hi,

Date in Thunderbird 0.8 is 2101.02.06 6:28 (YYYY.MM.DD H:MM) after 
importing from OE6.

I'm having this problem since a long time now and this is the only 
showstopper to make the switch from OE, the problem is there 
for ~900 imported mails out of ~90000. Mostly old ones are affected, 
I can spot no other pattern. Apart from the timestamp, everything 
else (subject, sender, content) seems to be OK for these mails.

Message sources:

in Outlook Express (XP Pro EN SP2):
(probably imported from Eudora many years back, and this 
was an outgoing message of mine copied to another folder)
----------------------------------------------------------------
Received: ; Sat Dec 20 02:26:53 1997 +0200
To: "K.... B...." <koncz@b***e.hu>
From: "V.... S...." <[e-mail]>
X-Priority: 3
X-MSMail-Priority: Normal
Subject: Re: Betutipus
Cc: 
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2014.211
Bcc: 
X-Attachments: 
In-Reply-To: <[e-mail]>
References: <[e-mail]>
<[e-mail]>
X-Eudora-Signature: <Standard>
----------------------------------------------------------------

in Thunderbird 0.8 after importing:
----------------------------------------------------------------
From - Mon Jan 1 00:00:00 1965
X-Mozilla-Status: 0011
X-Mozilla-Status2: 00000000
Received: Sat Dec 20 02:26:53 1997 +0200
To: "K.... B...." <koncz@b***e.hu>
From: "V.... S...." <[e-mail]>
X-Priority: 3
X-MSMail-Priority: Normal
Subject: Re: Betutipus
Cc: 
X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2014.211
Bcc: 
X-Attachments: 
In-Reply-To: <[e-mail]>
References: <[e-mail]>
<[e-mail]>
X-Eudora-Signature: <Standard>
----------------------------------------------------------------

Brgds

Comment 7

[Wayne Woods]

Is it possible your mail had a corrupt "sent" date to begin with?

By default, OE displays date received, rather than date sent, and the mail data
that you provided didn't show the sent date at all.

In contrast, Tb shows date sent rather than date received. So if the date was
corrupt to begin with, it wouldn't have been obvious in OE, but it will be in Tb.

Comment 8

[BlackWinny]

Hi everybody


This is another occurrence of this bug :

Platform context :
- Win XP Pro SP2 French
- Thunderbird 0.9 French (20041103)


In the list of mails, after I imported them from my old Outlook Express 6
archives, this one appears as sent on 06 Feb 2101. But the real date was 12 Apr
2001.

I have a track to search the origin of the bug. See this comparizon in the
following lines. This is the original header as seen in Outlook Express :

--------------------------------------------------------------------------------
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>
Received: from apeiba.wanadoo.fr (192.168.156.17) by ms15.wanadoo.fr; 13 Apr
2001 05:10:38 +0200
X-Priority: 3
X-MSMail-Priority: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Received: from lsmtp.copernicserver.com (193.252.19.20) by apeiba.wanadoo.fr; 13
Apr 2001 05:10:38 +0200
Received: from lsmtp (localhost) by lsmtp.copernicserver.com (LSMTP for Windows
NT v1.1b) with SMTP id <[e-mail]>; Thu, 12 Apr 2001
19:40:32 -0400
Message-ID: <[e-mail]> (added by apeiba.wanadoo.fr)
From: "Copernic.com" <[e-mail]>
To: <[e-mail]>
Subject: Nouveau: Copernic 2001 v5.0
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_1D2FC_01C409F8.E30D5E20"
--------------------------------------------------------------------------------

And now this is the header of the same message after importation to Thunderbird
0.9 FR :

--------------------------------------------------------------------------------
From - Mon Jan 1 00:00:00 1965
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
Return-Path: <[e-mail]>
Received: from apeiba.wanadoo.fr (192.168.156.17) by ms15.wanadoo.fr; 13 Apr
2001 05:10:38 +0200
X-Priority: 3
X-MSMail-Priority: Normal
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165
Received: from lsmtp.copernicserver.com (193.252.19.20) by apeiba.wanadoo.fr; 13
Apr 2001 05:10:38 +0200
Received: from lsmtp (localhost) by lsmtp.copernicserver.com (LSMTP for Windows
NT v1.1b) with SMTP id <[e-mail]>; Thu, 12 Apr 2001
19:40:32 -0400
Message-ID: <[e-mail]> (added by apeiba.wanadoo.fr)
From: "Copernic.com" <[e-mail]>
To: <[e-mail]>
Subject: Nouveau: Copernic 2001 v5.0
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_1D2FC_01C409F8.E30D5E20"
--------------------------------------------------------------------------------

With an hexadecimal editor I did a binary comparizon between the old header and
the new one. The ONLY difference is the presence of this line (which I don't
know the meaning of) :

--------------------------------------------------------------------------------
From - Mon Jan 1 00:00:00 1965
--------------------------------------------------------------------------------


Best regards to all !

Jacques
(France)

Comment 9

[Andrey]

For me it is a showstopper as well. I checked every milestone build with hope
that I can import all my emails and be set, but alas... OE is my friend?

I don't think it is a very hard to reproduce and fix this bug, but I guess all
the developers are quite busy with other bugs.

tandr

(In reply to comment #6)
> Hi,
> 
> Date in Thunderbird 0.8 is 2101.02.06 6:28 (YYYY.MM.DD H:MM) after 
> importing from OE6.
> 
> I'm having this problem since a long time now and this is the only 
> showstopper to make the switch from OE, the problem is there 
> for ~900 imported mails out of ~90000. Mostly old ones are affected, 
> I can spot no other pattern. Apart from the timestamp, everything 
> else (subject, sender, content) seems to be OK for these mails.
> 
> ...

Comment 10

[u157000]

I can confirm that the bug still exist in 1.0RC1

It looks so that TB's OE import function will decide the 
date solely based on the message source. This is in sync 
with the observation that msg timestamp for incoming msgs 
will differ from OE, because OE shows the reception 
timestamp, while TB shows the sending timestamp.

Looks like the import function ignores all OE internal 
date from the .dbx file.

This might be the reason why the "Flag" field is not 
coming over either. This is the other showstopper for 
me, as I used to use Flags extensively.

IMO the solution would be to either parse the mail headers 
using a more relaxed method, by looking for date patterns 
in other lines (like "Received:"), in case there was no 
"Date:" line. Or to read out the internal OE mailbox data 
for Date and Flag information.

Brgds

Comment 11

[WADA:World Anti-bad-Duping Agency]

As seen in mail sample of comment #4 and comment #5, mail has no Date: header,
and timestamp of mail separator of Unix Mbox becomes 1965/1/1 00:00:00, and day
field in the separater is "Jan 1" instead of "Jan 01".  
> From - Mon Jan 1 00:00:00 1965
This was same on Opera when mail with no Date: header was imported from Outlook
Express, and year of 2101 was also displayed on Opera.
This indicates that Outlook Express saves "Mon Jan 1 00:00:00 1965" as date
iformation of the mail in his mail data base if the mail does not have Date: header.
This is not a problem on Outlook Express because Outlook Express uses Received:
header as "Mail Date" on displaying the mail, when Date: header is not included
in the mail (See Bug 73565 and DUPed bugs of it)
I do not know why 1965( Programer's birth year? :-) ),

As seen in Bug 73565, if mail with no Date: header is received from mail server,
Epoc Time(1970/01/01 in GMT, or 1969/12/31 in Local time) is displayed as Mail
date on Mozilla(and also on Opera). 
So I guess that the date information in mail DB of Outlook Express(1965/1/1) is
used as mail date on import, then saved in *.msf on import.
I also I guess that one of the reason why 1965 produces 2101 is ;
 - 1965/1/1 causes negative date difference, because Mozilla/JavaScript
   (ECMAScript, then Opera too) uses Epoc Time(1970/01/01)
   as the base of date calculation.

Comment 12

[WADA:World Anti-bad-Duping Agency]

Change Product/Component to Core/MailNews:Import since common issue of Mozilla
Mail&News and Thunderbird.

Comment 13

[WADA:World Anti-bad-Duping Agency]

*** Bug 272303 has been marked as a duplicate of this bug. ***

Comment 14

[WADA:World Anti-bad-Duping Agency]

*** Bug 259936 has been marked as a duplicate of this bug. ***

Comment 15

[Adam Guthrie]

*** Bug 303859 has been marked as a duplicate of this bug. ***

Comment 16

[WADA:World Anti-bad-Duping Agency]

Bug 181034 looks to be first report of this issue, although summary of Bug 181034 doesn't refer to "year 2101".

Comment 17

[Magnus Melin [:mkmelin]]

*** Bug 350286 has been marked as a duplicate of this bug. ***

Comment 18

[OMA]

I'm too having this problem. 

After importing my really big (2,60 GB) Outlook Express mail folder, I see some tens of messages are improperly dated with random years such as 1970 (most of the improper dated mail comes from this year), 1996, 1997 (my oldest messages should really be only from 1998) and also some hundreds of messages have future dates such as 2007, 2012, 2014, 2027, 2040 and 2101 (most of my "future" mail is from 2101 Smile ). Seeing all this mess, I can't make the switch yet.

It seems this bug is still not fixed even if it was reported years ago. I hope it can be fixed sometime soon. This is the only reason that prevents me from switching to Thunderbird (well, that and its known fragility with big mail folders).

BTW, I've also seen Thunderbird doesn't import any of the flags associated with messages from Outlook Express. I.e: imported messages don't have the "read" flag (all imported mail is marked as new), don't have the "red flag" sign, and don't have the "responded" flag, either. Also, it doesn't import what account the messages come from ("account" column). See bug: https://bugzilla.mozilla.org/show_bug.cgi?id=93753

I'm using the latest version released this month (1.5.0.7). I hope all these import issues are eventually fixed. 

Comment 19

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #18)
>  (A) 1970 (most of the improper dated mail comes from this year)
>  (B) future dates such as 2007, 2012, 2014, 2027, 2040
>  (C) and 2101 (most of my "future" mail is from 2101 Smile )

This bug is for 2101 case when mail of no Date: header is imported from MS Outlook Express. (your (C) looks to be this case.)

Your (A) is perhaps Bug 32216.
(See all bugs listed in dependency tree of Bug 73565 which is pointed by Comment #11.)  
And your (B) is perhaps future date in Date: header set by spammer.

Anyway, check "Date:" header of funny mails first.
And read a thread in MozillaZine forum
 ( http://forums.mozillazine.org/viewtopic.php?t=465596 )
and read bugs or articles pointed in it.
OMA, please note that bugzilla.mozilla.org is NOT support forum.

Comment 21

[WADA:World Anti-bad-Duping Agency]

mailnews.use_received_date=false(default)/true is already added in latest-trunk nightly, and user can choose mail-date(date in "Date column") from Received: instead of from Date: header. This enhancement can be a relief of this bug's case for some(I think many though) users. See Bug 341548 for detail.

Comment 22

[WADA:World Anti-bad-Duping Agency]

I've opened Bug 387387 for funny phenomenon when older year than Epoc Time(1970) in Date: header. When 1965, year of mail date became 2101. It is similar phenomenon to this bug.

Comment 25

[Phil Lacy]

Attachment: OE (outlook express) import of mails-imports timestamp

OE has timestamp in index. This is used by OE as received date. This patch uses that date in 'From - ..." msg header line.  Files don't show as 2101 or some epoch date on import.

Comment 26

[David :Bienvenu]

Comment on attachment 359688 [details] [diff] [review]
OE (outlook express) import of mails-imports timestamp

thx for working on this, Phil!

       // write out the from separator.
+      rv = !NS_OK;

better to do:
rv = NS_ERROR_FAILURE;

this patch conflicts with patch for bug 160027 - can you generate a version that will apply against the trunk?

this comment is a little confusing - I'd either just skip one of them, or say that nsLocalFileWin.cpp also has this code.

+// copied from mozilla/xpcom/io/nsLocalFileWin.cpp
+// copied from nsprpub/pr/src/{io/prfile.c | md/windows/w95io.c} :

If I understand this fix, it's basically to create a proper From envelope header with the right date, so the local mail folder parser code will use that in case the Date: header is missing. Is that right?

Comment 27

[Phil Lacy]

Attachment: OE (outlook express) import of mails-imports timestamp

TB will read this first line 'FILE - ...' for a receive date stamp.  OE records this date when it is received by OE. Prior to fix TB recorded this date as yr 1965 date and it was shown as such in folder pane as 2105.

Import of msgs goes to a separate folder in TB e.g. 'outlook express import'
Also, until recent changes, if the folder was selected in tree it would rebuild and show contents. Now you have to rebuild the index. I think this is a bug unless we are still working on this area.
Import of OE (and maybe other readers) has always formed invalid .msf files. I'm pretty sure it is a result of the creation and import codes are on different threads.

Comment 28

[Phil Lacy]

Comment on attachment 360860 [details] [diff] [review]
OE (outlook express) import of mails-imports timestamp

be sure and check for a separate import folder. and you have to rebuild the index. attached is email to try.

Comment 29

[Phil Lacy]

Attachment: email without date for trial

download this email and drag to OExpress inbox. The date shown will be current date and time. Import with this patch will show that date. No date is in header of msg.

Comment 30

[David :Bienvenu]

I checked in a fix for the having to rebuild index problem this morning...I'll try this patch today, thx!

Comment 31

[David :Bienvenu]

Comment on attachment 360860 [details] [diff] [review]
OE (outlook express) import of mails-imports timestamp

thx very much for the patch and the test case. I've checked this in, with some minor formatting cleanup...

Comment 32

[David :Bienvenu]

fix checked in.

Comment 33

[WADA:World Anti-bad-Duping Agency]

(In reply to comment #27)
> Prior to fix TB recorded this date as yr 1965 date
> and it was shown as such in folder pane as 2105.

"Future if before epoc time" is bug 387387.