751821 - Dates values before 1970 are corrupted in Storage calendar items since Lightning 1.4

Status: RESOLVED FIXED

(Calendar :: Provider: Local Storage, defect)

Description

[Christoph]

User Agent: Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20100101 Firefox/12.0
Build ID: 20120420145725

Steps to reproduce:

Just as usual I synchronized my Android cellphone with MyPhone Explorer from FJ Software after updating to TB 12.01 and Lightning 1.4.


Actual results:

All yearly occurring days (birthdays/weddingdays etc) before 1.1.1970 produce an corrupted date. More users report this problem in the forum of FJ Software and found out that this happened with dates before 1970. See here (in German): http://www.fjsoft.at/forum/viewtopic.php?t=16470

I found another post on the German sunbird-kalender.de forum regarding an unclear error depending on pre-1970 dates: http://www.sunbird-kalender.de/forum/viewtopic.php?f=13&t=3781 (youtube-Video showing the problem).


Expected results:

Synch dates as usual!

Were there any changes made in 1.4 that could lead to the problem?

Comment 1

[Christoph]

I just downgrded to TB 11 and Lightning 1.3. Result:

DATA CORRUPTION IN THE ICS-FILE!

It's getting even worse! All recurring dates before 1970 are now corrupted in my ics file! All dates in the ics file have now a year with 1970 or 1971 the day and month is still OK. This is getting critical!

Comment 2

[derekund]

I have Tbird 12.0.1 and lightning 1.4
I too have recently started to have issues with recurring dates (birthdays) corrupting to negative months and years. A recent May date appeared as December 8 -35, -25164. 
This has lost me inportant personal information. I AM NOT Happy
I see the latest Lightning is now shown as 1.3! Nobody said the 1.4 was a trial or has it been withdrawn.
Also needing to set up a separate address to report bugs is a pain.
Mozilla ia starting to get like Microsoft and Apple it seems. Keep the users in the dark and feed them Sh*t

Comment 3

[derekund]

I can't be sure when this happens but it may be related to when reminders activate.
Not all dates pre 1970 are affected at once, but it does seem to be older dated events (like Birthdays) which are affected. Hope this helps

Comment 4

[Matthew Mecca [:mmecca]]

(In reply to derekund from comment #3)
> I can't be sure when this happens but it may be related to when reminders
> activate.
> Not all dates pre 1970 are affected at once, but it does seem to be older
> dated events (like Birthdays) which are affected. Hope this helps

@derekund: Are you also using MyPhoneExplorer?

Comment 5

[Thomas.Weinreich]

Hi, I have also problem with some series dates (birthday's) before 1970. The problem existed since Lighning 1.4 and it still remains with Lighning 1.5.1. with Thunderbirg 13.0.1

The problem was triggered after I have manually changed a birth **year** in Thunderbird to 1950 an start after this a sychronisation to my Mobile Device an then to the Calender (Mobile, then Ligning 1.4) in the folling way:

1. Entry of Birthday data in the Thunderbird address book
2. Sync address book with mobile device address book (Application Birdisync)
3. Create series entries in the mobile calendar by the application "Manila Birthday"
4. Sync with PC (to Lightning)

*** AFTER a restart of Thunderbird the dates in Lighning are  incorrect with an incorrect start and end date ***

The Problem can be reproduced and is also tested from the birdysync support team, to reproduce the problem: Manually create a yearly event with a date before 1970 in Thunderbird - Restart Thunderbird

Are you working on the correction of the bug already and can we expect a correction in the next release?

Best Regards
Thomas

Comment 6

[Chris]

The problem is not specific to BirdieSync (tests were done with BirdieSync extension disabled).

To reproduce the problem:

 - Manually create a yearly event with a date before 1970 in
Lightning (for instance: 12h30 - 13h30, 12th of June 1962, Yearly)
 - Restart Thunderbird
 The event has now an incorrect start and end date.

If you look at Lightning database, the value of the stored "event start" and "event end" columns of the event before Thunderbird restart are incorrect: "1.84465056529096e+19" and "1.84465056565096e+19"

Could you have a look at this problem ? It impacts all users who store birthdays in Lightning database (and want to synchronize them with their mobile device).

Thanks a lot.

Comment 7

[Matthew Mecca [:mmecca]]

(In reply to Chris from comment #6)
> To reproduce the problem:
> 
>  - Manually create a yearly event with a date before 1970 in
> Lightning (for instance: 12h30 - 13h30, 12th of June 1962, Yearly)
>  - Restart Thunderbird
>  The event has now an incorrect start and end date.

I still haven't been able to reproduce this, for me the dates still remain correctly set at 1962. Can you reproduce this with a new clean Thunderbird profile with only Lightning installed?

Comment 8

[Chris]

Tha

Comment 9

[Chris]

(In reply to Matthew Mecca [:mmecca] from comment #7)

Thanks a lot for looking at this problem.
I tried with a new empty profile (with only Lightning installed and Test Pilot) and still could reproduce the problem.
Note that the problem doesn't occur if you use a day event. You must set a start and end time (like 12h30 - 13h30).

Comment 10

[Merike Sell (:merike)]

Something doesn't float quite right with such events. Using steps in comment 6 got me:
RECURRENCE-ID;TZID=Europe/Tallinn:20120612T123000
DTSTART;TZID=Europe/Tallinn:20120612T123000
After restart it had changed to:
RECURRENCE-ID;TZID=Europe/Tallinn:20120612T122959
DTSTART;TZID=Europe/Tallinn:20120612T122959
This was with 1.7a2 from 20120615.

In 1.3 the after restart result is
RECURRENCE-ID;TZID=Europe/Tallinn:20120612T124750
DTSTART;TZID=Europe/Tallinn:20120612T124750
and in 1.4 it is
RECURRENCE-ID;TZID=Europe/Tallinn:19620612T122959
DTSTART;TZID=Europe/Tallinn:19620612T122959
so whatever causes it seems to mess up less since 1.4

Comment 11

[Matthew Mecca [:mmecca]]

(In reply to Matthew Mecca [:mmecca] from comment #7)
> (In reply to Chris from comment #6)
> > To reproduce the problem:
> > 
> >  - Manually create a yearly event with a date before 1970 in
> > Lightning (for instance: 12h30 - 13h30, 12th of June 1962, Yearly)
> >  - Restart Thunderbird
> >  The event has now an incorrect start and end date.
> 
> I still haven't been able to reproduce this, for me the dates still remain
> correctly set at 1962. Can you reproduce this with a new clean Thunderbird
> profile with only Lightning installed?

I can reproduce this on Windows 7 but not on Ubuntu. After restart there are multiple instances of:

Timestamp: 6/27/2012 6:57:55 PM
Error: uncaught exception: [Exception... "Component returned failure code: 0x80070057 (NS_ERROR_ILLEGAL_VALUE) [nsIScriptableDateFormat.FormatDate]"  nsresult: "0x80070057 (NS_ERROR_ILLEGAL_VALUE)"  location: "JS frame :: resource://calendar/modules/calUtils.jsm -> file:///C:/Users/.../extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/calendar-js/calDateTimeFormatter.js :: formatDateLong :: line 148"  data: no] 


I also tried adding non-repeating events with a start date in 1962, none of these appear in the unifinder with the "All Events" filter selected or in the views after restart.

Comment 12

[Andreas Schott]

The problem with dates before 1970 is just related to a restart of Thunderbird.
I have 13.0.1 and Lightning 1.5.2.

I created a new calendar with one entry and exported:

BEGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
BEGIN:VEVENT
CREATED:20120629T140202Z
LAST-MODIFIED:20120629T140229Z
DTSTAMP:20120629T140229Z
UID:d1280b1d-5f75-4f4e-bffd-a5739a832ff1
SUMMARY:Alexandra *1961
RRULE:FREQ=YEARLY
DTSTART;VALUE=DATE:19610616
DTEND;VALUE=DATE:19610617
TRANSP:TRANSPARENT
END:VEVENT
END:VCALENDAR

I closed and restarted thunderbird. Now the export shows:

EGIN:VCALENDAR
PRODID:-//Mozilla.org/NONSGML Mozilla Calendar V1.1//EN
VERSION:2.0
BEGIN:VEVENT
CREATED:20120629T140202Z
LAST-MODIFIED:20120629T140229Z
DTSTAMP:20120629T140229Z
UID:d1280b1d-5f75-4f4e-bffd-a5739a832ff1
SUMMARY:Alexandra *1961
RRULE:FREQ=YEARLY
TRANSP:TRANSPARENT
END:VEVENT
END:VCALENDAR

Thus simply the DTSTART and DTEND dissappeared, which means
there is no longer any information about the birthday.
This update really messed up most of all my birthday
entries, which I now have to find again in some written
paper documentation.

Comment 13

[Matthew Mecca [:mmecca]]

(In reply to Matthew Mecca [:mmecca] from comment #11)
> I can reproduce this on Windows 7 but not on Ubuntu.

I take that back, on Ubuntu (32 bit) dates before 1970 are 1 second off, while on Win 7 (64 bit) the date is completely corrupted.

Comment 16

[Stefan Sitter [:ssitter]]

For your information: The problem with dates before 1970 was identified as a bug that was introduced in Thunderbird 12. 

Lightning 1.6 - that was released yesterday for Thunderbird 14 - contains a workaround for the Thunderbird bug. Therefore it should be possible to correctly create events with dates before 1970 again. 

Events that were broken using the previous Thunderbird 12 and 13 release currently stay broken until a fix is available. You could try to workaround by editing and recreating the events.

Comment 17

[Matthew Mecca [:mmecca]]

Attachment: Fix v1

Converts the incorrect unsigned nativeTime values back to negative before converting to a calDateTime.

Some results have a rounding error of 1024 microseconds that rounds to a full second off the converted date value, I'm not sure if there's any way around that.

Comment 18

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Comment on attachment 653232 [details] [diff] [review]
Fix v1

Review of attachment 653232 [details] [diff] [review]:
-----------------------------------------------------------------

Maybe you can truncate the microseconds instead of rounding? I guess since its currently not possible to create a date so far into the future anyway, so this won't distract anything. We should get rid of this code in a few years though since the original bug is fixed :)

Comment 19

[Matthew Mecca [:mmecca]]

Attachment: Fix v2

This should fix the microsecond errors.

Comment 20

[Matthew Mecca [:mmecca]]

Pushed to comm-central - http://hg.mozilla.org/comm-central/rev/28fdfdda0bae