1791038 - Cannot retrieve full timezone definitions stored on events in database

Status: RESOLVED FIXED

(Calendar :: ICAL.js Integration, defect)

Description

[bjoern]

User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:104.0) Gecko/20100101 Firefox/104.0

Steps to reproduce:

I had a working Thunderbird 92 which synced my Nextcloud calendar via the CalDAV protocol in both direction. My distribution (Fedora) updated Thunderbird now to version 102.2.1.

Actual results:

Since the update the calendar gets no longer synced from the server to Thunderbird. New entries I create with Thunderbird get synced to the server just fine. But changes from the server are not synced back to Thunderbird. I also tried to remove the calendar completely and re-added it to Thunderbird. But nothing changes.

In the error console from Thunderbird I see this messages:

Unknown function icaltimezone(data) {
this.wrappedJSObject = this;
this.fromData(data);
} passed at 1: [resource:///modules/calendar/Ical.jsm:37] unwrapSingle
2: [resource:///modules/calendar/Ical.jsm:27] unwrapSetter
3: [resource:///modules/calendar/Ical.jsm:23] unwrap/<
4: [resource:///modules/calendar/calStorageHelpers.jsm:189] newDateTime
5: [resource:///modules/calendar/CalStorageItemModel.jsm:653] getEventFromRow
6: [resource:///modules/calendar/CalStorageCachedItemModel.jsm:148] getEventFromRow
7: [resource:///modules/calendar/CalStorageItemModel.jsm:175] getEvents/start/<
8: [resource:///modules/calendar/CalStorageDatabase.jsm:176] handleResultInner
Ical.jsm:37

TypeError: zone is null Ical.jsm:5520:26
convertToZone resource:///modules/calendar/Ical.jsm:5520
getInTimezone resource:///components/calDateTime.js:137
unwrapSetter resource:///modules/calendar/Ical.jsm:27
unwrap resource:///modules/calendar/Ical.jsm:23
newDateTime resource:///modules/calendar/calStorageHelpers.jsm:189
getEventFromRow resource:///modules/calendar/CalStorageItemModel.jsm:653
getEventFromRow resource:///modules/calendar/CalStorageCachedItemModel.jsm:148
start resource:///modules/calendar/CalStorageItemModel.jsm:175
handleResultInner resource:///modules/calendar/CalStorageDatabase.jsm:176
AsyncFunctionNext self-hosted:632

Calendar:
Exception { name: "NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS", message: "[JavaScript Error: "zone is null" {file: "resource:///modules/calendar/Ical.jsm" line: 5520}]'[JavaScript Error: "zone is null" {file: "resource:///modules/calendar/Ical.jsm" line: 5520}]' when calling method: [calIDateTime::getInTimezone]", result: 2153185313, filename: "resource:///modules/calendar/calStorageHelpers.jsm", lineNumber: 189, columnNumber: 0, data: XPCWrappedNative_NoHelper, stack: "newDateTime@resource:///modules/calendar/calStorageHelpers.jsm:189:19\ngetEventFromRow@resource:///modules/calendar/CalStorageItemModel.jsm:653:24\ngetEventFromRow@resource:///modules/calendar/CalStorageCachedItemModel.jsm:148:24\ngetEvents/start/<@resource:///modules/calendar/CalStorageItemModel.jsm:175:30\nhandleResultInner@resource:///modules/calendar/CalStorageDatabase.jsm:176:21\n", location: XPCWrappedNative_NoHelper }
CalStorageDatabase.jsm:190

Expected results:

I would expect the the calender continues to sync in both directions

Comment 1

[Magnus Melin [:mkmelin]]

Do you know what data that is?
As a workaround for now you can set the calendar.icaljs pref to false

Comment 2

[bjoern]

I checked the calendar today and some calendar entries are synced now. But if I add a new entry on the server the new entry doesn't get synced to Thunderbird if I trigger a re-sync. Probably if I wait another ~24hours it will show up. But that's still strange, I didn't had this kind of delays with older Thunderbird versions and the errors still look suspicious.

Maybe it is worth mentioning that I have about 20 quite large caldav calendars synced to Thunderbird. So maybe the calendar has some issues with the general size since Tunderbird 102?

I also added another fresh calendar with just one entry to isolate the failing calendar entry but it also isn't syned until now, rougly 15 min later.

I see now this additional error messages in the log (with a calender entry from one of the large calendars explicitly mentioned):

Calendar: [calStorageCalendar] Message: executeSyncItemStatement exception
Connection Ready: true
Last DB Error Number: 6
Last DB Error Message: database table is locked
Database File: /home/schiesbn/.thunderbird/nextcloud.default/calendar-data/cache.sqlite
Last DB Statement: [object AsyncStatementJSHelper]
Exception: [Exception... "Component returned failure code: 0x8052000e (NS_ERROR_FILE_IS_LOCKED) [mozIStorageStatement.executeStep]" nsresult: "0x8052000e (NS_ERROR_FILE_IS_LOCKED)" location: "JS frame :: resource:///modules/calendar/CalStorageDatabase.jsm :: executeSyncItemStatement :: line 131" data: no]
1: [resource:///modules/calendar/CalStorageDatabase.jsm:263] logError
2: [resource:///modules/calendar/CalStorageDatabase.jsm:133] executeSyncItemStatement
3: [resource:///modules/calendar/CalStorageMetaDataModel.jsm:46] deleteMetaDataById
4: [resource:///modules/CalStorageCalendar.jsm:519] setMetaData
5: [resource:///modules/CalDavCalendar.jsm:230] setMetaData
6: [resource:///modules/CalDavCalendar.jsm:984] addTargetCalendarItem
7: [resource:///modules/caldav/CalDavRequestHandlers.jsm:984] endElement
8: [resource:///modules/caldav/CalDavRequestHandlers.jsm:113] _walk
CalStorageDatabase.jsm:263

NS_ERROR_FILE_IS_LOCKED: Component returned failure code: 0x8052000e (NS_ERROR_FILE_IS_LOCKED) [mozIStorageStatement.executeStep] CalStorageDatabase.jsm:131

Calendar: Couldn't find (UTC+01:00) Amsterdam, Berlin, Bern, Rom, Stockholm, Wien 2 CalTimezoneService.jsm:206

Calendar: id is null when parsing
BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ddaysoftware.com//NONSGML DDay.iCal 1.0//EN
BEGIN:VTIMEZONE
TZID:W. Europe Standard Time
BEGIN:STANDARD
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYHOUR=3;BYMINUTE=0;BYMONTH=10
TZNAME:Mitteleuropäische Zeit
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYHOUR=2;BYMINUTE=0;BYMONTH=3
TZNAME:Mitteleuropäische Sommerzeit
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RS
VP=TRUE;SCHEDULE-AGENT=CLIENT:mailto:2_GE2DSMBUGIZTEMZWGE2DSMBUGK5YPDGGLPZ
[e-mail]
ATTENDEE;CN=firstname lastname;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT;SCHEDUL
E-AGENT=CLIENT:mailto:[e-mail]
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RSVP=TR
UE;SCHEDULE-AGENT=CLIENT:mailto:[e-mail]
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RS
VP=TRUE;SCHEDULE-AGENT=CLIENT:
CLASS:PUBLIC
DTEND;TZID=W. Europe Standard Time:20211213T160000
DTSTAMP:20211213T102040Z
DTSTART;TZID=W. Europe Standard Time:20211213T153000
ORGANIZER;CN=firstname lastname;SCHEDULE-AGENT=CLIENT;SCHEDULE-STATUS=1.1:ma
ilto:2_GE2DSMBUGIZTEMZWGE2DSMBUGK5YPDGGLPZVHIBMXEDYFODSP2X7ATP2ETSIH535KEC
foo@mailom
PRIORITY:5
SEQUENCE:0
SUMMARY:NC + FG
TRANSP:OPAQUE
UID:165406a0-44c5-410e-b5c0-29a260133e70
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
END:VEVENT
END:VCALENDAR
CalIcsParser.jsm:167

Calendar: Failed to parse item: BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//ddaysoftware.com//NONSGML DDay.iCal 1.0//EN
BEGIN:VTIMEZONE
TZID:W. Europe Standard Time
BEGIN:STANDARD
DTSTART:19701025T030000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYHOUR=3;BYMINUTE=0;BYMONTH=10
TZNAME:Mitteleuropäische Zeit
TZOFFSETFROM:+0200
TZOFFSETTO:+0100
END:STANDARD
BEGIN:DAYLIGHT
DTSTART:19700329T020000
RRULE:FREQ=YEARLY;BYDAY=-1SU;BYHOUR=2;BYMINUTE=0;BYMONTH=3
TZNAME:Mitteleuropäische Sommerzeit
TZOFFSETFROM:+0100
TZOFFSETTO:+0200
END:DAYLIGHT
END:VTIMEZONE
BEGIN:VEVENT
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RS
VP=TRUE;SCHEDULE-AGENT=CLIENT:mailto:2_GE2DSMBUGIZTEMZWGE2DSMBUGK5YPDGGLPZ
[e-mail]
ATTENDEE;CN=firstname lastname;PARTSTAT=ACCEPTED;ROLE=REQ-PARTICIPANT;SCHEDUL
E-AGENT=CLIENT:mailto:[e-mail]
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RSVP=TR
UE;SCHEDULE-AGENT=CLIENT:mailto:[e-mail]
ATTENDEE;PARTSTAT=NEEDS-ACTION;CN=firstname lastname;ROLE=REQ-PARTICIPANT;RS
VP=TRUE;SCHEDULE-AGENT=CLIENT:
CLASS:PUBLIC
DTEND;TZID=W. Europe Standard Time:20211213T160000
DTSTAMP:20211213T102040Z
DTSTART;TZID=W. Europe Standard Time:20211213T153000
ORGANIZER;CN=firstname lastname;SCHEDULE-AGENT=CLIENT;SCHEDULE-STATUS=1.1:ma
ilto:2_GE2DSMBUGIZTEMZWGE2DSMBUGK5YPDGGLPZVHIBMXEDYFODSP2X7ATP2ETSIH535KEC
foo@mailom
PRIORITY:5
SEQUENCE:0
SUMMARY:NC + FG
TRANSP:OPAQUE
UID:165406a0-44c5-410e-b5c0-29a260133e70
X-MICROSOFT-CDO-BUSYSTATUS:BUSY
END:VEVENT
END:VCALENDAR

I will try the "alendar.icalj" alter today and let you know if this helps

Comment 3

[Sean Burke [:leftmostcat]]

I haven't been able to reproduce this issue, but potentially have a fix for it. Could you extract the build at https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/a9aNn9yJQi6EpN1SvN9Ilg/runs/0/artifacts/public/build/target.tar.bz2 to your home directory and run the thunderbird executable from there? (As long as you don't overwrite your system Thunderbird install, this should create a new profile for you.) From there, you should be able to add the problematic calendar and see if the bug is still present.

Comment 4

[Martin Schröder [:mschroeder]]

The error console messages have some similarities with those from bug 1773571. Could this be a duplicate?

Comment 5

[Sean Burke [:leftmostcat]]

I believe so, but have not been able to get repro steps for either that or the bug already marked as a duplicate of it.

Comment 6

[bjoern]

Hi Sean,

I tried your build. I added the 20 calendars and right away many calendar entries where synced. From one calendar (funnily my main calendar) still most of the entries where missing but after closing and re-starting Thunderbird everything seems to be there. Maybe that's more a issue about refreshing the UI than the sync of the calendar?

Also all reminder popped up after the restart, something I just realize now that this also doesn't happen on my default Thunderbird 102 installation.

So definitely better than before where I had to wait hours until the calendar was there.

Also when I add a new entry to the calendar via the web interface of Nextcloud and trigger a sync of the calendar on Thunderbird afterwards, the new entry appears right away!

So overall your changes seems to improve it a lot! Thanks a lot for your work!

Comment 7

[Sean Burke [:leftmostcat]]

Attachment: fix-storage-timezone-usage.diff

[Approval Request Comment]
Regression caused by (bug #): unknown
User impact if declined: calendars with local storage containing custom timezone definitions will not behave properly
Testing completed (on c-c, etc.): build containing patch provided to reporter
Risk to taking this patch (and alternatives if risky): change is relatively contained and, with calendar.icaljs enabled, only affects code which is not in a working state; small risk of improper behavior with calendar.icaljs disabled

Comment 8

[Wayne Mery (:wsmwk)]

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

[Triage Comment]
Approved for esr102

Comment 9

[Wayne Mery (:wsmwk)]

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

[Triage Comment]

Comment 10

[Geoff Lankow (:darktrojan)]

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

This needs to be fixed in Daily first. Obviously there things are slightly different but that's the process.

As for this patch, it looks alright except calTimezone.js should only be imported if ical.js is in use, and only when needed.

Comment 13

[Sean Burke [:leftmostcat]]

Attachment: Bug 1791038 - eliminate stray ical.js-incompatible timezone def. r=#thunderbird-reviewers

Comment 14

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/9c7406c75816
eliminate stray ical.js-incompatible timezone def. r=#thunderbird-reviewers

Comment 15

[Geoff Lankow (:darktrojan)]

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

I'll approve this on the conditions that you include the test from the other patch, and that the fix is verified on 102 before it ships. Thomas can do that, I'd NI him but he's blocking them.

Comment 16

[Wayne Mery (:wsmwk)]

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

(In reply to Geoff Lankow (:darktrojan) from comment #15)

Comment on attachment 9295842 [details] [diff] [review]
fix-storage-timezone-usage.diff

I'll approve this on the conditions that you include the test from the other patch, and that the fix is verified on 102 before it ships. Thomas can do that, I'd NI him but he's blocking them.

[Triage Comment]
Approved for esr102

Comment 17

[Rob Lemley [:rjl]]

Attachment: bug_1791038_esr102.patch

esr102 patch with the test added

Comment 18

[Rob Lemley [:rjl]]

Comment on attachment 9298660 [details] [diff] [review]
bug_1791038_esr102.patch

[Triage Comment]
Approved previously

Comment 19

[Rob Lemley [:rjl]]

Thunderbird 102.4.0:
https://hg.mozilla.org/releases/comm-esr102/rev/0049dbc29447

Comment 20

[Thomas D. (:thomas8)]

Attachment: 1791038_testcase1.ics

(In reply to Geoff Lankow (:darktrojan) from comment #15)

I'll approve this on the conditions that you include the test from the other patch, and that the fix is verified on 102 before it ships. Thomas can do that

(In reply to Sean Burke [:leftmostcat] from comment #3)

I haven't been able to reproduce this issue, but potentially have a fix for it.

Me neither.

• I've converted one of the problem events from comment 2 into a testcase (which required some manual tweaking of newlines; I believe I got that right) - see attachment.
• Steps to reproduce are not clear to me, and probably not reduced. Since this is about the timezone problem, I don't think NextCloud is required for reproducing this.
• I've successfully imported the test event to a Google Calendar via their web interface, but it won't sync with Thunderbird's respective Google Calendar at all (both on 102.3.3 w/o this patch and on 102.4.0 candidate with this patch, Win10) - but then, we also have this issue about Google Calendar sync quota?
• I'm also getting errors even when trying to just import the testcase into a local calendar (both on 102.3.3 w/o this patch and on 102.4.0 candidate with this patch, Win10), but they don't look related to the timezone problem (see error log in comment 21, attachment 9299128 [details]). Another TB-generated ics imported alright.
  • Calendar: id is null when parsing
  • mail.import: Error: noItemsFound

Comment 21

[Thomas D. (:thomas8)]

Attachment: console-export-2022-10-19_0-43-23.txt

(In reply to Thomas D. (:thomas8) from comment #20)

• I'm also getting errors even when trying to just import the testcase into a local calendar (both on 102.3.3 w/o this patch and on 102.4.0 candidate with this patch, Win10), but they don't look related to the timezone problem. Another TB-generated ics imported alright.

Fwiw, here's the error logs of testcase 1 for the local calendar import.