427617 - ics import fails if event has RDATE property with value type PERIOD (worked in 0.7)

Status: RESOLVED FIXED

(Calendar :: Import and Export, defect)

Description

[Tom Pendleton]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.13) Gecko/20080311 Firefox/2.0.0.13

Import of certain .ics files produces this error:
Unable to read from file:C:\Documents and Settings\Thosp\My Documents
\Tennessee_Trails_Hike_04-12-2008.ics
[Exception... "Component returned failure code: 0x8000ffff
(NS_ERROR_UNEXPECTED) [calIRecurrenceItem.icalProperty]"  nsresult:
"0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame ::
file:///C:/Documents%20and%20Settings/Thosp/Application%20Data/Thunderbird/Profiles/sgbdalrp.default/extensions/%7Be2fda1a4-762b-4020-b5ad-a41df1933103%7D/js/calItemBase.js
:: anonymous :: line 604"  data: no]

Reproducible: Always

Steps to Reproduce:
1. Go to theleafchronicle.com click on more events, click on event click on icon that says MS Outlook.  Save as .ics in My Documents
2.  Go to Thunderbird Lightning and click on IMPORT.
3.  BINGO!
Actual Results:  
Under 0.8 nothing happens.

Expected Results:  
Event should have been add to calendar and visible, just as it did with 0.7...

Comment 1

[Tom Pendleton]

Attachment: Event File

Comment 2

[Tom Pendleton]

This file, and several others from the same source have passed ICal validation from http://severinghaus.org/projects/icv/.  I copied the text from the QuickView screen and pasted it there.

However, events from this source worked with Lightning 0.7

Comment 3

[Stefan Sitter [:ssitter]]

Confirmed. Sunbird 0.5 (20070614) and Sunbird 0.7 (20071023): File can be imported. Event is shown in view and unifinder.

Sunbird 0.8 fails during import:

Error: [Exception... "Component returned failure code: 0x8000ffff (NS_ERROR_UNEXPECTED) [calIRecurrenceItem.icalProperty]"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: file:///[...]/js/calItemBase.js :: anonymous :: line 604"  data: no]

Regression range: Works in Sunbird 0.8pre (2007-12-03-04)
                  Fails in Sunbird 0.8pre (2007-12-04-05)

Checkins during regression range: <http://tinyurl.com/5am6qr> 
Bug 400950 seems to be the only related checkin.

Question: It is an error caused by the changes in Bug 400950? Or do we now fail on some invalid .ics that was silently ignored before?

Note: The import succeeds if I remove the following line from the .ics file:
    RDATE;VALUE=PERIOD:20080412T080000/20080412T080000

Comment 4

[Daniel Boelzle [:dbo]]

I'd say the ics is valid: <http://tools.ietf.org/rfcmarkup?doc=draft-ietf-calsify-rfc2445bis-08#section-3.8.5.2>.
We've never supported PERIODs, and I suspect we've filtered those RDATEs silently out in the past, while we don't do that any more. But I need to investigate if and how we could fix this.

Comment 5

[Ulrich Luetzner]

Here is another sample with repeating dates e.g. birthdays - #1 causes error(uses "rdate") - #2 is fine(uses "rrule"):

#1 (calendar file with repeating date for next 10 years exported from Lotus Notes 8):

BEGIN:VEVENT
DTSTART;TZID="W. Europe":20040428T170000
DTEND;TZID="W. Europe":20040428T170000
TRANSP:OPAQUE
RDATE;VALUE=PERIOD:20040428T150000Z/20040428T150000Z
 ,20050428T150000Z/20050428T150000Z,20060428T150000Z/20060428T150000Z
 ,20070428T150000Z/20070428T150000Z,20080428T150000Z/20080428T150000Z
 ,20090428T150000Z/20090428T150000Z,20100428T150000Z/20100428T150000Z
 ,20110428T150000Z/20110428T150000Z,20120428T150000Z/20120428T150000Z
 ,20130428T150000Z/20130428T150000Z,20140428T150000Z/20140428T150000Z
 ,20150428T150000Z/20150428T150000Z,20160428T150000Z/20160428T150000Z
 ,20170428T150000Z/20170428T150000Z,20180428T150000Z/20180428T150000Z
 ,20190428T150000Z/20190428T150000Z,20200428T150000Z/20200428T150000Z
 ,20210428T150000Z/20210428T150000Z,20220428T150000Z/20220428T150000Z
 ,20230428T150000Z/20230428T150000Z,20240428T150000Z/20240428T150000Z
 ,20250428T150000Z/20250428T150000Z,20260428T150000Z/20260428T150000Z
 ,20270428T150000Z/20270428T150000Z,20280428T150000Z/20280428T150000Z
DTSTAMP:20080808T134701Z
CLASS:PRIVATE
SUMMARY:Hochzeitstag
UID:33157D9E2E2AF518C1256E7F00545F07-Lotus_Notes_Generated
X-LOTUS-PARITAL-REPEAT:TRUE
X-LOTUS-UPDATE-SEQ:1
X-LOTUS-UPDATE-WISL:$S:1;$L:1;$B:1;$R:1;$E:1
X-LOTUS-START;TZID="W. Europe":20040428T170000
X-LOTUS-END;TZID="W. Europe":20040428T170000
X-LOTUS-NOTESVERSION:2
X-LOTUS-APPTTYPE:4
X-LOTUS-CHILD_UID:CD6A826620A35A2EC1256E7F005473DF
END:VEVENT



#2 (repeating date generated with Sunbird):
BEGIN:VEVENT
CREATED:20080819T153936Z
LAST-MODIFIED:20080819T154053Z
DTSTAMP:20080819T153936Z
UID:8f2103b9-389f-47b1-bcf6-46f179f586ad
SUMMARY:Geb. Test 
RRULE:FREQ=YEARLY;INTERVAL=1;BYMONTHDAY=25;BYMONTH=8
DTSTART;VALUE=DATE:20080825
DTEND;VALUE=DATE:20080826
CATEGORIES:Birthday
TRANSP:TRANSPARENT
END:VEVENT

Comment 6

[Ulrich Luetzner]

Lotus Notes 8 uses RDATE on ALL anniversaries. There is no way to manually remove those RDATE records from the exported file before importing it in e.g. Sunbird. Same problem with Lightning.

Comment 7

[Martin Schröder [:mschroeder]]

Only the release driver can grant blocking-calendar0.9.

Comment 8

[Daniel Boelzle [:dbo]]

I think this is not a regression, because those RDATEs have been filtered out in previous versions, but of course dataloss. We could ignore the error, but this would silently cause dataloss on writing (e.g. subscribing to such a file) which is IMO not acceptable.
As far as I see, this requires major changes to the core code and thus won't makes it for 0.9. Sorry.

Comment 9

[Joseph Walton]

I was using Sunbird to view calendars exported from Lotus Notes so this is a regression for me. I didn't know that entries were being missed out, so I was already suffering silent data loss. Would it be possible to load the incomplete calendar, display a warning and mark it as read-only to prevent overwriting?

Is this a bug that can be pushed back to upstream libical?

Comment 10

[Daniel Boelzle [:dbo]]

(In reply to comment #9)
> already suffering silent data loss. Would it be possible to load the incomplete
> calendar, display a warning and mark it as read-only to prevent overwriting?
We're already in string freeze for 0.9.

> Is this a bug that can be pushed back to upstream libical?
AFAIK it's not a problem of libical.

Comment 11

[tsylla]

In some cases, this bug causes pretty major confusion. Currently, I use Thunderbird/Lightning in a Lotus Notes 8 environment. When someone reschedules an instance of a recurring meeting from Notes, the e-mail I receive appears to be blank. It looks like Lightning parses the ICS, and when it fails on the RDATE/PERIOD, all parsing of the message stops, so nothing shows in the message pane. The ICS does not even appear in the attachment list, so I can't detach it to munge it by hand.

Is there any hope for this to be fixed in a future Lightning?

Comment 12

[Daniel Boelzle [:dbo]]

Attachment: patch - v1

This is a patch is taking over the PERIODs' start dates, so such calendars are at least viewable then, although the end date of such RDATEs may not be correct.
On roundtrip the PERIODs are exchanged by DATE-TIMES, thus dataloss is still immanent.

Comment 13

[Philipp Kewisch [:Fallen][☁️📆]]

Comment on attachment 371727 [details] [diff] [review]
patch - v1

Looks good, r=philipp

Comment 14

[Daniel Boelzle [:dbo]]

Pushed to comm-central <http://hg.mozilla.org/comm-central/rev/aef2a3c5c542>

-> FIXED

Set up follow-up bug 489747 for proper support of end date.

Comment 16

[Philipp Kewisch [:Fallen][☁️📆]]

These bugs are likely targeted at Lightning 1.0b1, not Lightning 1.0. If this change was done in error, please adjust the target milestone to its correct value. To filter on this bugspam, you can use "lightning-10-target-move".