986936 - No update to ICloud caldav possible since March 2014

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect, P1)

Description

[AWA]

User Agent: Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; MDDRJS)

Steps to reproduce:

For more than a year I used TB/Lightning to read and update my icloud calendar. Since a few days Lightning tells me with yellow rectangle that the calendar is momentarily not available.
It's configured as caldav with location like https://p01-caldav.icloud.com/1234567890/calendars/home/. User and Password are my Apple-Id with password. 

I tried a beta version TB 28.0 / Lightning 3.0b1 with the same result.



Actual results:

When I start TB, Lightning is able to read the calendar, but after the first update TB disables it. TBs Error Console reports the following messages:
Timestamp: 23.03.2014 15:45:44
Error: p01-caldav.icloud.com : server does not support RFC 5746, see CVE-2009-3555

Timestamp: 23.03.2014 15:46:01
Warning: Use of getAttributeNodeNS() is deprecated. Use getAttributeNS() instead.
Source File: resource://calendar/modules/calXMLUtils.jsm
Line: 32

[JavaScript Warning: "CalDAV: Unexpected response, status: HTTP/1.1 200 OK, href: /1234567890/calendars/home/"]

[JavaScript Warning: "There has been an error reading data for calendar: iHome.  However, this error is believed to be minor, so the program will attempt to continue. Error code: DAV_REPORT_ERROR. Description: There has been an error reading data for calendar: https://p01-caldav.icloud.com/1234567890/calendars/home/. It has been disabled until it is safe to use it."]

[JavaScript Warning: "There has been an error reading data for calendar: iHome.  However, this error is believed to be minor, so the program will attempt to continue. Error code: READ_FAILED. Description: "]

[JavaScript Error: "[calCachedCalendar] replay action failed: null, uri=https://p01-caldav.icloud.com/1234567890/calendars/home/, result=2147500037, op=[xpconnect wrapped calIOperation]"]


Expected results:

I'd expect that Lightning updates the entries correctly, as it work before.

Comment 1

[unnilennium]

Same here.

first connexion works at startup
updates are  ok as long no modification is done locally or remotly on the agenda.

fails at the next update after any modification of an existing appointment or creation of a new one locally or remotely.


same errors in the console as previously reported

Comment 2

[bugzilla]

Tested TB 28 with 3.0b and same exact error exists with same error log.

Comment 3

[Glenn]

@AWA

Same error on TB 24.4.0 with Lightning 2.6.4

At first glance it looks like a possible DST issue ("daylight saving time" issue) ... All dates, were the reminder could not be "closed" have been entered during wintertime (I know this because that problem currently occurs on birtday-reminder-entries ... which I entered in the beginning of the year)
The calenderfile itself has also been stored in wintertime.


Here (in Germany) the summertime began last weekend (31. March 2014) ALL Reminder that popped up since then - could not be closed :-/

Comment 4

[AWA]

@Glenn

The issue occurred before summertime, but as you described there might be a more general problem with Lightning.
So now we need someone to fix it!

Comment 5

[unnilennium]

summer time is not the same depending the location. 
In north America it starts march 9th, almost everywhere on the north hemisphere it is march 30st.
Also, south hemisphere have different date.
However, even if other possibility must be investigated, could be something not to neglect !

Comment 6

[AWA]

@Glenn

The issue occurred before summertime, but as you described there might be a more general problem with Lightning.
So now we need someone to fix it!

Comment 7

[t-master]

Same error on Thunderbird 24.4.0 with Lightning 2.6.4 on OSX since about Mid March.
First modification/synch after startup is working. Afterwards same errors as described by unnilennium and AWA

Comment 8

[t-master]

I did some digging and found out that this sounds completely like bug 695117, which apperantly was fixed in 2011. I hope that this information helps to find and solve the problem faster.

Comment 9

[AWA]

Attachment: calDavRequestHandlers.js

Thank you t-master! This was a good hint.
ICould sends back a new response with the base url and status 200, which is unhandled by calDavRequestHandlers. The new handler reports this still as a warning, but does not count it as an error. 
This works for me and it would be great, if other could test it.

Comment 10

[unnilennium]

works good for me  !

thank you AWA !!!

Comment 11

[t-master]

Attachment: Untested Patch for iCloud responses for dictionaries

@AWA: I had some time to spare, so I did some clean up, created a separate else if branch for this problem, added some logging and created a patch.

Works for me.
@All: Please test it to make sure everything works properly.

This problem is not handled by the patch for Bug 695117 as it's only for file responses (the href does not end with ".ics").

Comment 12

[bugzilla]

Comment on attachment 8413988 [details] [diff] [review]
Untested Patch for iCloud responses for dictionaries

Review of attachment 8413988 [details] [diff] [review]:
-----------------------------------------------------------------

Works for me!  (TB 24.4.0/Lightning 2.6)

Comment 13

[bugzilla]

Thanks very much to t-master and AWA!  This bug was very seriously hindering my productivity and the new patch in https://bugzilla.mozilla.org/attachment.cgi?id=8413988 directly fixes it.  Very much appreciate your time and effort in finding the correct fix!

Comment 14

[Martin Schröder [:mschroeder]]

Comment on attachment 8413988 [details] [diff] [review]
Untested Patch for iCloud responses for dictionaries

Pushing in Philipps's review queue.

Comment 15

[Philipp Kewisch [:Fallen][📱📆]]

Thanks for the patch. This seems simple enough and low risk, so I will make sure this is in TB31/Lightning 3.3.

Comment 16

[AWA]

Comment on attachment 8413988 [details] [diff] [review]
Untested Patch for iCloud responses for dictionaries

Review of attachment 8413988 [details] [diff] [review]:
-----------------------------------------------------------------

Still working, Thank you for the facelift, looks better now!

Comment 17

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 8413988 [details] [diff] [review]
Untested Patch for iCloud responses for dictionaries

Review of attachment 8413988 [details] [diff] [review]:
-----------------------------------------------------------------

r=philipp. I will upload a new patch for checkin.

::: calDavRequestHandlers.js
@@ +605,5 @@
> +                } else if (r.status &&
> +                            r.status.indexOf(" 200") &&
> +                            r.href &&
> +                            r.href.length > 1 &&
> +                            r.href.substr(r.href.length - 1, 1) == "/") {

r.href.endsWith("/") is possible with the new ES6 features.

@@ +608,5 @@
> +                            r.href.length > 1 &&
> +                            r.href.substr(r.href.length - 1, 1) == "/") {
> +                    // iCloud returns status responses for directories too, 
> +                    // so we just ignore them if they have status code 200
> +                    cal.LOG("CalDAV: Ignoring unknown response for directory, status: " + r.status + ", href: " + r.href);

I don't think we need to log this, lets just ignore it. I also suggest a slight change in comment so its clear why we want to ignore this block from a technical POV:

                    // iCloud returns status responses for directories too
                    // so we just ignore them if they have status code 200. We
                    // want to make sure these are not counted as unhandled
                    // errors in the next block

Comment 18

[Philipp Kewisch [:Fallen][📱📆]]

Attachment: Patch for checkin

Here is the patch for checkin with those comments taken into account. I think we should push this to aurora so it makes Lightning 3.3 / Thunderbird 31.

Thank you all for your work and testing!

Comment 19

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/comm-central/rev/cdbaa2cbb942
https://hg.mozilla.org/releases/comm-aurora/rev/c6b6d2fad11a

Comment 20

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 8416706 [details] [diff] [review]
Patch for checkin

I'm also pushing this to beta and will include it in 2.6.5. I will do the comm-esr24 push once I get approval for pushing to the relbranch.

https://hg.mozilla.org/releases/comm-beta/rev/0333bb8908fd

Comment 21

[Philipp Kewisch [:Fallen][📱📆]]

Comment on attachment 8416706 [details] [diff] [review]
Patch for checkin

Mark, I would like to manually include this with Lightning 2.6.5, which has been built with Thunderbird 24.5.0. Therefore, I'd like to push it to comm-release's default and GECKO2450_2014042411_RELBRANCH, then retag THUNDERBIRD_24_5_0_RELEASE and THUNDERBIRD_24_5_0_BUILD1. Do you approve?

Comment 22

[Mark Banner (:standard8)]

Comment on attachment 8416706 [details] [diff] [review]
Patch for checkin

Sure, I don't see an issue there.

Comment 23

[Philipp Kewisch [:Fallen][📱📆]]

https://hg.mozilla.org/releases/comm-esr24/rev/51447af3c1f5
https://hg.mozilla.org/releases/comm-esr24/rev/af50ef7041e6