874654 - Infinite loop at refresh when Caldav server is up but calendar is not available

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect)

Description

[mguessan]

User Agent: Mozilla/5.0 (Windows NT 5.1; rv:21.0) Gecko/20100101 Firefox/21.0 (Beta/Release)
Build ID: 20130511120803

Steps to reproduce:

- Setup a Caldav calendar in Lightning pointing to a DavMail instance
- Sync calendar
- Switch off network


Actual results:

Infinite refresh loop, high CPU usage


Expected results:

Mark calendar as temporarily not available

Comment 1

[mguessan]

Note that this issue is similar to:
Bug 746962 - Never ending loop when caldav url is misconfigured (principal or home url) 
and
Bug 429126 - infinite loop at refresh when a calendar is deleted from the server outside of Lightning

but in this case the calendar did not disappear (404), it's just not available temporarily (network issue) and the server returns a 503 service unavailable

The fix is simple: just detect server error (between 500 and 599 return code) and abort sync.

Related DavMail tracker entry:
https://sourceforge.net/tracker/?func=detail&aid=3609878&group_id=184600&atid=909904

Comment 2

[mguessan]

Attachment: Fix for infinite loop on server unavailable issue

Detect server unavailable response status and abort sync.

Note that I also tried to add an additional condition later after the
!str || request.responseStatus == 404
test but this is too late as the convertByteArray triggers an error in Thunderbird console.

Comment 3

[David von Oheimb]

I can confirm both this bug and the patch.

As I reported for bug 746962,
this misbehavior occurs when a correctly specified CalDAV calendar server is not reachable (because for instance there the network connection of the client is down or the server is in some company intranet but the laptop accessing it is outside that zone). 

I realized this problem using DavMail, as reported here: http://sourceforge.net/tracker/index.php?func=detail&aid=3609878&group_id=184600&atid=909904
When trying to synchronize a calendar while the server is not accessible,
exceptions are thrown all the time, causing very high CPU load.

DavMail correctly returns a "HTTP/1.1 503 Service Unavailable" error,
yet Lightning immediately retries the connection over and over!
It should give up at the latest after trying a few times and 
then wait (not using a busy loop) for some time before re-trying.

The patch provided by mguessan solves the problem.

Comment 4

[Jacob Fortune]

This is also similar to bug 738815

Comment 6

[Philipp Kewisch [:Fallen][☁️📆]]

Need this for 1.9.2 for apple icloud.

Comment 7

[Stefan Sitter [:ssitter]]

According to various reports and Comment 6 it seems that Apple blocks all Lightning connections due to this problem.

Comment 8

[Philipp Kewisch [:Fallen][☁️📆]]

I'm not surprised, the logical result of people switching their user agents. We need to land this in time for Lightning 2.6, but as always for Lightning CalDAV changes, I procrastinate on them because we have no unit tests to guard us against regressions.

Maybe we should just land it and see how it goes during the beta phase. Objections?

Comment 9

[Jesse Thompson]

Our CalDAV service has around 2200 active users of Lightning.  The observed occurrence of an end-user misconfiguring Lightning in a way that triggers this bug is about once every 2 months.  The impact of this problem is high; a single misconfigured Lightning client can create 5 GB of web server access logs in a day, which introduces a strong likelihood of logs filling partitions and causing a server outage.

Comment 10

[Philipp Kewisch [:Fallen][☁️📆]]

Jesse, what CalDAV server are you using? Does the patch also fix the issue for you?

Comment 11

[Jesse Thompson]

Oracle Communications Calendar Server 7.  I have not tried the patch yet.

Comment 12

[Philipp Kewisch [:Fallen][☁️📆]]

Comment on attachment 752399 [details] [diff] [review]
Fix for infinite loop on server unavailable issue

Ok, lets give it a try. Its far less scary than the rest of them.

Comment 13

[Philipp Kewisch [:Fallen][☁️📆]]

Comment on attachment 752399 [details] [diff] [review]
Fix for infinite loop on server unavailable issue

We're going to want this on aurora so it will be part of Lightning 2.6 / Thunderbird 24 ESR.

Comment 14

[Philipp Kewisch [:Fallen][☁️📆]]

Pushed to comm-central changeset b0a0d184737c

Comment 15

[Philipp Kewisch [:Fallen][☁️📆]]

Backported to releases/comm-aurora changeset cd19366c9f93

Comment 16

[Jacob Fortune]

(In reply to Philipp Kewisch [:Fallen] from comment #10)
> Jesse, what CalDAV server are you using? Does the patch also fix the issue
> for you?

I work with Jesse.  I tested the patch--it does not fix the issue for us.  Apache (which is an upstream proxy server) is returning a 405 error to Lightning (method not allowed).  In every instance where we have seen this occurring, the users have mistyped some part of the URL and pointed at an Apache directory that either doesn't exist or doesn't allow requests.  A common example is using the following:
https://ourserver.edu/day/home/[e-mail]/calendar/
Note the mistyped "day" rather than appropriate "dav".

Perhaps this is a different problem we need addressed.  Here's a suggested patch for the issue we're having.  The following code is in the same location of calDavCalendar.js.  It will switch the calendar off.  A user manually switching it on will just cause it to be disabled again on next sync, which is ideal behavior.  This should alert the user that something is wrong.

            if (request.responseStatus == 405) {
                // Bad method
                thisCalendar.setProperty("disabled", "true");
                thisCalendar.setProperty("auto-enabled", "true");
                thisCalendar.completeCheckServerInfo(aChangeLogListener, Components.results.NS_ERROR_ABORT);
                return;
            }

Should I open a new bug for this issue?

Comment 17

[Philipp Kewisch [:Fallen][☁️📆]]

We can add 405 to the check, fine to do so in this bug. I'll patch this myself before checkin.

Comment 18

[Jacob Fortune]

(In reply to Philipp Kewisch [:Fallen] from comment #17)
> We can add 405 to the check, fine to do so in this bug. I'll patch this
> myself before checkin.

Great, thank you!

Comment 19

[Jesse Thompson]

This is good because I don't see a way in Apache (other than hooking in to the request cycle with mod_perl) to rewrite PRODFIND methods against the inverse of valid DAV URIs.

Comment 20

[mguessan]

Don't you think we should make the test more general ?
- 50x means temporarily not available
- 40x means permanently invalid

Comment 21

[Philipp Kewisch [:Fallen][☁️📆]]

Attachment: Fix - v2

This is the new patch. I'd appreciate a second set of eyes, the tree is closed anyway.

Comment 22

[Matthew Mecca [:mmecca]]

Comment on attachment 783600 [details] [diff] [review]
Fix - v2

Untested, but looks good codewise. r=mmecca

Comment 23

[Frederic de Bordeaux]

This is exactly similar to Bug 901450 - Infinite loop at refresh when ICS server is up but calendar is not available

Comment 25

[Philipp Kewisch [:Fallen][☁️📆]]

Comment on attachment 783600 [details] [diff] [review]
Fix - v2

This is the patch that needs to be checked in, on aurora and beta since the merge happened today.

Comment 26

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/comm-central/rev/ee8137dfd0f9

Comment 27

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/releases/comm-aurora/rev/ddf2bab9b52c
https://hg.mozilla.org/releases/comm-beta/rev/131ed16e2581

Comment 28

[David von Oheimb]

Fix v2 confirmed, thanks!