920230 - Google CalDAV OAuth2 seems to fail re-authenticating when necessary

Status: RESOLVED FIXED

(Calendar :: Provider: CalDAV, defect)

Description

[Merike Sell (:merike)]

Google CalDAV v2 only works for limited time period. After some time (not sure exactly, an hour or similar) calendar goes unavailable and the only way I've managed to get it back is restarting Thunderbird. After restart I get a popup asking if I want it accessing calendar, after which it works again for some time. The popup never asks for password though, that part doesn't seem to get lost, just the access bit to calendar.

Last time it went unavailable this was present in error console:
CalDAV: Status 401 on initial PROPFIND for calendar Google

Comment 1

[st.ganser]

I have a similiar problem in combination with Google CalDAV v2. The only difference is, that accessing my calendar fails immediately after the popup disappears. Also the error is Status 400 instead of 401.

Comment 2

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

The 400 error might be the new webdav-sync support Google just introduced, see bug 920755.

Comment 3

[st.ganser]

(In reply to Philipp Kewisch [:Fallen] from comment #2)
> The 400 error might be the new webdav-sync support Google just introduced,
> see bug 920755.

Yes, it's the same problem as described in Comment #7 of bug 920755.

Comment 4

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Attachment: Fix - v1

Ah how I love it to see a risky patch for a .1 release. Here is a patch that fixes the issue. The problem is that preparing the http channel needs to happen asynchronously, because it can happen that the OAuth token needs refreshing, which is an extra request.

In case you haven't seen them before, the patch uses the new "arrow functions". The cool thing about them is that they don't modify the this context. It also uses default parameter values, which is also new.


this.foo = 1

let myfunc = (param1, param2) => {
  this.foo = 2;
};

myfunc()
dump(this.foo) // shows 2

I've tested add/modify/delete operations on Google CalDAV v2, Yahoo and Egroupware. Yahoo helped me test retrieval via ctags/etags, the others use webdav-sync.


If you are observing this bug I'd appreciate if you could take a moment and test it. Please test the most exotic things you can find.

Comment 5

[Tom Guyette]

How do we apply this?

Comment 6

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

(In reply to Tom Guyette from comment #5)
> How do we apply this?


I admit its a little more complicated than changing a few lines in the last bug. There is a program that does it:

http://jungels.net/articles/diff-patch-ten-minutes.html
http://gnuwin32.sourceforge.net/packages/patch.htm

It will say it cannot find the file, you will have to either change the path in the patch header or tell the program about the path when executing it.

Comment 7

[Merike Sell (:merike)]

So I applied this patch and the other from 920755 yesterday (using earlybird in case there's a difference). Sometime during today when I was away Thunderbird crashed (imap related) and after restarting Thunderbird it again asked me to give access to calendar. The log goes like this:

CalDAV: Found principal url from DAV:owner /caldav/v2/merikes%40gmail.com/user
CalDAV: webdav-sync Token: null
CalDAV: skipping unfound deleted item : /caldav/v2/[e-mail]/events/...
CalDAV: Still need to fetch 40 elements.
CalDAV: skipping unfound deleted item : /caldav/v2/[e-mail]/events/...
CalDAV: skipping unfound deleted item : /caldav/v2/[e-mail]/events/...
CalDAV: New webdav-sync Token: /caldav/v2/merikes%40gmail.com/events/sync/...
aChangeLogListener=null
calendarURI=https://apidata.googleusercontent.com/caldav/v2/[e-mail]/events/ 
iscached=false
this.mQueuedQueries.length=25

Sadly I don't have the log from just before the crash which could explain how I lost the token. Unless it was restart-related instead..

Comment 8

[Matthew Mecca [:mmecca]]

Comment on attachment 812640 [details] [diff] [review]
Fix - v1

Review of attachment 812640 [details] [diff] [review]:
-----------------------------------------------------------------

Looks good. r=mmecca

::: calendar/providers/caldav/calDavCalendar.js
@@ +2752,4 @@
>              if (this.verboseLogging()) {
>                  cal.LOG("CalDAV: send(" + requestUri.spec + "): " + uploadData);
>              }
> +            var uploadData = serializer.serializeToString();

var to let

Comment 9

[Kosuke Kaizuka]

Patch v1 (with one change by Matthew's review) works fine on Lightning 2.6 / Thunderbird 24.0 / Win7Pro SP1 (x64).

Comment 10

[Nicola Ferralis]

Patch v1 (with one change by Matthew's review) works also for me on Lightning 2.6 / Thunderbird 24.0 / OS X - 10.8.5

Comment 11

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Attachment: Fix - v2

Comment 12

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

Thank you all for testing.

Please check in this patch to all branches: c-c, c-a, c-b and comm-esr24.

Comment 13

[Rolf Gloor]

Hi Philipp

I still have some issues (despite including patch v2, w matthews fix, and fix from bug 920755).
However, I compared the attachement diff's (#814035 as well as #812640) for calDavCalendar.js and especially the line numbers:
(First there is an offset of a missing line in "my" file. Could be somewhere in the header comments.)

However, I am somehow missing 41 lines of code between the diff-lines:
[1506-10521] and [1663-1675].

I therefore went back to the mozilla repository http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/releases/2.6
and downloaded / extracted the official release file again.

Still missing the 41 lines compared to your file.

The code - according to your diff files starting in your original at line 1663 starts in the newly downloaded release file (Lightning 2.6) at line 1621 (Delta: 41 missing lines plus the 1 line offset).

There might be the reason for something not working.

Comment 14

[Rolf Gloor]

CORR / Feedback

a) the lines are still missing   -   and I don't know where they are / went  ;-)

b) the missing events are now slowly dripping in.

So things seams to work out. [dumbs up]

Comment 15

[Ryan VanderMeulen [:RyanVM]]

https://hg.mozilla.org/comm-central/rev/61db450ae0a5
https://hg.mozilla.org/releases/comm-aurora/rev/ac780e34d452
https://hg.mozilla.org/releases/comm-beta/rev/cc6a332fda84
https://hg.mozilla.org/releases/comm-esr24/rev/2b266e0bc22b

Comment 16

[shopping]

I have tested the changes: it does not help here with a 'similar' bug

https://bugzilla.mozilla.org/show_bug.cgi?id=920988

Still with the new versions all calendars show with exclamation mark, no events displayed, but creating
new events succeeds (show up in google calendar afterwards, but not in lightning)

Best
Daniel

Comment 17

[Philipp Kewisch [:Fallen] [:📆][:🧩]]

You are going to have to apply all related patches at once. Please apply all patches of the bugs in the "Depends On" field of bug 921131. The new builds should be out by tomorrow, I'll do a blog post when they are available.

Comment 18

[shopping]

Any news on availability of the fix? Now I'm one month without calendars in Thunderbird and would love it to have them back :-)

Best
Daniel

Comment 19

[shopping]

(In reply to shopping from comment #18)
> Any news on availability of the fix? Now I'm one month without calendars in
> Thunderbird and would love it to have them back :-)
> 
> Best
> Daniel

I have loaded 2.6.1 from https://addons.mozilla.org/de/thunderbird/addon/lightning/versions/:  works like a charm again!! Thank you very much!!!

Best Daniel