Closed Bug 1468069 Opened 6 years ago Closed 6 years ago

Beta (60.0b7) not retaining Lightning's Google account credentials

Categories

(Calendar :: General, defect)

Product:
Calendar
Component:
General
Version:
Lightning 6.2
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: davross, Assigned: darktrojan)

References

Details

(Keywords: regression)

Attachments

(1 file)

Bug 1468069 - OAuth credentials overwritten
59 bytes, text/x-review-board-request
Fallen
: review+
Fallen
: approval-calendar-beta+
jorgk-bmo
: approval-calendar-esr+
Details 
User Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Build ID: 20100101

Steps to reproduce:

Created a new user profile so as to more effectively test this problem, and it reproduces in this new profile.

When adding Google Calendar, I need to have 2-way communication so I implement CALDAV. It connects. Sweet. Along the lines of`https://apidata.googleusercontent.com/caldav/v2/{CALENDAR_ID}/events` - where {CALENDAR_ID} is replaced with each Google Calendar's ID found in https://calendar.google.com/calendar/ >> My calendars >> Options (hover over the calendar to expose 3 dots) >> Setting and sharing >> Integrate calendar >> Calendar ID 


Actual results:

Somewhere down the line Thunderbird/Lightning forgets these credentials however. On restarting I have to go through the process again.


Expected results:

Lightning should retain Google Calendar credentials. 

Sadly I've lost my mobile phone so can't currently log in to test as I've 2FA enabled, and SMS is the only viable 2FA solution in Thunderbird. Using the Web UI instead as I can use my hardware U2F/2FA key.
Component: Untriaged → Security
David, 

Thanks for your report. Does TB forget the credentials *while* it's running or only after a restart? If it only happens after a restart, does it consistently forget after the *first* restart, or does it take several restarts?
Flags: needinfo?(davross)
Please note that we shipped TB 60 beta 7 despite bug 1466430. I'm using the beta and can't see a calender problem, but it's possible that some problems are hiding somewhere. We can only be sure once we investigated the test failures.

I don't think this is related to Josiah's recent password work in mailnews in bug 516464 and bug 1461106.
Component: Security → General
Flags: needinfo?(philipp)
Flags: needinfo?(makemyday)
Product: Thunderbird → Calendar
Version: 60 → Lightning 6.2
Oh wow I was about to switch OFF 2FA on my account just to test this and spotted U2F (I presume WebAuthn?) works. WOW! I missed that advance.

On the first restart Thunderbird has immediately lost the CALDAV (read/write) credentials. Calendars that are using ICAL (read only) stay connected.

I've not actually tested running it for an extended period of time, to see if it also drop the CALDAV credentials. Will do this test right away.
Flags: needinfo?(davross) → needinfo?(jsbruner)
Flags: needinfo?(jsbruner)
30 minutes of operation - credentials stayed.
Sleep for 20 minutes & wake from sleep - credentials stayed.
On restart - 2x credentials for ICAL stayed - 1x credential for CALDAV stayed - 3x credentials for CALDAV were removed.
Create new user profile:
1) Add GMail account.
2) Add 1x Google CALDAV account. Restart - credentials stay.
3) Add 1x Google CALDAV account. Restart - credentials stay.
4) Add Zoho Mail email account. Restart - credentials stay.
5) Add 1x Google CALDAV account. Restart - credentials from step 3 were removed. All others stay.
6) Remove 1x Google CALDAV account (added in step 3). Restart - credentials stay.
7) Add 1x Google CALDAV account (different than 1 in steps 3 & 6). Restart - credentials from step 5 were removed.
* correction step 2 is an ICAL account.

Additional note that both Gmail and Zoho Mail accounts are IMAP.
8) Remove 1x Google CALDAV account (added in step 5).
9) Add 1x Google ICAL account. Restart - credentials stay.
10) Add 1x Google CALDAV account (different from 3,5,6,8). Restart - credentials from step 7 were removed.
11) Remove Zoho Mail account. Restart - credentials from step 7 are still removed.
12) Re-Add step 7 credentials. Restart - credentials from step 10 are removed.
13) Re-Add step 10 credentials. Restart - credentials from step 2 are removed.
* correction. Step 13 saw credentials from a CALDAV account removed. But I've lost track. Anyway I'm sure you get that it's pretty messy.
Create new user profile:
1) add 1x Gmail account
2) add 1x GoogleICAL account
3) add 2x Google CALDAV accounts
4) restart - credentials are again forgotten for 1 of the CALDAV (addition of 2nd email account not require to reproduce)
5) Re-Add the credentials
6) restart - credentials forgotten for the other CALDAV
7) Re-Add the credentials
8) restart - credentials forgotten for the 1st CALDAV again. Seems to loop to a different one each restart, if the missing credentials are re-added - otherwise it stays as the same calendar demanding I login, with all the others functioning.

Yes I always have "Don't ask again on this computer" option selected.
Create new user profile:
1) add 1x Gmail account
2) add 2x Google CALDAV accounts
3) restart - credentials are forgotten for 1 of the CALDAV accounts
Downloaded fresh install of Thunderbird Beta, created a new user account, and followed steps from comment 12. Once again credentials of 1 CALDAV account is forgotten.
regression?
Blocks: tb60found
Status: UNCONFIRMED → NEW
Ever confirmed: true
Regression. Bug 1449483. Lightning is overwriting the credentials of the one calendar with those of the next because they share the same origin ("oauth:").
Depends on: 1449483
Assignee: nobody → geoff
Status: NEW → ASSIGNED
Comment on attachment 8987240 [details]
Bug 1468069 - OAuth credentials overwritten

https://reviewboard.mozilla.org/r/252506/#review259026

Thanks for fixing this. It is too bad we need to do special cases since I wanted to get rid of this function, but I guess we have to keep it at that.
Attachment #8987240 - Flags: review?(philipp) → review+
Attachment #8987240 - Flags: approval-calendar-beta?(philipp)
Keywords: checkin-needed
Flags: needinfo?(philipp)
Attachment #8987240 - Flags: approval-calendar-beta?(philipp) → approval-calendar-beta+
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/9046e6ee5675
fix OAuth credentials being overwritten. r=philipp
Status: ASSIGNED → RESOLVED
Closed: 6 years ago
Keywords: checkin-needed
Resolution: --- → FIXED
Flags: needinfo?(makemyday)
Target Milestone: --- → 6.4
TB 60 beta 9, Calendar 6.2 (BETA_60_CONTINUATION branch):
https://hg.mozilla.org/releases/comm-beta/rev/b13c03b7342bf5028d2f03926daa8d14a37b671f

Now I need to go and fix the linting errors :-(
Target Milestone: 6.4 → 6.2
Pushed by mozilla@jorgk.com:
https://hg.mozilla.org/comm-central/rev/226cb0a4c069
fix linting error. r=me DONTBUILD
Is there a timeline to have the coinciding Provider for Google Calendar (PGC) built for 60.0b9? I installed 60.0b9 just now and the issue is still present. I'm still using PGC 41.b6 as there's nothing newer.
There's been a lively discussion on the non-public Thunderbird drivers mailing list about this issue. Since builds switched from Buildbot to TaskCluster, this add-on has apparently been dropped from the build :-(

Maybe the Calendar module owner has a way forward.

Tom, the Gdata provider must be built somehow in the process since we run a test on it. Would it be possible to dig it out somewhere from some artefact?
Flags: needinfo?(philipp)
Flags: needinfo?(mozilla)
(In reply to Jorg K (GMT+2) from comment #24)
> Maybe the Calendar module owner has a way forward.

I think the best way forward would be to bundle gdata as part of thunderbird, the same way that lightning is.
 
> Tom, the Gdata provider must be built somehow in the process since we run a
> test on it.

I'm not sure that it is: https://hg.mozilla.org/releases/comm-beta/file/66305b3923d3/calendar/test/unit/xpcshell-shared.ini#l32

> Would it be possible to dig it out somewhere from some artefact?

It isn't uploaded as an artifact, since the upstream code didn't gracefully handle uploading additional addons.
Flags: needinfo?(mozilla)
Depends on: 1471326
Tom, you commented out execution of the test here:
https://hg.mozilla.org/comm-central/rev/8bbed059de0a#l13.14

Sadly that wasn't reviewed at all and IMHO wouldn't have passed review since we need the test for that very purpose.

I filed bug 1471326 to reinstate it.
Flags: needinfo?(philipp)
(In reply to Arthur K. from comment #23)
> Is there a timeline to have the coinciding Provider for Google Calendar
> (PGC) built for 60.0b9? I installed 60.0b9 just now and the issue is still
> present. I'm still using PGC 41.b6 as there's nothing newer.
Geoff, how did you test this? If I understood Philipp correctly on IRC, your change does not get integrated into the add-on. Or maybe the add-on is now incompatible for other reasons.

I guess Arthur is still using this one:
http://ftp.mozilla.org/pub/calendar/lightning/candidates/6.2b6-candidates/build1/win32/gdata-provider-4.1b6.en-US.win32.xpi
Flags: needinfo?(geoff)
I used the CalDAV provider like the reporter. The gData provider works the same way though, so it should be fixed too.
Flags: needinfo?(geoff)
Sorry, I don't understand anything here.

Apparently there are two "data providers", CalDAV and gData. CalDAV is "native" to Lightning and gData comes with an extra add-on, the "Provider for Google Calendar", right?

So why does the reporter talk about CalDAV in comment #0 and right down to comment #13 and then Arthur who came from bug 1470424 - which I perhaps incorrectly duped here - asks about Provider for Google Calendar?

So does the Provider for Google Calendar add-on need to be rebuilt? Or should I un-dupe bug 1470424?
Flags: needinfo?(geoff)
Yes, there are two providers, and they're both broken by this bug, so you've correctly duped bug 1470424. The Provider for Google Calendar is what we're trying to resurrect in bug 1471326. It won't need any extra work for this bug because the problem is within Lightning.
Flags: needinfo?(geoff)
Sorry, this is still not clear. If the problem is within Lightning and it was fixed in TB 60 b9, why does the "old" TB 60 b6 Provider for Google Calendar not work? As far as I'm aware, it's code hasn't changed, or am I missing something? So resurrected or old, it should work now, no? Hence my question from comment #27 (quote):
  If I understood Philipp correctly on IRC, your change does not get integrated
  into the add-on. Or maybe the add-on is now incompatible for other reasons.
Flags: needinfo?(geoff)
Or maybe in bug 1471326 you were already able to build a resurrected version to see whether it's working?
I have managed to make a build of the current code, and it appears to work. I haven't got any old versions to test with, but they should work for this at least, but they might be broken for other reasons.
Flags: needinfo?(geoff)
I've switched to 60.0b9 overnight and the problem persists. Will now attempt with a fresh user profile.
Perhaps is Bug 1468912 related.

The import of an ics network-based calendar also fails:

https://www.ferienwiki.de/exports/feiertage/2018/de/niedersachsen
Problem does NOT persist in new user profile. I was also able to add ICS calendars (tested Firefox Release Calendar and Florian's from Comment 35). Restarted Tbird 60.0.b9 and it has retained all accounts.
Keep in mind that - at least for me - Lightning is not updated automatically when installing the Thunderbird update. The only workaround that I currently know is to uninstall Lightning and reset the preferences "extensions.installedDistroAddon.{e2fda1a4-762b-4020-b5ad-a41df1933103}". Upon next restart Thunderbird will re-install the Lightning package shipped with Thunderbird.
David, what is listed in the Saved Passwords section of Thunderbird's preferences? There should be a row for each calendar and the first column should have "oauth:[letters and numbers] (Google CalDAV v2)". Before this patch landed the letters and numbers would be missing. You'll have to reauthorise each account once to recover from the broken state.
Flags: needinfo?(davross)
Saved Passwords section is filled as expected. As mentioned in Comment 36, creating a new user profile fixed the CalDAV credentials. I'm happy. Issue is correctly flagged as resolved.
David, thanks for the "reply".  I'm resetting the NI, which I'm assuming accidently wasn't unset


(In reply to Geoff Lankow (:darktrojan) from comment #16)
> Regression. Bug 1449483. 

In such cases it is customary to regression keyword. And version of the regression(which was already set)
Flags: needinfo?(davross)
Keywords: regression
Attachment #8987240 - Flags: approval-calendar-esr+
Are we going to see an updated Provider for Google Calendar as well?
(In reply to Arthur K. from comment #42)
> Are we going to see an updated Provider for Google Calendar as well?

Answered my own question. I see that is in bug 1475224
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: