Closed Bug 1737414 Opened 3 years ago Closed 2 years ago

Adding a google calendar with a google username that is not either gmail.com or googlemail.com email request fails

Categories

(Calendar :: Provider: CalDAV, defect)

Product:
Calendar
Component:
Provider: CalDAV
Version:
Thunderbird 91
Platform:
Desktop
All
Type:
defect

Tracking

(thunderbird_esr91+ wontfix, thunderbird96 affected)

Status:
RESOLVED FIXED
Milestone:
97 Branch
Tracking Flags:
Tracking Status
thunderbird_esr91 + wontfix
thunderbird96 --- affected

People

(Reporter: roger, Assigned: mkmelin)

Details

Attachments

(1 file)

Bug 1737414 - use Google OAuth also if the user entered the apidata.googleusercontent.com url. r=darktrojan
48 bytes, text/x-phabricator-request
Details | Review

User Agent: Mozilla/5.0 (Android 9; Mobile; rv:93.0) Gecko/93.0 Firefox/93.0

Steps to reproduce:

  1. Use file/new/calendar to a a network calendar.
  2. In the username field add a valid google username that is a valid email address, but does not have a domain part that is either gmail.com or googlemail.com.
  3. In the Location field add either gmail.com or googlemail.com.

This is the only way I have found to get thunderbird to produce that correct url for requesting an OAuth2 token for the caldav google api. The fact that I have had to use undocumented approach may well be considered a documentation bug.

Actual results:

The correct OAuth2 url is produced.

console.log: Calendar: [CalDavProvider] Trying to detect calendar using attemptGoogleOauth method
console.log: Calendar: [CalDavProvider] Checking collection type at https://apidata.googleusercontent.com/caldav/v2/roger%40xxxxxxx.co.uk/user

The above URI looks good to me. However this reveals what the real problem is on my system.

console.log: Calendar: CalDAV: OAuth token expired or empty, refreshing

This is entirely expected and a state change is raised.

JavaScript error: resource:///modules/OAuth2.jsm, line 171: NS_ERROR_NOT_IMPLEMENTED: Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIRequest.name]
JavaScript error: resource:///modules/OAuth2.jsm, line 171: NS_ERROR_NOT_IMPLEMENTED: Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIRequest.name]
JavaScript error: resource:///modules/OAuth2.jsm, line 171: NS_ERROR_NOT_IMPLEMENTED: Component returned failure code: 0x80004001 (NS_ERROR_NOT_IMPLEMENTED) [nsIRequest.name]

The state change fails because the aRequest parameter does not have a name property.

Expected results:

The state change should work and a request for a new token sent.

Component: Untriaged → Provider: CalDAV
OS: Unspecified → All
Product: Thunderbird → Calendar
Hardware: Unspecified → Desktop

If the domain has google MX records set up, it will be work.
https://searchfox.org/comm-central/rev/a2472c026fcf9bc25733330e9fa1e26c0429be3c/calendar/providers/caldav/CalDavProvider.jsm#245-246

But I'm not sure what you're suggesting.

The beardandsans.co.uk domain does have mx records but they point to my isp's mail server not at google. This is the standard way to set up a google account that is not also a gmail account.

This used to work using the old google caldav provider extension.

Roger

How are you expecting Thunderbird to figure it out then?

My knowledge of the mailnews codebase is zero. It used to work before thunderbird was upgraded during a move from Ubuntu 21.04 to 21.10. My calendar access stopped working. It was working with the old google calendar extension in the version in 21.04. After the upgrade that was failing to access the calendar. All the documentation I could find pointed to use the built in method. In trying to get this going I ended up having to look at the OAuth2 code. I realised that the GoogleOAuth provider was failing. But that if I used googlemail or gmail in the location field and my google username (i.e. email address) in username the code would construct the correct google calendar api authorisation url. However when the expired or absent response came back it would fail at the point shown in the logs.

I do not know the code base well enough to take this much further. So I suppose the fundamental question is what does a user have to do to access a google calendar in thunderbird when their google account name is a non google mail address? Is this is not answered somewhere in the documentation then it is a documentation bug. If code changes are also required then this is a bug.

The provider for google calendar add-on only does - you guessed it, google. So no need for it to figure out anything.

You should be able to find the complete URL to the address to the google calendar. It's usually https://apidata.googleusercontent.com/caldav/v2/example@gmail.com/events

I don't see how we could figure out that this is what you want to use without you giving the URL.

That url is hardcoded in mailnews/base/OAuth2.jsm. Somwhere near line 180. I cannot tell you exactly where at the moment.

But I will do when I get back to computer.

Roger

Forget my last comment. You can see that a correctly formatted URL is generated in the trace contained in the original report. So the google related content wrapped around my a@b.c username is hard coded somewhere.

I am wasting my time here. Does thunderbird's built in calendar code not support google calendar accounts that have usernames that are not google domains?

It works for G-suite at least (like @mozilla.com)

The hard coded uri is set up in the attemptGoogleOauth method at line 253 in source/comm/calendar/providers/caldav/CalDavProder.jsm.

I think mozilla is another domain that is hard coded for special treatment elsewhere. There is a table somewhere that maps well known domains to auth scopes.

My apologies to everyone.
In my "Forget my last comment" post above I meant to say.
"Am I wasting my time?" definitely not "I am".

My excuse is that when I composed that post I was sat in a hospital pre-op area waiting for a procedure to remove a cataract!

The op was successful and looking at the thread again today. I realised what I had actually sent. Oops.

Like I said, not hardcoded. We check the MX records in DNS. If you don't have those set, we can't know.
https://searchfox.org/comm-central/rev/0e154d21c2179c0f4cad51fa61bbab20f494dbc2/mail/components/accountcreation/FetchConfig.jsm#208

I don't know if it helps you here but maybe the Google 'Calendar-ID' (aka calid) is different if you don't use a Gmail-address?

Note: I'm using the Dutch localized versions, so the translations above can be different from the English menu-items.

The https://apidata.googleusercontent.com/caldav/v2/roger%40xxxxxxx.co.uk/user link is my redacted version with the xxx.. replacing my hostname. This exactly matches my Calendar-ID. I have switched over to evolution mail now and that works.

The real problem occurs after this is logged.

console.log: Calendar: CalDAV: OAuth token expired or empty, refreshing.

All this is in my intial report.

This message is generated at
thunderbird/calendar/providers/caldav/modules/CalDavSession.jsm:197 (in the source I am looking at.

It is in the function prepareRequest at line 194 in the same file.

This function is called as a result of calling the attemptGoogleOauth function that is defined at line 237 of CalDavProvider.jsm

  async attemptGoogleOauth(location) {
    if (!this.username) {
      return null;
    }

    let usesGoogleOAuth = cal.provider.detection.googleOAuthDomains.has(location.host);

    if (!usesGoogleOAuth) {
      // Not using Google OAuth that we know of, but we could check the mx entry.
      // If mail is handled by Google then this is likely a Google Apps domain.
      let mxRecords = await DNS.mx(location.host);
      usesGoogleOAuth = mxRecords.some(r => /\bgoogle\.com$/.test(r.host));
    }

    if (usesGoogleOAuth) {
      let uri = Services.io.newURI(
        `https://apidata.googleusercontent.com/caldav/v2/${encodeURIComponent(this.username)}/user`
      );
      return this.handlePrincipal(uri);
    }

    return null;
  }

The googleOAuthDomains are defined in calProviderDetectionUtils.jsm

  googleOAuthDomains: new Set(["gmail.com", "googlemail.com"]),

Once again, as I said above, it is this that is going wrong. The URL for accessing the calendar is already correct at this point, it has already "been worked out by the provider". The code above is corrupting it.

My evolution setup works now. So I am not going do any more research into it.

Assignee: nobody → mkmelin+mozilla

Thanks Magnus,

That is what I was trying to get across. My apologies for all the typos and mistakes. I least I can see what I am typing a bit better now!

Roger

Status: UNCONFIRMED → ASSIGNED
Ever confirmed: true

Actually, I realize the patch isn't quite right...

Should be alright after all.

Tested on Ubuntu 21.10

Gets as far as the Google SSO 2FA screen final steps. When you enter the 2FA verification code it fails with the following message.

The page you are trying to view can not be shown because the authenticity of the received data could not be verified.

The URL in question is

https://localhost/?code=4%2F0AX4XfWgDQCNk_GJ5FDs1OY6IOIcSzQgMdMQoTPn4EkJj4MMUWQ_oEtZTdAXcNV5Wt2n5RA&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcalendar

That does not look right. I am also seeing the following error in the thunderbird logging.

IPDL protocol error: Handler returned error code!

###!!! [Parent][DispatchAsyncMessage] Error: PClientManager::Msg_ForgetFutureClientSource Processing error: message was deserialized, but the handler returned false (indicating failure)

Roger

Hmm, sounds strange. (SSL error for Thunderbird's own connection to localhost. which isn't even a real connection?)
Does OAuth for other google things (non-custom) work for you?
Are you running a web server on localhost? I think we've seen some reports about that confusing things - though I can't reproduce.

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/36b0e2732579
use Google OAuth also if the user entered the apidata.googleusercontent.com url. r=darktrojan

Status: ASSIGNED → RESOLVED
Closed: 2 years ago
Resolution: --- → FIXED
status-thunderbird96: --- → affected
status-thunderbird_esr91: --- → affected
tracking-thunderbird_esr91: --- → +
Target Milestone: --- → 97 Branch

(In reply to Magnus Melin [:mkmelin] from comment #20)

Hmm, sounds strange. (SSL error for Thunderbird's own connection to localhost. which isn't even a real connection?)
Does OAuth for other google things (non-custom) work for you?
Yes.
Are you running a web server on localhost? I think we've seen some reports about that confusing things - though I can't reproduce.
Yes.

How do I go about turning on more detailed logging/tracing in order to move this forwards.

R.

I don't have good suggestions for that.
BUT, try turning the web server off, and see if you can now set up your calendar. The fix landed on daily.

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: