Closed Bug 1853257 Opened 2 years ago Closed 2 years ago

Adding Thunderbird CalDAV in v115.x tries to use OAuth page instead of CalDAV

Categories

(Calendar :: Provider: CalDAV, defect)

Product:
Calendar
Component:
Provider: CalDAV
Version:
Thunderbird 115
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED WONTFIX

People

(Reporter: garret, Unassigned)

References

Details

Steps to reproduce:

When I just updated from Thunderbird 102.15.1 to 115.2.2 on Windows 10, I suddenly got a popup OAuth window for one of my calendars. This calendar is using Fastmail calendars. Fastmail has an overall account password for logging in via the web, but for individual SMTP/IMAP/CalDAV, etc. access, Fastmail has app passwords (which can be generated and deleted independently, with fine-grained permissions). I attempted to enter my Fastmail app password again (interestingly Thunderbird 115 shows me a list of usernames and passwords it had saved, but I entered the correct app password manually), and Fastmail told me that I was trying to enter an app password, but that Fastmail was expeting my main password. That's when I realized that Thunderbird was using some OAuth login, and not the CalDAV login.

Actual results:

I then unsubscribed for the calendar altogether, and tried to add it back:

  • I entered me@example.com as the username.
  • I left the location URL blank.

I got the OAuth popup window again.

I was only able to add the Fastmail CalDAV calendar using my Fastmail app password by manually entering https://caldav.fastmail.com/ as the location. (See https://www.fastmail.help/hc/en-us/articles/1500000278342-Server-names-and-ports .) That's when Thunderbird used the CalDAV server instead of the web OAuth form for authentication.

Expected results:

My domain example.com has CalDAV auto-discovery enabled via the DNS entry _caldavs._tcp.mydomain.com. See https://www.fastmail.help/hc/en-us/articles/360060591153-Manual-DNS-configuration for more details . See RFC 6764 for the specification.

When I enter me@example.com as the location of the new calendar, I would expect Thunderbird to query the DNS and find the entry for _caldavs._tcp.mydomain.com and automatically attempt to authenticate with CalDAV, rather than switching to some OAuth form. I'm pretty sure that's the way it used to work.

As mentioned I was able to work around this by entering https://caldav.fastmail.com/ as the location. But has behavior changed in this area? Does Thunderbird 115 have a different algorithm for automatically determining the location of the calendar? Does it no longer follow RFC 6764? (Or am I mistaken in that it ever did?)

(In the bug description _caldavs._tcp.mydomain.com should be _caldavs._tcp.example.com.)

Component: Untriaged → Provider: CalDAV
Product: Thunderbird → Calendar

One difference is that Fastmail now supports OAuth2 (bug 1785240).
I believe you're mixing up terms. There is no "CalDAV login". That you got OAuth2 shows that it autodiscovered fastmail for you as it should

I believe you're mixing up terms.

Perhaps we're communication past each other, so let me back up and make sure I clearly lay out things step-by-step, so there is no confusion.

  • CalDAV is governed by RFC 4791. In itself it has no provision for OAuth2. It does mention HTTP Basic authentication over TLS.
  • Fastmail provides traditional CalDAV access at https://caldav.fastmail.com/ . It uses HTTP Basic authentication over TLS. The authentication is based upon Fastmail "app passwords", which are separate per-device or per-application logins with fine-grained permissions. Fastmail's HTTP Basic Authentication for CalDAV https://caldav.fastmail.com/ does not support the regular Fastmail login with coarse-grained permissions. (It says it plainly on https://www.fastmail.help/hc/en-us/articles/1500000278342-Server-names-and-ports .)
  • RFC 6764 provides a method for defining the default CalDAV server through a DNS configuration. Thus I could configure DNS records for example.com saying that I wanted to use https://caldav.fastmail.com/ as my CalDAV server. A client would look up the appropriate DNS entries (in this case _caldavs._tcp) for example.com, find that CalDAV should use https://caldav.fastmail.com/, and attempt to use HTTP Basic authentication over TLS to log into https://caldav.fastmail.com/. (Notice nowhere at this point has OAuth come into the picture.) Until now Thunderbird used to honor RFC 6764, discover https://caldav.fastmail.com/ , and allow HTTP Basic authentication to log in using the Fastmail app password. This is now broken in Thunderbird, which is why I'm filing this ticket.
  • Fastmail also now has a separate authorization service using OAuth2. This is done, not via https://caldav.fastmail.com/, but via the Fastmail web site. It also functions differently: rather than the client providing an app password, the client provides no password at all (which is how OAuth works), but the user must authenticate using the main Fastmail login. This is different than the app password used to log in via HTTPS Basic Authentication at https://caldav.fastmail.com/, and has a coarse permission structure. (Currently OAuth automatically gives the client access to email, calendars, and contacts; while HTTP Basic login via using an app password at https://caldav.fastmail.com/ would allow an app password to be given only permission to calendars, for example.)

As mentioned, Thunderbird at point in the past honored RFC 6764, found the _caldavs._tcp entry pointing to https://caldav.fastmail.com/, and would then ask for a password (for Fastmail this would be the app password), after which it would authenticate with https://caldav.fastmail.com/ using HTTP Basic authentication over TLS. Now, however, apparently Thunderbird has some knowledge of Fastmail "baked into it", and when I enter a domain with calendar hosted by Fastmail, Thunderbird completely ignores my RFC 6764 entries in the DNS, and instead jumps directly into the Fastmail OAuth dialog.

I understand and agree that, if a domain has no RFC 6764 entries configured, it is handy that Thunderbird recognizes that a domain uses Fastmail as the CalDAV server and is helpful by directly accessing Fastmail's OAuth UI. What is not helpful is that Thunderbird forces Fastmail's OAuth UI, even when my domain has RFC 6764 entries defined for CalDAV. This new approach (which is new Thunderbird behavior) completely defeats the purpose of RFC 6764 because it ignores the RFC 6764 entries in my domain and forces an OAuth UI, forcing me to use the main Fastmail password rather than the fine-grained app passwords available via https://caldav.fastmail.com/ HTTPS Basic login.

The fix I'm requesting is simply to bring back the old behavior of adhering to RFC 6764 and using HTTPS Basic login at https://caldav.fastmail.com/ when the RFC 6764 entries are present. Bringing up Fastmail's OAuth UI as a fallback when RFC 6764 entries are not available in the DNS for the domain is completely fine—however that new behavior should not override and ignore RFC 6764 records such as _caldavs._tcp when they are present.

There's no incompatibility. Authentication still happens the same way - OAuth just sets you up with basically stand in credentials to use instead of the normal username/password.
I'm not sure what problem you're describing. You're hosted at fastmail, you can authenticate and use the calendar if you just follow the prompts. If you don't follow the prompts, it won't work. You should NOT enter app passwords to the OAuth dialog, but the normal main ones.

The problem is that I should not be seeing the OAuth dialog at all! If RFC 6764 is being followed, I should get a dialog to enter my Fastmail app password , which will be used for HTTPS Basic authentication via https://caldav.fastmail.com/. If RFC 6764 were being followed, OAuth should not come into play at all. What I am describing is how it used to work.

you can authenticate and use the calendar if you just follow the prompts

First of all, I think there is a confusion about authentication versus authorization. OAuth is an authorization protocol. If OAuth is used Fastmail performs authentication (or not—maybe I'm already logged in, in which case no extra authentication occurs) and authorizes my access, returning some token back to Thunderbird.

In HTTPS Basic Authentication directly with https://caldav.fastmail.com/, Thunderbird truly is performing authentication using my app password credentials.

I want to authenticate directly with https://caldav.fastmail.com/ using my app password credentials, which as I mentioned uses finer-grained permissions than the account-level Fastmail password.

Expectation: My RFC 6764 DNS records are correctly set up, so I should be seeing a dialog to enter my app password for authentication at https://caldav.fastmail.com/.

Reality: Instead, Thunderbird is delegating to OAuth for authorization, not using my app password, but using my account-level credentials.

Authentication still happens the same way

  • One is an authentication; the other is an authorization protocol.
  • One uses the Fastmail app password; the other uses the Fastmail account-level credentials.
  • One has fine-grained permissions; the other has coarse "you either get full access or you don't" permissions.

So there are three ways right there that the two flows are different.

Again I stress that this is changed behavior. It worked before. Now RFC 6764 is not being followed.

This works fine with other CalDAV clients, e.g. Davx5. It used to work fine with Thunderbird.

If this is not clicking after you re-read carefully my in-depth description yesterday, I would be happy to set up a Zoom web meeting with you and walk you through it.

I also want to point out that I spent days verifying all this with Tier 3 senior Fastmail technical support so that I would make sure that I was providing correct information. Their last message to me, after having noted that Thunderbird defaults to OAuth even if I have RFC 6764 DNS entries set up to use CalDAV HTTPS Basic authentication, was:

Yes, this is true, you need to manually enter caldav.fastmail.com as the server name to force a password prompt [instead of using OAuth]. Unfortunately there isn't much we can do here. Hopefully TB addresses this in their future updates.

The OAuth creds obtained through the OAuth process are credentials authorized for a specific purpose: in this case, for calendar access. That's why it says in the window that it needs you to authorized access to your calendar.
This has nothing to do with RFC 6764. By using OAuth it's just different credentials being used to authenticate the requests. They are not account-level. Through the OAuth setup you authorize creating a set of credentials that will be used for a specific purpose, to be used by a specific app (Thunderbird).

Why it changed? Because Fastmail added OAuth2 support.

The OAuth creds obtained through the OAuth process are credentials authorized for a specific purpose: in this case, for calendar access.

That's interesting. Because on 2023-10-09, Fastmail Tier 3 support told me:

Currently, with OAuth, the client is automatically given access to Email, Calendars and Contacts. So once you authenticate Thunderbird through OAuth, it gets access to your emails, calendars and contacts.

I don't know how to explain this discrepancy. Do you think maybe they misspoke?

That depends on which scopes are requested for the specific case you want. If it's only the calendar, then only the calendar scope ("https://www.fastmail.com/dev/protocol-caldav") will be asked. https://searchfox.org/comm-central/rev/451150f82556f927edc20a47d2b5f6fb396aa341/calendar/providers/caldav/modules/CalDavSession.jsm#262
Like I said, the OAauth window will say what it wants access to.

Like I said, the OAauth window will say what it wants access to.

This ticket is completely unrelated to "whether the OAuth window will say what it wants access to".

This ticket is requesting that, when user@example.com requests to add a calendar, if the DNS for example.com has a SRV record indicating _caldavs._tcp.example.com. with a value of 0 1 443 caldav.fastmail.com., that Thunderbird first attempt to authenticate to caldav.fastmail.com using Basic HTTP authentication over TLS before attempting to authenticate via OAuth. This request reflects previous Thunderbird behavior.

I recognize that you are attempting in various ways to convince me that either I am mistaken in one or more details, or that I don't really want what I assert that I want. As courteously as I can, I am assuring you that I have spent significant time researching this; as well as talking to Fastmail Tier 3 support; and in fact I'm am pretty darn sure that I really do want what I am requesting.

I understand and acknowledge that Mozilla may decide not to fulfill this request, and may close this ticket as "WONTFIX" or take any other action, as it has the right to do. I am simply raising a user request, which at this point I am confident is clear and has been succinctly stated. Thank you for your consideration.

(In reply to Garret Wilson from comment #11)

This ticket is requesting that, when user@example.com requests to add a calendar, if the DNS for example.com has a SRV record indicating _caldavs._tcp.example.com. with a value of 0 1 443 caldav.fastmail.com., that Thunderbird first attempt to authenticate to caldav.fastmail.com using Basic HTTP authentication over TLS before attempting to authenticate via OAuth. This request reflects previous Thunderbird behavior.

That is wontfix. For the absolute majority of users it is much easier not to have to create an app specific password through the website.
Your request is essentially "never use OAuth2 for Fastmail". The RFC is followed, only with other creds than what's

When setting up the calendar only, the OAuth2 creds created are only valid for calendar. The granularity is thus even the same (well, actually less, since they are for Thunderbird only) than those of an app password.

Status: UNCONFIRMED → RESOLVED
Closed: 2 years ago
Resolution: --- → WONTFIX
See Also: → 1984376
You need to log in before you can comment on or make changes to this bug.