1748416 - Unable to authenticate with Google mail using OAUTH2 and 2FA if web server is running on https://localhost

Status: NEW

(Thunderbird :: Account Manager, enhancement)

Description

[dev]

Steps to reproduce:

• Setup a GMail IMAP account with OAUTH2 enabled
• Auth window comes up and asks for the users email and password
• User is then challenged for their 2FA/U2F token

Actual results:

Once the token is accepted the window redirects to a page showing "insecure connection" and the URL bar shows https://locahost/....

I believe this was because I had a process running locally which was listening on port 443 (and 80) which intercepted the connection.

Expected results:

An error or warning should've been displayed saying that the auth process can't be completed because another process is listening on the required port, alternatively the port could be changed to avoid conflicts

Comment 1

[Magnus Melin [:mkmelin]]

Our connection to localhost isn't real though, AFAIKT...

Comment 2

[sztankovszky.bence]

Same issue here: Ubuntu 20.04 running Thunderbird 91.5.0 (64-bit)
If there's a Docker container running, web service exposed on host to tcp/80 (0.0.0.0:80->80/tcp) then using OAuth2, after a successful 2FA auth Thunderbird reports authentication failure.
Stopping docker container eliminates the issue.

Comment 3

[me]

I can confirm this too. I'm not sure if my web server's automatic localhost:80 -> localhost:443 redirect had anything to do with the issue here.

I was able to work around this by temporary shutting down my server.

I ran into this with Office 365's OAUTH2, not Google's.

Comment 4

[dezertdezine]

Thunderbird: 91.8.0 / Windows 10
Running Apache (localhost:80 & localhost:443)

Authentication appears to complete all steps correctly, but the OAuth password is not saved.

Pausing Apache temporarily allows authentication to complete correctly.

I imagine this will stump quite a few folk over the next few weeks, which is ​​Google's deadline for this:

May 30, 2022, ​​Google will no longer support the use of third-party apps or devices which ask you to sign in to your Google Account using only your username and password.

Comment 5

[Magnus Melin [:mkmelin]]

Maybe we can fetch("localhost") before starting the OAuth process. If it responds, tell the user they need to stop that process before proceeding.

https://searchfox.org/comm-central/rev/f27ea99a053692618e52893f1d543022053a47cc/mail/base/content/browserRequest.js#103

xref bug 1174797

Comment 8

[justin.cherniak]

Would running the access code callback on an alternate, random port be a possible fix? See what ports are in use, pick a random one, run access code workflow there?

Comment 9

[Magnus Melin [:mkmelin]]

No, not in general. The redirection endpoint is defined at the provider, and they may or my not allow using other ports.

Comment 11

[Mahmoud Al-Qudsi]

Microsoft works around this by using an application protocol redirect (which Google let them set up). The OAuth2 dialog redirects to outlook://foo/bar on Windows, which is registered to launch outlook.exe and pass in the url as a command line argument, which gets routed internally to the correct page.

I don't know if all platforms support custom protocols, but this should be the approach on all platforms that do.

Comment 12

[Mahmoud Al-Qudsi]

Actually, please ignore the caveat in my last comment: since the redirect is performed in a webview under gecko/thunderbird control, it can fully handle a redirect to thunderbird://oauth2/authorize?code=... without involving the system protocol handler at all, under any and all supported platforms and I don't see any reason why this can't be the preferred way instead of a localhost redirect (so long as it's possible to configure the oauth2 developer api token to redirect to thunderbird:// with the provider, and that should certainly be doable one way or the other).

Comment 13

[footmenkingdom]

I guess we found a solution, now need to create a patch.

Yeah, I got some issues with Thunderbird and local webserver too, using Thunderbird = disabling webserver which is not the case for me because it's supposed to run 24/24...

Comment 17

[nortschi1]

So maybe there is an entry in QA or other documentation that I could not find. Any hint how I should have searched?

Comment 21

[Paolo Marani]

Same here trying to authenticate to office365 via OAUTH2. My localhost is blocked by IIS running a web server. I need to turn IIS off just before authentication, and this is a pita! If a random port cannot be used, try to change using a protocol trick or at least inform the user that the port is currentrly busy and OAUTH2 cannot succeed in that specific case. There should be a way to detect if port 80 is in use by another application.

Comment 22

[Toby Pilling [:tobyp]]

As far as I can understand, we can't use a custom protocol like thunderbird:// as suggested in comment 12 because Google doesn't allow that and the road to approval is rocky. Beyond that, there are other providers to consider who also use loopback flows.

There are a couple of options I see as being possible as a shorter term fix, given that Google will accept any port on 127.0.0.1 or localhost.

1.) Use a fixed high port and add redirectionEndpoint: "https://127.0.0.1:4242", to our Google config in Oauth2Providers.sys.mjs - this may also have conflicts for some people (but much fewer) and could also be blocked by firewall/AV software.
2.) Append a dynamic high port after asking the OS for a free one - more complex and could also be blocked by firewall/AV software.

Magnus/Brendan what are your thoughts on this? I've tested option 1 and seems to work fine.

Comment 23

[Mahmoud Al-Qudsi]

Technically option 2 isn't really that much more complicated than option 1 (on both Windows and *nix, it's just one call to getsockname() after binding to 127.0.0.1:0 to get the actual port). For what it's worth, the default Windows firewall treats listening to port 80 the same way it treats listening to port 9865 - it blocks it by default if it's binding to anything other than 127.0.0.1 (or ::1, probably), but listening on localhost generally goes through. (On Linux, binding to port 0 to request any available port is a very normal part of many network utilities, so I'd be less concerned about that.)

I just feel like this problem is worth solving correctly for once and for all. There are a lot of ports, but there are also a lot of users running a lot of random software! (If you do go with option 1, I wouldn't pick anything remotely nice or memorable to minimize the chances of a conflict with a manually chosen port, though.)

Comment 24

[Magnus Melin [:mkmelin]]

I don't see how that would work. The redirection endpoint is registered at the provider and IIRC should be enforced for security reasons.
Updating endpoint would have to happen for all providers at the same time, for all versions out in the wild, which we obviously can't do. So it would mean registering new clients globally and force everyone everywhere to log in again with new client id.

Other non-dynamic ports could have the same issue as we have now.
I think we just need to inform the user when we detect this situation. Alternatively figure out if/how we can notice the call to the redirection endpoint even if the port had something running on it already.

Comment 25

[Toby Pilling [:tobyp]]

I don't think the redirection endpoint is registered with Google. I have access to the account and there doesn't seem to be anyhing there that would allow me to change it or add multiple valid redirect UIRs like I can in our Yahoo! account. According to Google documentation, we're just supposed to use any port at the loopback address - https://developers.google.com/identity/protocols/oauth2/native-app

As I mentioned in comment 22, I added redirectionEndpoint: "https://127.0.0.1:4242", to our config and it works fine. While I understand that other non-dynamic ports can still run into conflicts, I'd guess that we can find one that has fewer conflicts than port 80.

Comment 26

[FUMIO OYAMADA]

The redirection endpoint is registered at the provider and IIRC should be enforced for security reasons.

When authenticating with OAuth 2.0 on a web service, the endpoint is pre-configured for security reasons.
However, it is common for PC client apps to use a random port on 127.0.0.1.

To comply with the specification, many OAuth providers allow you to register authorized redirect URIs in a format that does not specify a port number, such as http://127.0.0.1, which effectively allows all ports within http://127.0.0.1:port.

Loopback IP address http://127.0.0.1:port or http://[::1]:port

Query your platform for the relevant loopback IP address and start an HTTP listener on a random available port. Substitute port with the actual port number your app listens on.

Note that support for the loopback IP address redirect option on mobile apps is DEPRECATED.

It should be acknowledged that Thunderbird is not a compliant implementation.

Comment 27

[kljbonnema]

Relevant documentation:
RFC 8252 – OAuth 2.0 for Native Apps
https://developers.google.com/identity/protocols/oauth2/resources/loopback-migration
https://developers.google.com/identity/protocols/oauth2/native-app

In summary - Gmail enforces 127.0.0.1 but allows a redirect to a different port. This will work on TB desktop.
For android (and probably iOs) the use of 127.0.0.1 is restricted for security reasons. Google specifies alternatives for native apps. This is probably the way to go for TB mobile.

Comment 28

[Paolo Marani]

The same EXACT issue exists also for other OAuth2 providers, for example, Microsoft Office 365 mail.
It should be possible to use a dynamic port for the loopback after credentials are entered.
The bare minimum would be to just inform the user that some other application is busy on the required 80 port.
But switching to a random free port would be even better, allowing OAuth2 authentication to work flawlessly.