1518126 - OAuth2 uses prefs.js userName rather than realuserName

Status: RESOLVED FIXED

(Thunderbird :: Security, defect)

Description

[Jan]

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36

Steps to reproduce:

Under Account Settings, switched a GMail IMAP account from password auth to OAuth2 auth.

This is an older gmail account where user name was initially freely selectable (not @googlemail.com). Later the user name changed to <username>@googlemail.com and this was set in the Thunderbird GUI. This worked fine until OAuth2 became forced upon by Google.

Due to the account username history, prefs.js contains 

user_pref("mail.server.server8.realuserName", "<username>@googlemail.com");
user_pref("mail.server.server8.userName", "<username>@hut.fi");



Actual results:

Upon switching to OAuth2, an authentication pop-up window opened.

In the title of the window it said authenticating <username>@hut.fi, i.e., the old user name as in prefs.js userName.

The pop-up authentication finished seemingly successfully.

However, checking for new mail on this account leads to authentication error.

To resolve this I did the following:

Went to Account Settings and completely deleted the account. Then removed in Google settings the secure app entry of Thunderbird. Next, again in Thunderbird I added back the <username>@googlemail.com account.  Then went through the OAuth2 pop-up window steps. They again complete successfully. After this the prefs.js contained userName (and lacked a realuserName entry). Subsequent checks for new email work fine. No more authetication error message.


Expected results:

Would have expected OAuth2 pop-up to use the correct user name internally stored in Thunderbird in the authentication, i.e., the most current user name also shown in GUI in Account Settings (rather than the old internally stored user name from times of the account creation in Thunderbird).

That is, would have expected OAuth2 pop-up window to use prefs.js realuserName, when present, rather than prefs.js userName.

Comment 1

[Jorg K (CEST = GMT+2)]

Sigh, more of the realhostname/realuserName mess :-(

Comment 2

[st601486]

Pinging this ticket.

I have run into the same problem. In my work environment, we used to have an on-premises Exchange server, using local usernames. We migrated to Office 365 and, in some cases, people just updated the server to outlook.office365.com and changed their username to their full email address.

Now we are trying to enable 2-factor authentication, which requires Modern Authentication (Microsoft is also planning on disabling Basic Password authentication, in any case, but that's been delayed because of the pandemic); but some users, who have been around since before on-premises went to cloud, are running into problems since their realuserName is correct but their userName is their old, on-premises username, which will not work with Office 365.

Also, somewhat related, the error messages that come up when authentication fails is weird, because it also refers to the hostname instead of the realhostname value.

Comment 3

[st601486]

This bug is turning into a deal-breaker. I've got a user with like 50 filters built up and a lot of emails. Deleting the account and trying to re-add is going to be a headache. Could someone take a look at this?

Comment 4

[st601486]

Not sure if the "need info" means I should put in more info, but I probably should have anyways. I've tried Thunderbird 78.1.1 on Mac and 78.1 on Windows.

Steps to reproduce issue. This requires some faking out if starting fresh. It's helpful if you have an Office 365 account that is not yet using two-factor authentication.

1. Set up new profile for an "existing account" in Thunderbird
2. Use the Configure Manually option and enter the old Exchange "on-premises" server information and old, on-premises username, which, in our case, was a local username without any @<domain> stuff)
3. Click Advanced config to save the settings (since I've already migrated to cloud, Thunderbird's validation step fails) Use Basic authentication (which is what we've been using up to now)
4. Manually fix the username to the full email address (which O365 uses) and set the server to outlook.office365.com. Keep Basic authentication for now.
5. Check mail to confirm that you can get email, assuming you do not have 2-factor enabled yet.
6. Go back into the email configuration and change to OAuth2
7. Check for mail. Note that the old on-premise username gets autofilled (which is the first sign of problems). Fix the address and log in as you normally would.

Expected result: You get email
Actual result: You get an error message that authentication failed to the old "on-premises" server

Comment 5

[Magnus Melin [:mkmelin]]

https://searchfox.org/comm-central/rev/45c9a2810df38a7d8b4a9a396e244a9b438f3729/mailnews/base/src/OAuth2Module.jsm#37 - I guess this should be realUsername instead

Comment 6

[st601486]

So what's the process for fixing the bug in Thunderbird 78?

This could also a problem for our users who decide to get married and change their primary email address to reflect their married name or people who move between groups inside our organization (some groups use custom domains) As far as I can tell Microsoft Office 365 wants us to use the primary email address. Just updating the userName field manually seems to cause issues with filters and manually selected junk mail and sent messages folders.

So far, the only "solution" seems to be to delete the profile and start again, if someone's username (which in the Office 365 cloud seems synonymous with email address) changes.

Comment 8

[Magnus Melin [:mkmelin]]

Attachment: bug1518126_oauth2_realusername.patch

Seems this is all that's needed.

It's slightly tricky to test it properly. What I tested:

1. set up a gmail account, make sure it works
2. go to account settings | server settings and change username to (not have @gmail.com) - run into bug 1668742 (YMMV, I'm not sure that's allowed for all gmail accounts, for mine it seems to be)
3. restart tb, then get the Oauth2 auth prompt again, and after that it still works

If I change the username to some other email address things don't work, but they shouldn't either. For this case we don't have the best UI - there's a notification about auth failure, but one has to figure out that what's needed is to (re) enter the correct server username for things to work.

Comment 9

[Alessandro Castellani [:aleca]]

Comment on attachment 9179174 [details] [diff] [review]
bug1518126_oauth2_realusername.patch

Review of attachment 9179174 [details] [diff] [review]:
-----------------------------------------------------------------

Following all the steps works as reported.

I get this error in the console when interacting with the OAuth dialog, not sure if it's related in any way.
```
JavaScript error: resource://gre/modules/LoginManagerPrompter.jsm, line 775: TypeError: can't access property "show", PopupNotifications is undefined
JavaScript error: resource://gre/modules/LoginManagerParent.jsm, line 971: NS_ERROR_XPC_JAVASCRIPT_ERROR_WITH_DETAILS: 
[JavaScript Error: "can't access property "show", PopupNotifications is undefined" {file: "resource://gre/modules/LoginManagerPrompter.jsm" line: 775}]'
[JavaScript Error: "can't access property "show", PopupNotifications is undefined" {file: "resource://gre/modules/LoginManagerPrompter.jsm" line: 775}]'
when calling method: [nsILoginManagerPrompter::promptToSavePassword]
```

Comment 10

[Magnus Melin [:mkmelin]]

Yeah that's existing, and harmless though annoying.

Comment 11

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/16aa7a3301fd
OAuth2 prefs should use realuserName instead of username. r=aleca

Comment 12

[st601486]

(In reply to Magnus Melin [:mkmelin] from comment #8)

It's slightly tricky to test it properly.

In case this is of use for testing, here is a slightly contrived scenario to reproduce the issue for an Office 365 user. Assume autodiscover does not work to set your settings - I've never had it work, but maybe that's our company. These steps were followed on Thunderbird 78.3.1 the Mac

1. Chose New > Existing Mail Account
2. Enter info, but in a typo in your email address [This is contrived error but not unimaginable]
3. Wait for the automatic discovery to fail. This never works in our environment anyways, so it's not unexpected.
4. Manually enter correct server (outlook.office365.com 993 SSL, smtp.office365.com 587 STARTTLS)
5. Click Advanced config so that you can change Authentication to OAuth2 [At this point the typo in the username gets saved in prefs.js]
6. Check for mail. You notice the typo in your email address when the SSO (single sign on) page shows up. Go back to Account Settings / Server Settings and fix the typo in username
7. Quit Thunderbird and reopen to get prompted for the OAuth2 stuff again. Notice the typo still appears in the SSO page. Fix the typo in the SSO page. Still fails.

As far as I can tell, you will "never" get this working without deleting the account and re-adding; or manually messing with username in prefs.js.

The side-effects this scenario do not expose are any problems with default Sent, Junk, etc mailboxes and mailbox filters.

Comment 13

[Magnus Melin [:mkmelin]]

Comment on attachment 9179174 [details] [diff] [review]
bug1518126_oauth2_realusername.patch

[Approval Request Comment]
User impact if declined: after changing username, can't get OAuth2 to work
Testing completed (on c-c, etc.): on c-c
Risk to taking this patch (and alternatives if risky): small change, and shouldn't cause problems.

Comment 14

[Wayne Mery (:wsmwk)]

Comment on attachment 9179174 [details] [diff] [review]
bug1518126_oauth2_realusername.patch

[Triage Comment]
Approved for beta

Comment 15

[Rob Lemley [:rjl]]

Thunderbird 82.0b3:
https://hg.mozilla.org/releases/comm-beta/rev/b491ee4114ca

Comment 16

[Wayne Mery (:wsmwk)]

Comment on attachment 9179174 [details] [diff] [review]
bug1518126_oauth2_realusername.patch

[Triage Comment]
Approved for esr78

Comment 17

[Rob Lemley [:rjl]]

Thunderbird 78.4.0:
https://hg.mozilla.org/releases/comm-esr78/rev/c72357384f5c