Closed Bug 1673446 Opened 3 years ago Closed 2 months ago

imap repeated authentication/password in 78.4.0 (Yahoo OAuth + other IMAP accounts that do not have passwords saved)

Categories

(Thunderbird :: Security, defect, P1)

Product:
Thunderbird
Component:
Security
Platform:
Unspecified
All
Type:
defect

Tracking

(thunderbird_esr91 affected, thunderbird_esr115+ fixed, thunderbird117 affected)

Status:
RESOLVED FIXED
Milestone:
118 Branch
Tracking Flags:
Tracking Status
thunderbird_esr91 --- affected
thunderbird_esr115 + fixed
thunderbird117 --- affected

People

(Reporter: bill-gh2, Assigned: gds)

References

(
URL
)

Details

(Keywords: dupeme)

Attachments

(3 files, 3 obsolete files)

debugger console output
6.65 KB, text/plain
Details
output of printf statements
2.16 KB, text/plain
Details
Bug 1673446 - Don't delete ram password when stored login changes on a different account. r=mkmelin
48 bytes, text/x-phabricator-request
wsmwk
: approval-comm-esr115+
Details | Review

User Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0

Steps to reproduce:

No change to configuration, just an upgrade to TB. Now I'm prompted to re-enter my password if the previous imap-pull (or smtp-push) for this account was older than 40 minutes or so (time not measured). Once the password is entered once per fetch/send, the password works, this is not a problem with mis-typing the password.

The behavior is the same for imap fetch and smtp send.

Win10, TB-78.4.0, and I do have a TB master password (and it is entered correctly). I do not store my email passwords in the TB credentials manager (I use keepass).

Actual results:

After upgrading to TB-78.4.0, I am prompted for my password for both sending and receiving multiple times within the same TB session. This is only in one of my three IMAP accounts:

  • simple imap server at hostmonster; normal password (nothing else supported)
  • outlook.office365.com; oauth2
  • imap.mail.yahoo.com; oauth2
    "hostmonster" is the problem account, the other two behave normally.

The laptop's network connection does not appear to glitch, and it does not hibernate/sleep. The AV/FW setup is standard windows, nothing special, and nothing has changed between TB-78.3 and TB-78.4 (or any time recently).

Expected results:

Before upgrading to TB-78.4.0 (78.3?), I would enter my email account password (once for fetch and once on-demand for send) at the beginning of the TB session. Once done, unless the password was changed or I restarted TB, I was not prompted for this password again.

Console debugging the most recent time it just re-asked for the password:

```
Attached file debugger console output (obsolete) — 
Console debugging the most recent time it just re-asked for the password:

```

```

Debugging console, the most recent time it just re-asked for the password:

Attempt to set a forbidden header was denied: Content-Length 2 cardbookWebDAV.js:386:17
NS_ERROR_FAILURE: Component returned failure code: 0x80004005 (NS_ERROR_FAILURE) [nsIAutoCompleteInput.popup] LoginManagerChild.jsm:174
    observe resource://gre/modules/LoginManagerChild.jsm:174
    handleEnter chrome://global/content/elements/autocomplete-input.js:596
    handleKeyDown chrome://global/content/elements/autocomplete-input.js:561
    AutocompleteInput chrome://global/content/elements/autocomplete-input.js:38
[OverlayManager] Injecting into new window: chrome://messenger/content/newmailalert.xhtml
[OverlayManager] Injecting into new window: chrome://messenger/content/messengercompose/messengercompose.xhtml
[OverlayManager] Loading: chrome://cardbook/content/contactsSidebar/ovl_cardbookContactsSidebarMain.js
[OverlayManager] Injecting: chrome://cardbook/content/contactsSidebar/ovl_cardbookContactsSidebarMain.xhtml
[OverlayManager] Loading: chrome://cardbook/content/attachvCard/ovl_attachvCard.js
[OverlayManager] Injecting: chrome://cardbook/content/attachvCard/ovl_attachvCard.xhtml
[OverlayManager] Loading: chrome://cardbook/content/autocomplete/cardbookAutocompleteSearch.js
[OverlayManager] Loading: chrome://cardbook/content/autocomplete/cardbookAutocomplete.js
[OverlayManager] Loading: chrome://cardbook/content/ovl_cardbookComposeMsg.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookWindowUtils.js
[OverlayManager] Loading: chrome://cardbook/content/observers/cardBookObserverRepository.js
[OverlayManager] Loading: chrome://cardbook/content/observers/cardBookComposeMsgObserver.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookLocales.js
[OverlayManager] Loading: chrome://cardbook/content/ovl_cardbook.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookEncryptor.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookIndexedDB.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookActions.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookCardParser.js
[OverlayManager] Loading: chrome://cardbook/content/ovl_init.js
[OverlayManager] Injecting: chrome://cardbook/content/ovl_cardbookComposeMsg.xhtml
[OverlayManager] Stylesheet: chrome://cardbook/content/skin/cardbookAutocomplete.css
[OverlayManager] Stylesheet: chrome://cardbook/content/skin/cardbookComposeMsg.css
[OverlayManager] Adding <CardBookKey> (key)  insertafter <key_fullZoomReduce>
[OverlayManager] Adding <cardbookMenuItem> (menuitem)  insertafter <tasksMenuAddressBook>
[OverlayManager] Adding <cardbookComposeToolbarButton> (toolbarbutton)  insertafter <button-save>
[OverlayManager] Loading: chrome://cardbook/content/cardbookEncryptor.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookIndexedDB.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookActions.js
[OverlayManager] Loading: chrome://cardbook/content/cardbookCardParser.js
[OverlayManager] Loading: chrome://cardbook/content/collected/ovl_collected.js
[OverlayManager] Injecting: chrome://cardbook/content/collected/ovl_collected.xhtml
[OverlayManager] Loading: chrome://messenger/content/addressbook/abCommon.js
[OverlayManager] Loading: chrome://cardbook/content/lists/cardbookListConversion.js
[OverlayManager] Loading: chrome://cardbook/content/lists/ovl_list.js
[OverlayManager] Injecting: chrome://cardbook/content/lists/ovl_list.xhtml
[OverlayManager] Injecting into new window: chrome://calendar/content/calendar-event-dialog.xhtml
NS_ERROR_NOT_AVAILABLE
[OverlayManager] Injecting into new window: chrome://messenger/content/newmailalert.xhtml
[OverlayManager] Injecting into new window: chrome://global/content/commonDialog.xhtml 2
[OverlayManager] Injecting into new window: chrome://messenger/content/messengercompose/sendProgress.xhtml
[OverlayManager] Injecting into new window: chrome://global/content/commonDialog.xhtml
[OverlayManager] Injecting into new window: chrome://messenger/content/aboutDialog.xhtml
[OverlayManager] Injecting into new window: chrome://messenger/content/newmailalert.xhtml
[OverlayManager] Injecting into new window: chrome://messenger/content/aboutDialog.xhtml 2
OverrideError: An entry font-size-label of type message is already defined in this bundle
OverrideError: An entry window-close-key of type message is already defined in this bundle
OverrideError: An entry startup-label of type message is already defined in this bundle
OverrideError: An entry focus-search-shortcut of type message is already defined in this bundle
OverrideError: An entry close-button of type message is already defined in this bundle
OverrideError: An entry font-size-label of type message is already defined in this bundle
OverrideError: An entry window-close-key of type message is already defined in this bundle
OverrideError: An entry startup-label of type message is already defined in this bundle
OverrideError: An entry focus-search-shortcut of type message is already defined in this bundle
OverrideError: An entry close-button of type message is already defined in this bundle
NotSupportedError: CustomElementRegistry.define: 'conversation-browser' has already been defined as a custom element conversation-browser.js:853
    <anonymous> chrome://chat/content/conversation-browser.js:853
    <anonymous> chrome://messenger/content/customElements.js:34
    <anonymous> chrome://messenger/content/customElements.js:37
    observe resource://gre/modules/MailGlue.jsm:201
    initHTMLDocument resource:///modules/imThemes.jsm:741
    onStateChange chrome://chat/content/conversation-browser.js:62
Attempt to set a forbidden header was denied: Content-Length 2 cardbookWebDAV.js:386:17
[OverlayManager] Injecting into new window: chrome://global/content/commonDialog.xhtml
debuggee 'resource://devtools/shared/base-loader.js:289' would run builtin-modules.js:180:9
[OverlayManager] Injecting into new window: chrome://messenger/content/newmailalert.xhtml
Uncaught TypeError: this._rowMap[aRow] is undefined
    getCellText chrome://messenger/content/folderPane.js:1067
folderPane.js:1067:7
    getCellText chrome://messenger/content/folderPane.js:1067
Uncaught TypeError: this._rowMap[aRow] is undefined
    getCellText chrome://messenger/content/folderPane.js:1067
folderPane.js:1067:7
    getCellText chrome://messenger/content/folderPane.js:1067
[OverlayManager] Injecting into new window: chrome://messenger/content/newmailalert.xhtml
[OverlayManager] Injecting into new window: chrome://global/content/commonDialog.xhtml
NS_ERROR_NOT_AVAILABLE: ActivityManager.jsm:127
    getActivity resource://gre/modules/ActivityManager.jsm:127
    removeActivity resource://gre/modules/ActivityManager.jsm:80
    onFolderRemovedFromQ resource:///modules/activity/autosync.jsm:244
    _timerCallback resource:///modules/AppIdleManager.jsm:30

(Sorry for the repeat console outputs; tried inserting the data twice, got a "error: null" both times, though it appears it succeeded. Then I inserted the console line-by-line. I cannot delete or hide the comments.)

Attachment #9183820 - Attachment is obsolete: true

We've 2 reports about this password issue in 78.4.0 in german support forums. When one account is set up to use OAuth2, the other IMAP and POP accounts loose there login in every session, even though the passwords are stored in password manager. The reporters say that the problem will go away as soon as OAuth2's Yahoo account is switched back to password login (for testing purposes).

Thanks Alex. I'm guessing it's the presence of OAuth2 and not just Yahoo, is that right?

As an update, an interesting property: TB will notify me of new incoming mail on the problematic account (notification window), meaning that it still has authentication somewhere to see that a new message exists. However, the new messages in the notification window are not in the actual inbox, and when I try to find them it asks for my password. So it seems as if one thread keeps the auth and one thread does not.

Component: Account Manager → Security
Summary: imap repeated authentication in 78.4.0 → imap repeated authentication/password in 78.4.0
Whiteboard: [dupme?]

@bill-gh2
I've to ask, to understand the setting completely: Your passwords are intentionally not in Thunderbirds password store? Normal case woud be to be asked once per session for your passwords. But you are prompted every 40 Minutes instead of once per session?

Alex, that is correct. Up through TB-78.3, the password would be cached in the session (until TB restart). It's interesting to me that TB-78.4 still notifies me that there is new email (including From: and Subject:) in the windows notification window, but it does not necessarily download the email into my inbox (until I re-enter the password).
(And I haven't timed "40 minutes". 5 minutes is fine, an hour typically is not, so it's something in between.)

This is happening to me as well (Windows 10, Thunderbird 78.4.0). I don't save my passwords in the Password Manager. For me, during a session Thunderbird asks me for the ougoing SMTP password every time I send an e-mail, even if there is only one minute between e-mail sends. Have been using Thunderbird for more than a decade and have never seen this behavior before.

Confirm there are Support Forum issues mentioning the same.
https://support.mozilla.org/en-US/questions/1312628

This behavior persists in TB 78.6.0 (32-bit)on win10.

Also in TB 78.6.0 (64-bit) Windows 10 still new prompts for the password.

I am not sure, but probably it is related. For the account which shows this behaviour I have a few folder from which I have set Thunderbird to delete messages older then 2 months. The thrash folder is one of them. I noticed that in the Thrash folder there are still messages 3 weeks older than 2 months. I read my mail a few days per week, so, this should not happen. It might be that this is caused by the same problem, that Thunderbird forgets the password. I assume that for deleting the messages also the password is needed.

Also TB 78.6.1 (64-bit) Windows 10 still new prompts for the password.

Another data point:

I began seeing this behavior when I "upgraded" from TB 68 to TB 78. Currently I'm at TB 78.7.1 (32-bit) on Windows 7 SP 1 and it's still happening.

I have two email accounts for which I choose not to store the password in TB. They are hosted by completely different mail services. One of them uses IMAP and the other uses POP3. This annoying problem happens with both of them, exactly as the OP describes: password is prompted for and successfully entered when TB starts up, but some time later it starts re-asking for both receiving and sending. In fact, with the IMAP account, it re-prompts when I select other folders even when I just entered the password on a different one a few minutes before. And once it starts "forgetting" the password for SMTP server authentication, it subsequently requests the password on every single message I send. It's extremely annoying.

Like the OP, I have noticed that TB continues to update by showing new messages in the INBOX list, but does not re-prompt until I try to look at them.

There was mention above of this happening when TB also handles a Yahoo account using OAUTH2. I do have one of these (POP3). Unfortunately, Yahoo recently changed its authentication method for "insecure" <grrrr> mail clients and now mandates the use of a generated gibberish password unique to each individual device/program being used to access the email account. The proposed solution to switch back to password login is not supported. (It also makes no sense that two completely separate accounts could interfere with each other this way.)

At the time I did the TB upgrade to 78, there were no other changes to any servers or to my computer. Adjusting the parameter for TLS version didn't make any difference. (I didn't expect it to, since I was still able to send and receive, just with this extra PITA.)

I would really like the original behavior back, where I enter the password at TB startup (and the first time I send through the SMTP server after that), and then it retains the password info for the remainder of the session. Please let me know if I can help. I did try getting the Error Console output, but the timing of the actual errors don't correlate with the behavior as they show up right from the start (appended). Thank you.

------------ Sample Error Console messages --------

14:32:39.692 Error: Cannot find loader for bootstrap XPIProvider.jsm:1887:15

14:32:39.705
Error while loading 'jar:file:///C:/Program%20Files%20(x86)/Mozilla%20Thunderbird/omni.ja!/chrome/messenger/search-extensions/twitter/manifest.json' (NS_ERROR_FILE_NOT_FOUND) Extension.jsm:570
readJSON resource://gre/modules/Extension.jsm:570
onStopRequest resource://gre/modules/NetUtil.jsm:128

14:32:40.002
[Exception... "Component returned failure code: 0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH) [nsIXPCComponents_Utils.readUTF8URI]" nsresult: "0x80520001 (NS_ERROR_FILE_UNRECOGNIZED_PATH)" location: "JS frame :: resource://gre/modules/L10nRegistry.jsm :: L10nRegistry.loadSync :: line 658" data: no] L10nRegistry.jsm:658:19
loadSync resource://gre/modules/L10nRegistry.jsm:658
fetchFile resource://gre/modules/L10nRegistry.jsm:573
generateResourceSetSync resource://gre/modules/L10nRegistry.jsm:478
map self-hosted:240
generateResourceSetSync resource://gre/modules/L10nRegistry.jsm:473
generateResourceSetsForLocaleSync resource://gre/modules/L10nRegistry.jsm:415
InterpretGeneratorResume self-hosted:1151
next self-hosted:1099
generateBundlesSync resource://gre/modules/L10nRegistry.jsm:177
InterpretGeneratorResume self-hosted:1151
next self-hosted:1099
touchNext resource://gre/modules/Localization.jsm:167
regenerateBundles resource://gre/modules/Localization.jsm:552
activate resource://gre/modules/Localization.jsm:243
<anonymous> chrome://messenger/content/mailWidgets.js:37
<anonymous> chrome://messenger/content/customElements.js:34
<anonymous> chrome://messenger/content/customElements.js:37
observe resource://gre/modules/MailGlue.jsm:201

I'm not familiar with the normal flow and timeline for Thunderbird bugs. While I don't expect immediate turnaround (especially for a "convenience", not a security bug), what is the expected response for something like this? 4 months and 4 minor versions have passed since I originally posted the report, and the only responses appear to be assigning a component, assigned dupes, and me-too comments.

(I apologize, I feel clumsy on BMO, is it impolite to ping/request-info?)

Flags: needinfo?(vseerror)
Flags: needinfo?(bugzilla)

I'm also not sure what protocol is for asking about a bug's status. But I do feel this is a security issue, as it has meant I've saved my password for a very sensitive account. I'd very much prefer never to have the password for this account saved.

I'm seeing this as well with TB 78.7.1 on Linux Mint 20.1 Ulyssa with base Ubuntu 20.04 focal and with TB 78.8.0 on Win10.

I'm running thunderbird with several mail accounts, all IMAP. One of the accounts is a Yahoo accout that was switched to OAuth2 authentication during the upgrade from TB 68 to TB 78. IMAP-passwords are not stored in the password safe. I have filter rules sorting the mails into the appropriate IMAP folder. Thunderbird is configured to query the mail accounts upon startup and regularly afterwards, so I'm getting a password dialog for every mail account upon startup. Afterwards I could access all email in every IMAP folder fine with every TB version < = 68.x.

TB 78 seems to forget the entered passwords at some point in time. This happens rather sooner than later. Sometimes it pops up the password dialog when I switch from one folder to the next within the same IMAP account, even though I just entered the password the second before. Then I have to enter the password for accessing every single folder in that IMAP account. And after a certain time, it forgets the passwords again and the procedure starts anew. I also see new mail having being sorted in the correct folders, but when I click on the folder, I am queried for the password. I am also queried for the password after waking up from suspend to RAM.

I can workaround the bug by

  • storing the IMAP passwords in the password safe or
  • switching authentication mechanism back to password for the Yahoo account

When I switch only Yahoo IMAP authentication back to password, then as long as I don't send an email via Yahoo SMTP server with OAuth2 authentication, sending and accessing mail just works as expected with every account. After sending the first mail via OAuth2 autentication, password problems start again.

The status of this bug is: we need steps to reproduce, otherwise nobody can fix it. What you're seeing can't be very common, so we (you) need to find the common denominator.

Thank you, Magnus.

I appreciate the need for reproducibility. Yours is the first mention in this post that something is thought to be lacking, and since there are duplicate issues (in BMO as well as external forums, mentioned above), this does not appear to be an isolated event. I can't speak to "common", as I don't have a measure of the full deployment, how many users have mixed oauth2/non-oauth2, how many choose to not use TB's password manager, etc. Regardless, I'll try to fill in some more details.

The only variance for me is "time since I last looked at the non-oauth2 account". That is, if I bring TB up after a long time of no use, no matter which mailbox I look at in the non-oauth2 account, I have to retype in my password. Then, for some time (usually no more than "minutes"), I'm good and do not need to reauthenticate. If I put it down (minimized) for a while, it repeats. If it's been minimized for even hours, TB still notifies me of new mail (so some thread still retains auth), but when I switch to the TB window, the new emails are not listed, instead I am asked for a password. After I type in the password, only then does it add the new email (that the notification told me TB had received) to the inbox.

I have three email accounts: two oauth2 IMAP (yahoo and office365) and one simple IMAP (ssl/tls, normal password). The problem occurs when either of the first two are set to authenticate via OAuth2; if both are not OAuth2, the problem goes away.
Until last week, I've had only one extension installed, CardBook ... and I previously tested for a couple of weeks without it, no change.
I choose to not have the password stored by TB's password manager, instead using an external pwd mgr.

To me, it seems that the common denominators are:

  • windows (win10 pro, 64bit, though I think just windows is the commonality)
  • at least one OAuth2 auth account, at least one non-OAuth2 account
  • not using TB's password manager

I want to provide a more reproducible example, certainly. I tried to set up a clone of the account, but TB does not allow two accounts with the same username. I am not willing to delete the active account to try to set up another, as a previous test like that required restore-from-backup.

Would it be useful if I set up a new profile, step-by-step set up two of the accounts, and then do another dump of debug info? That doesn't necessarily help you reproduce it on your end (without my credentials), but perhaps it gives a fresh look at it.

What can I provide that will give sufficient information for this?

Sure, it's got to be something. I just say not common since the number of yahoo Thunderbird users is likely in 6 digits, so a few reports is not much.

Yes, setting up a new profile and add one account after the other (and don't change any defaults) could be helpful.

(In reply to Phil Thien from comment #19)

I'm also not sure what protocol is for asking about a bug's status. But I do feel this is a security issue, as it has meant I've saved my password for a very sensitive account. I'd very much prefer never to have the password for this account saved.

I second this opinion. The only accounts for which I don't save the password are the ones whose access security I care about most.

As described above, I am seeing the exact same behavior as bill-gh2. I completely understand the need for reproducibility in order to investigate, but four key common factors have been elucidated (maybe not fully clear if limited to Windows?), additional peculiar behavior related to authentication has been clearly described by multiple people, and some of us have also attempted to provide additional debugging/error data. If there is something specific you are looking for which we would reasonably be able to obtain and have not given you, please tell us what that is.

In the meantime, a simple test for replication would be to create a Yahoo email account and set it up to use Yahoo's required OAuth2 authentication method. (They have a set of steps to generate a device/app-specific password when using apps like TB.) Then remove the saved password for some other non-OAuth2 account in TB, restart, and see what happens when those passwords have only been entered manually.

Per the comments/references in this thread, there are eight distinct individuals reporting the problem here, in a TB version that was only released about six months ago. Unless you know how many Thunderbird users overall, and with Yahoo email accounts in particular, 1) have upgraded to TB 78, 2) know about Bugzilla, and 3) are willing and able to make the effort to report problems here, you can't make assumptions about how common it is. I was a sysadmin for 25 years, and a substantial fraction of users would be filtered out at each of those stages.

A note on the timing of this bug appearing for me: the Yahoo mandatory OAuth2 authentication change happened prior to my upgrade to TB 78, and this problem did not occur with TB 68. The Yahoo account is not displaying any difference in behavior with TB 78; only the ones without saved passwords are (one IMAP, one POP). Something changed in 78 that allows one account to interfere with others. Shouldn't they be completely independent?

Thank you.

(In reply to Magnus Melin [:mkmelin] from comment #21)

The status of this bug is: we need steps to reproduce, otherwise nobody can fix it. What you're seeing can't be very common, so we (you) need to find the common denominator.

I get the password prompt every single time I send an e-mail even if TB has not restarted. Note that I don't save the password in TB's password storage.

Note that this is not how TB behaves when using imaps: once I have typed in the password the first time after TB starts I never have to type it in again (until TB restarts, of course).

(In reply to Magnus Melin [:mkmelin] from comment #21)

The status of this bug is: we need steps to reproduce, otherwise nobody can fix it.

What exactly are you missing to reproduce the bug? All you need is one IMAP account with OAuth2 and one without OAuth2 authentication, configure them inside TB 78 on your favourite OS. Make sure to set a master password, use the password safe only for the oauth2 credential and configure querying of mails at startup and storing a copy of sent mails in your sent folder. And store the oauth2 credential in the password safe. So far for the test setup.

  1. restart TB
  2. enter your master password at the master password prompt
  3. enter the password for the IMAP server of your non-oauth2 account at the upcoming prompt
  4. click on the inbox folder of your non-oauth2 account
  5. send a message with your non-oauth2 account
  6. enter the password for the SMTP server of your non-oauth2 account at the upcoming prompt
  7. click on the sent folder - and there's the bug: you're getting another password prompt for the IMAP server

If you need more info from our end, just let us know what exactly you are looking for and how we can produce it. I'm sure we'll be grateful to supply it.

Flags: needinfo?(vseerror)

So using master password is a required step?

(In reply to Magnus Melin [:mkmelin] from comment #28)

So using master password is a required step?

No. I do not use one (because the idea is that the important email accounts have their passwords entered manually at TB startup). I expect ras_isaiah included that for security purposes. It is not a necessary condition to see the problem.

I have two Google accounts (one private, one for work), both using Oauth, One hotmail account (using IMAP) and one other IMAP account for another work mail server. (So, I do not have a Yahoo account en I do not use a master password.) I store most passwords in Thunderbird's password manager (where I use application specific passwords). The IMAP work mail server does not have the option for app specific passwords, so I use my general work password, which I do not want to store in the password manager. So, I retype it each time Thunderbird requests it. That used to be once a day, but now it is many times a day. It may be related with my setting to update new messages in my mail every 60 minutes. Sometimes, I can access folders without retyping the password, sometimes it requests a new password soon after reading another folder of the same server. (Maybe one of the other servers was checked for new messages in between.) I notice that after I while I am no longer informed about new messages in the IMAP Inbox. Similar things happens when sending mail to this server. Sometimes I can send several mails without password prompt, but than again suddenly it prompts for the SMTP and the IMAP password.
Not only is this very annoying, I am not sure about the security status. If my password disappears without explication, how do I know that it is not used (and exposed) at places where I do not know of?

In addition: I use this setup for a few years already and It always worked correctly. The problem started last November, when I upgraded from version 68 to 78.

(In reply to Magnus Melin [:mkmelin] from comment #28)

So using master password is a required step?

you asked for steps to reproduce the problem, I gave you a test setup and steps. Are there other setups and steps? Most probably.

Do my test setup and steps not trigger the bug on your machine?

Support Forum lists several people with issue: https://support.mozilla.org/en-US/questions/1304806

Example comments all from different users on different OS.

"To emphasise that I have the same problem (running Linux 5.4.0-48-generic #52-Ubuntu, and Thunderbird 68.10.0 64-bit). Exactly as "gargamon" describes, Yahoo required OAuth to be set as the IMAP authentication factor, but after that the other four accounts I have attached to Thunderbird (none of them is Gmail), which all have "Normal Password" IMAP authentication, frequently re-prompt for password and fail to save draft messages to the remote server if I take too long to compose an email. I haven't measured the timeout period but fancy it's about ten minutes. Yahoo does not prompt for its (imap) password at startup. The other accounts do, and I do not really want to switch this off (and can't find out how to), as an extra layer of security to prevent unauthorised access to my emails.
NOTE: I solved the problem only by removing Yahoo from the accounts attached to Thunderbird, which is inconvenient since now I have to log on to Yahoo webmail to get my emails. "

"In OSX, after creating an Oauth POP account, TBird is suddenly asking me for the passwords to my previously existing POP accounts, EVERY TIME I Send or Download. Previously (that's for many years) it only asked once, after restarting TBird."

"I too have this issue but in Windows: W7 78.5.1 (32-bit) I have 5 accounts on 3 servers. The only one that does not seem to have an issue is the yahoo one. The others are SSL/TLS Normal Password. The issue affects SMTP as well as IMAP. "

Possible thought:
If user does not want to save passwords and user creates an account either POP or IMAP that uses OAuth or has eg: Yahoo updated auto to OAuth, then if OAuth account is set up to auto check every X minutes, all session stored passwords seem to get forgotten prompting for password when connection to server occurs.

AS A TEST:
Could those effected by this issue, access the Account Settings > Server Settings for accounts that use OAuth and disable the option / uncheck the checkbox 'Check for new messages every X minutes'
Then restart Thunderbird and enter passwords as normal.
Observe what occurs regarding getting prompts for password.
Please report back with your observations and results.

Anje, great post, thank you for the detailed context. (I retract my previous supposition that "windows" was a meaningful attribute.)

And for your suggested steps, here's what I did:

  • Of my three accounts (1 imaps and 2 oauth2), I disabled "check start startup" and I disabled "check every ... minutes" for the oauth2 accounts, I kept them enabled for my imaps account;
  • I closed and reopened TB, it asked for the imaps password, nothing more;
  • over the course of two hours, I periodically read email in the imaps account, various folders, it never re-prompted for my password;
  • I sent one email from the imaps account, and it correctly prompted for its password; later in the same two hours, I sent another email, is did not re-prompt me for the password; so far so good;
  • at the two hour mark (coincidentally), I did a single "check for new mail" for one of my oauth2 accounts, entered my password for the oauth2 account, and things worked;
  • 10 minutes later (because I forgot to check at 3 or 5 minutes), I looked at one folder in my imaps account and it re-prompted for my password.

At this point, both oauth2 accounts are still disabled for "check at startup" and "check every ... minutes", so it appears to be the one singular oauth2 poll that affected my imaps connection.

I hope this is helpful and clear.

I was just redirected to this thread from an older open issue with the same problem. I'll summarize my comment, for brevity's sake: all passwords (or sessions?) are flushed as soon as an e-Mail account with OAuth2 does something (check/fetch/send e-Mail). Doesn't matter if POP3 or IMAP.

Obviously if the passwords are saved in Thunderbird the issue doesn't arise, because (I would guess) the passwords are automatically and silently re-filled.

(In reply to Magnus Melin [:mkmelin] from comment #21)

The status of this bug is: we need steps to reproduce, otherwise nobody can fix it. What you're seeing can't be very common, so we (you) need to find the common denominator.

Combining the details of both threads, it seems like the point-of-problem has been narrowed down to code referencing oauth2 is having side-effect on (all?) other remembered passwords in the current session. For any user that is using TB's password manager, the problem is masked, since forgotten passwords are just re-queried silently. I suspect the code that is touching oauth2 is not trivial, but that's a lot clearer than when this bug report started.

Magnus Melin, you were also involved earlier in Varstahl's referenced thread (I'm not criticizing), what's the best way forward? Is there any more debugging information that we can provide? Is there specific logic for retaining both or closing one of the bugs as duplicated?

I think it seems reasonable at this point to invite somebody(ies) involved in programming/managing "oauth2" and "session password management" to one or both of the discussions. (If that's you, Magnus, that's great!)

I took Anje's excellent suggestion a few hours ago and can confirm that since I disabled automatic checking for new mail on the Yahoo OAuth2 account, I have not been prompted for any unsaved IMAP, POP3, or SMTP passwords beyond initial entry. This is the expected behavior. As an additional check on the hypothesis, I did a manual "get messages" on the Yahoo account, followed by sending myself a test message from one of the others. I was prompted for its password to make the SMTP connection, even though I had just sent a message with it a few minutes prior.

A timeline which may help narrow down the introduction of the problem:

20200629 - upgraded to TB 68.6.0
20200826 - converted Yahoo accounts to OAuth2 with Yahoo-generated app-specific passwords (saved in TB)
- no new problems with these or any other email accounts handled by my TB
20200902 - upgraded to TB 68.12.0, no problems
20210111 - upgraded to TB 78.6.0; had to regenerate Yahoo passwords for new major TB version
- THIS IS WHEN THE PROBLEM BEGAN

So whatever is causing this behavior appears to have been introduced after the 68.12.0 release. A report in the other thread mentioned seeing it in 78.2.0.

Summary: imap repeated authentication/password in 78.4.0 → imap repeated authentication/password in 78.4.0 (Yahoo OAuth + other IMAP accounts that do not have passwords saved)

POP3 are affected as well, not only IMAP @Magnus Melin [:mkmelin]. This bug is still present if all the accounts use POP.

Varstahl, for clarity ... is that all POP, or is it a mixture of POP and OAuth2 (just no IMAP)?

In my case all the servers are set up as POP, and two of them are POP with OAuth2 authentication instead of Normal Password. I just wanted to stress that it's not only IMAP related.

Also, the issue is not Yahoo related, it's globally OAuth2 related. I just confirmed the same behaviour with Gmail OAuth2.

Besides the negative impact on POP3 and IMAP accounts (I have one of each displaying the problem), it also affects SMTP passwords which likewise have not been saved in TB's vault.

Varstahl, thanks for the clarification. And yes, to me at least this is not specific to Yahoo, as I have an office365.com oauth2 account that triggers the same behavior. Ruth, good point.

Another point to clarify: we've mentioned the protocols (IMAP, POP, and SMTP), but haven't said much about other authentication methods. For me, it seems clear that "OAuth2" impacts "Normal password", but I know nothing about other methods. Can anybody confirm or refute similar behavior using "Encrypted password", "Kerberos/GSSAPI", "NTLM", or "TLS Certificate"?

And another point: it appears to me that this does not affect all TB threads. I'll receive a notification window indicating I have a new email in the non-oauth2 account, which suggests that at least one thread has retained the cached password. The basic details are available (From: and Subject:), but it does not appear in my Inbox and cannot be viewed until I re-submit my password. I don't know enough about TB's threading model, but I hope knowing what threads are impacted might be helpful.

I am concerned about this from a "security" perspective: sensitive information is being mismanaged. The fact that the problem appears so far to be purely "reductive" is somewhat relieving, but it does not completely mitigate the fact that passwords are being mismanaged in the first place. Perhaps it is over-zealous cache invalidation, or perhaps it's a memory-mapping issue. The former would be nice, the latter very concerning.

Some feedback from Support Forum:

I've been running with this setup (unchecking the "Check for new messages every X minutes" checkbox on all my OAuth accounts) this past week after Varstahl's update. I periodically manually fetch messages for these OAuth accounts and seem to have similar results.

A few things - at least for me, it doesn't forget the password immediately after fetching the mail for an OAuth account, it seems to have some time in between, but definitely seems related since I can go all day without checking those accounts and not typing the password, but then within a fairly short (I think less than half hour) period of fetching mail for that account it does seem to forget the password again.

Also, it doesn't seem to happen with my gmail OAuth accounts, only yahoo. If I fetch from a gmail OAuth account, it doesn't seem to forget the password for quite a long while (if ever) whereas if I fetch from yahoo, that's when it forgets within ~30 minutes or less. However, since the
forgetting effect isn't immediate, it's a little harder to tell for sure what's going on - perhaps it needs the combination of gmail + yahoo to trigger the bug, or maybe it really is just yahoo by itself.

Thanks Anje.

I'm not an oauth2 expert, but I wonder if the expiry of the access token could be at fault here. I believe the expiration time is a policy set by the originating IdP (yahoo, google, office365), so that might explain why different providers have different time effects on non-oauth2 accounts.

Here's a scenario in my head, not backed by code:

  • TB retrieves the access token for an OAuth2 account, uses it;
  • So far so good, non-OAuth2 accounts are not impacted;
  • One of the following:
    1. TB internally measures the ExpireIn of the access token, and invalidates that account's current access token when due; or
    2. TB attempts to use a token that the IdP determines has expired (for whatever reason); TB notices the fail and invalidates the oauth2 account's previous access token either before or after attempting to get/use a Refresh Token.

It seems reasonable that TB would invalidate/remove access tokens that have been denied (for any reason, including "expired"). But perhaps it's not indexing the accounts correctly, therefore the invalidation is too broad-reaching.

Perhaps this is adversely affecting all methods (including other OAuth2 accounts), but accounts that are OAuth2 or have the password saved in the pw-mgr are "self-healing" in a sense.

(In reply to bill-gh2 from comment #43)

And another point: it appears to me that this does not affect all TB threads. I'll receive a notification window indicating I have a new email in the non-oauth2 account, which suggests that at least one thread has retained the cached password.

In my experience the session is invalidated when it reaches OAuth2. In your example the order of mail check might be IMAP/POP related for a number of accounts and then OAuth2. When it reaches OAuth2 the other credentials are discarded, so you get the notification of incoming e-Mails, but then it can't download them 'cause in the meanwhile it forgot the credentials. At least that's what happened on my end.

(In reply to Anje from comment #44)

A few things - at least for me, it doesn't forget the password immediately after fetching the mail for an OAuth account, it seems to have some time in between, but definitely seems related since I can go all day without checking those accounts and not typing the password, but then within a fairly short (I think less than half hour) period of fetching mail for that account it does seem to forget the password again.

In my experience it's instantaneous, at least on 78.7.1. No difference whatsoever between GMail and Yahoo. Added a new GMail OAuth2 account today to test exactly this, no difference at all.

--

New test: I just noticed that there was an update to Thunderbird, 78.8, so I just updated. While on 78.7.1 GMail OAuth2 would behave exactly the same as Yahoo, on 78.8 it doesn't immediately discard the passwords for the other accounts. Yahoo OAuth2 still drops immediately as before. I haven't yet tested if on 78.8 passwords are forgotten over time, but I'm setting up to test that.

Update, a little less than a day after: while GMail OAuth2 was causing issues in 78.7.1, it works fine on 78.8. Yahoo OAuth2 keeps having problems. If we can figure out what changed between 78.7.1 and 78.8 for GMail or OAuth2, we might figure out how to fix the issue for Yahoo as well, and possibly other OAuth2 providers.

Flags: needinfo?(bugzilla)

FWIW, yahoo oauth issues from the past 1.5yr https://mzl.la/30yeIEl

URL: https://support.mozilla.org/en-US/que...

Using 78.8.1. Still prompting more than once for my IMAP password. I have no Yahoo account. I have two Google accounts using Oauth. Running the 64-bit version on Windows 10.

I meant Oauth2.

(Up front, I changed my display name from "bill-gh2" (an email alias I use) to "r2evans", my github handle.)

I think we are better able to narrow down the problem area. Varstahl's comment that with 78.8, the google oauth connection no longer invalidates the imap passwords. I am trying various combinations, but I know that the yahoo oauth2 will invalidate imap, and I believe that the office365 oauth2 does not invalidate imap. Perhaps it's "just" in the yahoo code?

I don't see anyone assigned, so I'm guessing either that it's still a prioritization issue. Is it still, however, a problem with reproducibility? I thought the response after the previous concern (of lacking reproducibility) was helpful. If the devs still need more, though, please say something so this doesn't languish.

Thank you!

Magnus Melin [:mkmelin]
Alex Ihrig [:Thunderbird_Mail_DE]
Wayne Mery (:wsmwk)

In 78.8.1 (Windows 64-bit) the Google Oauth2 still invalidates the IMAP account. I have no Yahoo accounts. I never used Yahoo. I have two Google accounts using Oauth2. Thunderbird still repeatedly prompts for my IMAP account's password. So, it is not "just" in the yahoo code. So, it is also in the Google code.

Just to reiterate that the problem of invalidated passwords affects POP3 and SMTP authentication as well as IMAP, so its impact isn't specific to any one of those protocols (probably use the same password-caching code?).

I expect the TB developers are already painfully aware of this, but for others who aren't: even though Oauth2 is a "standard", there are apparently some fairly significant implementation differences between providers (see https://www.cloudsponge.com/blog/how-are-googles-microsofts-and-yahoos-oauth-implementations-different/ ). Also, according to Microsoft, there is an "ongoing compatibility problem between the updated Yahoo YDN OAuth provider endpoint and [Microsoft] Power Apps portals" (in article dated 20210302 https://docs.microsoft.com/en-us/powerapps/maker/portals/configure/configure-portal-authentication ). So those details may have a bearing on why Yahoo accounts more consistently trigger this problem.

Thanks, Ruth Milner. I shouldn't be surprised how implementation of Standards is almost always non-standard :-(

My biggest concern is not that the TB developers are aware of the complexities of OAuth2, but that this bug report is still Status: UNCONFIRMED, Severity "--", and Unassigned. I recognize that this doesn't necessarily mean that none of the TB-devs are aware of it, but I believe there has been ample discussion to confirm the consistent nature of the problem and most of the conditions required to trigger it. And being "security" (even if not egregiously leaking security information), I'd hope for a little more commentary/support/attention. (I don't want to sound like I'm expecting service for a free/libre product ... I'm not, I am very grateful for the efforts of I-don't-know-how-many-people.)

(If the intent of your comment was to reduce the number of "me too" comments, without being more familiar with BMO etiquette, I respectively disagree: part of the premise that this is not a lot of users should be confirmed or dispelled. And perhaps somebody else's comment will include a detail that has so far been missed. Either way, squeaky wheel and such ...)

r2evans wrote: "the premise that this is not a lot of users should be confirmed or dispelled"

Oh, I completely agree, and attempted to make that point in comment #25 above (third paragraph) - i.e. getting "only a small number of reports" does not at all translate to "only this many people are experiencing it". My most recent comment was to hopefully provide some insight into why the bug seems to appear more consistently with Yahoo accounts than with other providers using OAuth2. This had been puzzling me because I had (naively) thought that the same TB code would handle that particular authentication process the same way across the board, yet collectively the user reports didn't match that assumption. The every-provider-for-themselves implementation approach might explain that.

As you and others have pointed out, this bug is probably occurring for most (if not all) users who have an OAuth2 account plus a "normal password" one (maybe other types too), but TB silently recovers for any account handled by the built-in password manager so many people would never know about it - and hence never report it. However, the lack of noticeable symptoms in those cases does not mean everything is hunky-dory, and there might be further side effects or implications that haven't been identified/connected yet. Personally, that's my biggest concern on the security side.

In 78.9.0 (Windows 64-bit) the Google Oauth2 still invalidates the IMAP account. I have no Yahoo accounts. I have two Google accounts using Oauth2. Thunderbird still repeatedly prompts for my IMAP and SMTP account's password. I never used Yahoo. So, it is not "just" in the yahoo code, it is also in the Google code.

Bug still present in 78.9.1 (Windows 64-bit).

Bug still present in 78.10.0 (Windows 64-bit).

Hallo.
Same behaviour for me: I've never saved passwords in Password Manager (except for OAuth2 obviously), both Yahoo and Google's IMAP OAuth2 accounts with other IMAP accounts, and since upgrading from 68 to 78 TB asks for passwords again and again (receive, send, sent, saving drafts, ...).

Reading this thread I have now disabled the check every 10 minutes for the two OAuth2, but before I noticed that there were no more notifications from Yahoo and Gmail accounts: I had to manually select the IMAP inbox folder to make TB check and download new mails (and 'check every 10 minutes' was enabled); anybody else with this behaviour?

Regards.
Matteo

Bug still present in 78.10.1 (64-bits) Windwos.

Bug still present in 78.10.2 (64-bits) Windows, although I have the feeling that it takes longer to forget the password. ??

Bug still present in 78.11.0 (64-bits) Windows.

The Support Forum continues to report this issue:
https://support.mozilla.org/en-US/questions/1338306

I'm testing:
I have one gmail OAuth account - saved password incoming and smtp
I have two pop accounts - saved password incoming and smtp
I have one pop account - I have removed saved password, but this only effects my incoming because that account uses another smtp which does have a saved password.

Can all those who experience issue test the following:
NOTE: For this TEST: It is important that you do NOT choose to send any email from any mail account that does NOT save smtp/outgoing password.
Please read carefully, so the test results are useful and correct.

TEST-1:
Start Thunderbird.
Add password at prompt for all accounts that do not save password. I'm expecting OAuth account will have password auto saved.

Send email from Oauth account to all of the other accounts.

Do emails arrive on all other accounts either auto or if you select relevant Inbox and click on 'get Messages'.

If you have an account which uses a different smtp which DOES have password saved then please send email to all other accounts.
If you do not have any other account with a saved password for smtp, please for this test, do not use it to send any emails.
Please run this situation for a reasonable length of time - only send from accounts with saved password.

Do you find that all accounts continue to receive emails without requesting password?
Please report finding for TEST 1.

I am trying to prove that incoming password for accounts that do not save password remains fully functional in session providing you do not send from any account that does not save smtp password.

TEST-2:
Please - for the sake of trying to work out what is going on - could you now send an email to all other accounts from an account that does NOT have a saved password, you will get prompted for password, enter it - BUT on this occasion, please select the checkbox to save that outgoing password - this is important.
So you now have an account where incoming has no saved password, but outgoing does.
Please continue to send email only from accounts that now have a saved outgoing smtp password.

Are you still able to receive all incoming mail to all accounts?
Are you prompted for any password for incoming mail.
Please run for a reasonable length of time.

The above tests do need to run for a few hours as you need to be certain that not sending via an account which does not save password is not effecting an incoming on any account, meaning the incoming not saved passwords are remembered for session.

TEST 3:
Finally, send from an account that has no saved outgoing password to all other accounts. Enter password at prompt - do not save - just send.
Select Inbox and click on 'Get Messages' - repeat for each account to check for incoming mail.

Do you now find that after using an account that does not have password saved, other accounts are asking password for incoming mail?

Thank you for your co-operation.

The results should interesting.

Here are my test results (I have modified your test procedure for speed).

I am using 3 accounts:

  1. Free.fr, configured in POP3 + SMTP, password stored
  2. Outlook, configured in IMAP + SMTP, password not stored
  3. Yahoo, configured in POP3 + SMTP, password stored, using OAuth2, not used during the tests but this account has automatic mailbox check at start-up and periodically.

I have accounts 1 and 2 since a long time ago, but I first noticed the bug when account 3 (Yahoo) started to required OAuth2, and I reconfigured it with OAuth2 in Thunderbird.

The test can be long because Outlook remembers the connection. But ONLY when accessing the same folder. So checking another folder can make the test much faster, provided you have enough folders to do all the tests you want to do.

Your assumption seems wrong.

22:48 Launched Thunderbird
22:49 Read Outlook Inbox: IMAP password requested, entered, without storing it
22:50 Send mail from Free.fr to Outlook
Mail received in Outlook (inbox)
checking "Sent" folder in Outlook: OK, no password requested
22:52 Send mail from Outlook to Free.fr: SMTP password requested, entered, WITH storing it
Mail received in Free.fr, Thunderbird displaying notification "received mail" (I mention it because usually I do not get this pop-up)
22:53 Send mail from Free.fr to Outlook
Mail received in Outlook (inbox)
Thunderbird displaying notification "received mail"
checking "Spam" foldel in Outlook: IMAP password requested, this is not the bug

Looking at stored passwords: Outlook SMTP is stored.

Tested version: 78.11.0 (64 bits), on Linux (LUbuntu 21.04)

Thomas, can you reproduce this?

Flags: needinfo?(bugzilla2007)

Maybe I incorrectly assumed people were following the same steps to reproduce the issue, seems I was wrong. This is what works for me, 100% repro:

  1. Have one e-Mail account (POP or IMAP, doesn't matter) without storing the password within Thunderbird
  2. Have one account with OAuth2, preferably Yahoo, GMail also works but only in some builds, while seems fixed in the most recent ones.
  3. Fetch the e-Mails on the first account, how many times you want, no problems, doesn't matter if there are no new mails, password is asked only the first time.
  4. Fetch the e-Mails on the second account with OAuth2.
  5. Fetch the e-Mails on the first accounts, now it asks for the password again.

This is for the fast reproduction, but in reality anything relating to the mailbox with OAuth2 drops the password for the accounts with no OAuth2. So if instead of fetching you send an e-Mail, or Thunderbird just checks for mail in the background, all the other previously connected accounts stop working as well.

There was a specific reason for performing the tests in that order.
I'm trying to discover if once you have entered incoming password for an account that does not save it, you can move between all account folders with no problem - providing you do NOT send from an account that does not auto store password. In your case providing you only ever sent from eg: Free.fr

(In reply to Miguel Julier from comment #65)

Here are my test results (I have modified your test procedure for speed).
Tested version: 78.11.0 (64 bits), on Linux (LUbuntu 21.04)
I am using 3 accounts:

  1. Free.fr, configured in POP3 + SMTP, password stored
  2. Outlook, configured in IMAP + SMTP, password not stored
  3. Yahoo, configured in POP3 + SMTP, password stored, using OAuth2, not used during the tests but this account has automatic mailbox check at start-up and periodically.

The test can be long because Outlook remembers the connection. But ONLY when accessing the same folder. So checking another folder can make the test much faster, provided you have enough folders to do all the tests you want to do.

Yes that was the point - I wanted to be sure you could select any folder in Outlook or Free at any time, providing you had NOT sent from Outlook.
In other words, the session for incoming Outlook password was properly functioning.

This would rule out any stored session password issue between any non OAuth account.

22:48 Launched Thunderbird
22:49 Read Outlook Inbox: IMAP password requested, entered, without storing it

That is OK

22:50 Send mail from Free.fr to Outlook

That is ok because Free.fr stores outgoing password.

Mail received in Outlook (inbox)
checking "Sent" folder in Outlook: OK, no password requested

This is good, you were able to check another Outlook folder and not be asked for password - I'm assuming you could select any folder in Outlook and view emails even if you reselected a Free.fr folder - back and forth between any Free folder and any Outlook folder

22:52 Send mail from Outlook to Free.fr: SMTP password requested, entered, WITH storing it
Looking at stored passwords: Outlook SMTP is stored.
Mail received in Free.fr, Thunderbird displaying notification "received mail" (I mention it because usually I do not get this pop-up)
22:53 Send mail from Free.fr to Outlook
Mail received in Outlook (inbox)
Thunderbird displaying notification "received mail"

I note you were not requested a password for viewing email in Outlook Inbox - So I am assuming you did NOT select the Inbox to see the email instead you just noted a new mail had arrived- maybe folder turned blue.
Instead you selected the Outlook 'Spam' folder.

checking "Spam" foldel in Outlook: IMAP password requested, this is not the bug

But the incoming password had been forgotten for the session regardless of whether the outgoing had now been saved.
In which case, does the 'Activity Manager' show that the Yahoo OAuth2 account had accessed the server to check for mail and/or IDLE command pushed email before you selected the Outlook Spam folder ?
I would suspect that it did. This would explain why non Oauth POP or IMAP accounts are effected and why the timing can vary and why it can occur on various access to folders in the Outlook account which uses a session stored password as opposed to a saved password.

So whilst there are reports of various issues which make it sound like a different action is going on, it seems to boil down to one.

For some reason, when Oauth account checks server for mail using token, the session stored passwords for any account not using Oauth and not auto saving password is either not being used or has been removed.
It is occuring immediately following an Oauth connection which does not use password.
So at this point, could a null setting be being set for passwords in session ?

Workaround to help reduce password requests - For OAuth2 accounts - Setting a long time between auto checks or removing the auto check so relying on IDLE / Push may see less requests for passwords for accounts that do not auto save password because the Oauth2 accounts access to server may be reduced. When creating mail to send from OAuth2 accounts, perhaps consider using 'Send Later' and then sending as a batch.
But until fixed, maybe consider using the Password Manager to remember passwords in the interim if it starts to impact your sanity.

(In reply to Anje from comment #68)

22:53 Send mail from Free.fr to Outlook
Mail received in Outlook (inbox)
Thunderbird displaying notification "received mail"

I note you were not requested a password for viewing email in Outlook Inbox - So I am assuming you did NOT select the Inbox to see the email instead you just noted a new mail had arrived- maybe folder turned blue.
Instead you selected the Outlook 'Spam' folder.

checking "Spam" foldel in Outlook: IMAP password requested, this is not the bug

I DID select the Outlook Inbox are was requested no password.
In Outlook (IMAP), I keep access to the folders that Thunderbird recently modified: typically "Inbox", when receiving a mail, "Sent", when sending a mail (even though I do not open the "Sent" folder), and any folder I have recently opened. No password is requested for these folders (on the short term), while it is requested for the other folders.

And let me correct myself: the end of my previous message was "this IS the bug".

For me, I could reproduce the issue with:

account A: without OAuth2 (normal password), password not saved under password manager
account B: OAuth2 account. Specifically O365 account.

Thunderbird: IMAP advanced setting: set the maximum number of connections as 1, under both accounts.

  1. Logon account A. It only asks password once upon logon, and possibly once on the first email sending.
    It does not ask for passwords no matter how many clicks on 'Get Mails' and email sending.

  2. Logon account B. It does not ask for password because the token has been agreed upon when in account
    enrollment.

  3. Send an email under account B (OAuth2).

  4. Send an email under account A. Thunderbird immediately asks for password of account A.

  5. Click 'Get Mail' under account A. Thunderbird immediately asks for password of account A.

After typing password for both (4) and (5), Thunderbird does not ask for password of account A, until we do
step (3) and it ends up with (4) and (5) again.

This is what I have observed:

  1. When sending emails via OAuth2, the memory-saved password of account A becomes empty. Thunderbird
    asks for password when Password.isEmpty() == true.

  2. While we think that an IMAP connection must be persistent until Thunderbird program quits, in fact it is not:

    1. When one sends an email via SMTP,
    2. At the same account, the existing IMAP connection is closed. Another new connection is established.

I have no ideas why the existing IMAP connection must be closed when sending an email. Is it that the
'maximum number of connections' under IMAP advanced setting also counts for SMTP connection? But
SMTP connections are usually not staying connected (persistent): once the email is sent, the SMTP connection
could be closed (unlike IMAP connections which are persistent for performance issue).

I use 78.8.1 (64-Bit) on a Linux Mate System ( Ubuntu 20.04.2 LTS) and I have this Problem since Version 68.10
(https://www.thunderbird-mail.de/forum/thread/85604-konto-verlangt-immer-wieder-passwort-nach-einer-weile/?postID=468614#post468614)
Since I changed to oauth2 it always loose LogOn when I change the Account. With every Account. (I have 2 oauth from AOL and 6 Accounts with "normal" Password-LogOn)

Bug still present in 78.12.0 (64-bits) Windows.

The bug is STILL PRESENT in 91.0 (64-bit) Linux. Amazing!!!
Also, I have found that it is unable to subscribe any mail folders...........
Seems that OAuth2 module is an abandoned-ware.

By the way, I have just found out that the issue is absent when connecting with GMail (via OAuth2). It looks perfect: folders are available, it does not ask for passwords of my non-OAuth2 mail account.
But it ends up with same issue when with Microsoft 365 account... :-(

Bug still present in 78.13.0 (64-bits) Windows using a combination of IMAP servers and GMail (via OAuth2) servers.

Of course the bug is still in the code. It won't go away by itself. Someone has to fix it. So far thunderbird developers haven't shown any interest in doing so, even though we've told them how to reproduce the bug long time ago. All it would take is setting a breakpoint on the password memory cache invalidator, walk through the given steps and wait for the breakpoint to catch program execution. Should be a quick fix. But I don't know the code. Perhaps there is no password memory cache invalidator function and something just overwrites memory. Then we would be facing a serious security issue and I'd assume they would prioritize the bug up to the very top of their todo list. Perhaps the oauth2 code was donated, is badly documented and nobody dares to touch it. We don't know. I read this as: This won't get fixed. Take it or leave it. As long as you're not paying them to work on this issue or volunteering yourself, you just have 2 choices: Dump the affected email account or thunderbird. Or both ;o)

(In reply to Fred. Zwarts from comment #76)

Bug still present in 78.13.0 (64-bits) Windows using a combination of IMAP servers and GMail (via OAuth2) servers.

In fact I have just tried 91.0 under Windows x64, with GMail (OAuth2) and a non-OAuth2 IMAP account. It looks like
working perfectly ok. Thunderbird does not ask for non-oauth2 account password regularly (I did not save passwords).
All mail subfolders are available.

To me, the problem is on Microsoft 365 account. It gives the following problems:

  1. Thunderbird asks for password of the non-OAuth2 account from time to time.
  2. Thunderbird fails to subscribe folders under the Microsoft 365 account. While right-clicking the account and selecting "subscribe" could see all the available folders, none of them could be subscribed even clicking them on and 'subscribe'.

I understand that when Thunderbird is free for use, we cannot demand much. But if Mozilla has no intentions to fix the issue of the OAuth2 module, they should just take the module away.

With 91.0 OAuth2 works for me with GMail but still not with Yahoo. When I look in "Saved Passwords" in "Thunderbird Preferences - Privacy & Security" I find the following entries for GMail and Yahoo, maybe this could be a hint:
oauth://accounts.google.com (https://mail.google.com/)
oauth://login.yahoo.com (mail-w)

Sorry, I spoke too soon. In fact with 91.0 OAuth2 under GMail, it still asks for password for my non-OAuth2 account. But the timeout (and hence asks for password) is around several hours (instead of around several minutes under Microsoft 365). Accordingly, I don't think that OAuth2 under 91.0 really fixes anything.
In a security point of view, if this module is unmaintained, it should be removed.

Still for me OAuth2 under GMail works with 91.0 after having thunderbird a couple of days open, but I agree, if nobody is looking into it, it makes no sense to have it.

(In reply to rhstb from comment #79)

With 91.0 OAuth2 works for me with GMail but still not with Yahoo. When I look in "Saved Passwords" in "Thunderbird Preferences - Privacy & Security" I find the following entries for GMail and Yahoo, maybe this could be a hint:
oauth://accounts.google.com (https://mail.google.com/)
oauth://login.yahoo.com (mail-w)

re: the Yahoo account
When creating this yahoo account, did you uncheck the checkbox 'Remember Password' because you did not want Thunderbird to keep a record of the Oauth token password ?

I have tried to replicate by removing the saved password for a pop account.
then restart Thunderbird.
enter password at prompt for the account that does not have a stored password. Do not select checkbox to remember password.
I am able to move freely between:
pop account with a stored password,
mailbox://mail.btinternet.com (mailbox://mail.btinternet.com)

imap gmail account using Oauth which uses Password Manager 'Remember Password'
I see two entries one for normal password and one for Oauth
imap://imap.gmail.com (imap://imap.gmail.com)
oauth://accounts.google.com (https://mail.google.com/)

pop account with no stored password - prompted for password on start up - uses that password for session until I restart Thunderbird.
No reference to this account in SAved Passwords - 'Saved Logins' as it was removed and is not selected to Remember PAssword.

Sent out emails from password saved account to all other accounts.
Not seeing any problem with session related password. I can select account with no password saved and 'Get Messages' with no problem.
I can select and move between any account or folder repeatedly.

Currently testing on 78.13.0 Windows 10 OS

Can people with the problem please try the following and report back on results.
Please follow instructions and please do not skip over anything.

Check there are no programs running on computer which can 'clean up' any files eg: CCleaner, wiseCleaner or even some Anti-virus products can clean up files. If you have that type of product and use it then please make your thunderbird profile folders and files are exempt.

Restart Thunderbird
For any account pop or imap that asks for password at prompt - please enter password and select the checkbox 'Remember Password' before clicking on 'OK'.
So all accounts have password stored in Thunderbird.
Run for a while checking various accounts perhaps run for a day.

Exit and Restart Thunderbird a couple of times and use as it is for a while.
After using Thunderbird for a couple of sessions where passwords were saved...upon starting Thunderbird for a third time...

Allow time for all accounts to access servers for first time to check for mail.
Access stored passwords ...Menu app icon > Options/Preferences > 'Privacy & Security'
Passwords section - click on 'Saved Passwords'

For any accounts set up to use 'OAuth' please leave alone - let Thunderbird remember the password. Then the OAuth token can be stored, so it gets used instead of the password.

If you have a pop or imap account that does Not use OAuth - it uses the normal password and you would like to be asked for password - select that mailbox:// account line and click on 'Remove'. Repeat for the smtp:// line for same account.
Please only do this on one account at this point, so session will only need to store one password.

Please let forum know if you see something like : oauth://login.yahoo.com (mail-w)

Restart Thunderbird.
Enter password at prompt for the account that does not have password stored.

Please report on whether setting all accounts to remember password, running for a while to allow for a couple of sessions and then in a completely different session removing one of the stored passwords, had any effect on repeat requests for password in same session.
Please also mention if you had any program cleaning up files.

Anje,

before everybody is jumping through hoops. Are you saying, you can't reproduce the bug with the steps I've given in comments 20 and 27? From the description you're giving, your setup is different.

(In reply to Anje from comment #82)

(In reply to rhstb from comment #79)

With 91.0 OAuth2 works for me with GMail but still not with Yahoo. When I look in "Saved Passwords" in "Thunderbird Preferences - Privacy & Security" I find the following entries for GMail and Yahoo, maybe this could be a hint:
oauth://accounts.google.com (https://mail.google.com/)
oauth://login.yahoo.com (mail-w)

re: the Yahoo account
When creating this yahoo account, did you uncheck the checkbox 'Remember Password' because you did not want Thunderbird to keep a record of the Oauth token password ?

when I choose the OAuth protocol, a popup displaying a yahoo login page opens (it's the same with gmail), so there is no possibility to check or uncheck "Remember Password".

(In reply to ras_isaiah from comment #84)

Anje,

before everybody is jumping through hoops. Are you saying, you can't reproduce the bug with the steps I've given in comments 20 and 27? From the description you're giving, your setup is different.

In your case you only have imap accounts and one uses OAuth.
IF the OAuth account uses saved password then all is ok until you send, but perhaps on first sending you are asked for password and the act of asking for a password upsets the incoming session password.
So you have a problem when an smtp password request messes up session held password for incoming.

But in cases reported, the one common factor is one account uses OAuth and that password is not saved either, so it is likely that OAuth account was not initially created with a saved password regardless of whether set up using OAuth or later changed to OAuth.

It's important to rule out whether the OAuth account is causing the problem - so OAuth account needs to be set up/created from the start to remember password to see if all other pop and imap accounts (not remembered password) work properly in the session.

Normally, I have all accounts, both pop and imap(using OAuth), using Password Manager, so all passwords are saved. Hence, I would not see the problem.

I let the OAuth account remember password as it uses a token anyway - the account was created that way from the start. But I deliberately removed the saved password from a pop account to force that account to prompt for password when I restarted. I did not choose to allow it to be saved in order to see if I would get repeat requests for password after moving between other accounts and I did not get a repeat request for password. I only got a request for password when I initially restart Thunderbird, no matter how many times I restarted, so the pop account password was properly remembered for the session. I cannot reproduce the problem using that set up.

So what happens if people chose to remember password for all accounts - like me and did so for a couple of sessions and then reverted/removed saved password for one non oauth account, restarted, entered password at prompt but did not select to save - whether this act could trigger something to work correctly.
But it is important that all OAuth accounts need to be set up to use a remembered password and that may require the OAuth account to be removed and then recreated using 'Remember Password' checkbox.

I would like people to report back on whether recreating OAuth accounts set up to 'Remember Password' from the start then allows all other non oauth accounts not using a saved/remembered password to function correctly with passwords temp saved for session.

If this works in so much as it stops the repeat requests from non oauth accounts then we have progress.
Either way it may also help to point to what triggers the problem in the first place.

no, it's not all ok until I send. The steps I gave just trigger the bug right away. Otherwise you have to give it some time until it shows. Click a bit through the mail folders of a non-OAuth account, eventually you'll get the password prompt. Alternatively, grabbing a coffee gives thunderbird enough time to screw up the password cache as well ;o)

I'd like to point out as that IMAP and SMTP password prompting works as expected when the OAuth2 account is not used. The problem is really in the Oauth2 code.

Have you tried reproducing the bug with an IMAP non-OAuth2 account?

I'm not sure I've ever seen a bug generate so much confusion.

in cases reported, the one common factor is one account uses OAuth and that password is not saved either

This is not a common factor. In fact, I and others have reported that the OAuth account password is saved in TBird (otherwise you have to remember/write down/retype a long string of gibberish, from Yahoo in my case).

But in my testing, it doesn't actually matter whether the OAuth account password is saved or not. Either way, after that account is accessed by Thunderbird, any manually-entered passwords are re-requested. Users here have suggested that all ordinary passwords are probably affected, but in cases where Thunderbird has them stored it can re-retrieve them from the vault instead of prompting the user, so the problem isn't visible for those accounts. It should be possible to trace the code execution and verify whether that's what's happening.

I hope the line of recent questions doesn't have the goal that everyone seeing the problem should re-create the OAuth (or other) accounts in a very specific way, for several reasons: 1. that would be a lot of people having to re-create accounts; 2. a lot of users will never find out it's necessary; 3. it shouldn't be necessary; and 4. that's a workaround, it doesn't fix whatever is actually producing the problem. And I'm with the users who are concerned that something which breaks cached passwords may be a sign of a security-related bug like a buffer overflow.

Remember what I noted previously about the OAuth2 protocol allowing individual email providers to make their own tweaks. This could well explain why it seems to behave differently/inconsistently with different hosting services (Yahoo, Gmail, 365).

Personally, I have disabled checking the Yahoo account from Thunderbird. Most nights I'll do a one-time check right before I exit Tbird, but otherwise I look at it through K-9 on my phone. But I hope that "switch to a different mail client" isn't the permanent solution.

But in my testing, it doesn't actually matter whether the OAuth account password is saved or not. Either way, after that account is accessed by Thunderbird, any manually-entered passwords are re-requested.

But that is not the case for everyone. I'm not reproducing the issue. If it is not reproduceable it is not so simple to work out what is going on.

As I had everything using a stored password and then removed stored password to test and it worked ok for me, it was not an unreasonable request to see if resetting accounts to same as my setup to use saved password for all accounts for a short period - three sessions and then trying one non-OAuth account with no saved password to understand what was the effect.

Is there anyone willing to test this?

I am presuming no one has any program running which cleans up files.

If people do not want to mess up current settings in profile, then would anyone be willing to create a new profile, add one OAuth account which uses 'Remember Password' and one other account which does not use 'Remember Password' and test for a few sessions to see whether there are constant requests for password?

OK, but you stated that nobody seeing the problem had saved the OAuth account password, even though it's clear from multiple users' reports that they have. ras_isaiah's "how to reproduce" instructions in comment #27 actually say to save it. So I wasn't sure that your proposed steps were actually testing the same problem.

A key reason why your setup can't reproduce the bug might be that your OAuth account is on Gmail. Some people have reported that Gmail OAuth doesn't trigger the problem for them. varstahl noted this in comment #67, as have others. I re-read all the comments to date, and nobody has reported using a Yahoo OAuth account that kept things working normally. So the most reliable test would be to create a Yahoo account that authenticates with OAuth and has its password/token saved, and then see what happens on the non-OAuth+non-saved accounts when TBird is checking Yahoo regularly.

(In reply to Anje from comment #83)

Please let forum know if you see something like : oauth://login.yahoo.com (mail-w)

Yes, this URL entry has been in there for my Yahoo account since I first had to set it up to use OAuth, about a year ago. The token provided by Yahoo to use for OAuth authentication is 52 characters long, unlike the 16-character hash shown in the POP and SMTP servers' password entries.

To confirm what previously stated, as noted in comment #47 GMail was affected by the same issue as of version 78.7.1, but under the same profile, same settings, same external programs, it stopped making other clients forget the passwords since 78.8. In order to test the issue further, I created another throwaway Yahoo Mail account, and the issues persist up to today (I'm on 78.13 currently).

@Anje I'm willing to create a new profile, start from scratch and test these out, and I've read your comments, I've just been unable to due to work time constraints. But again, as Ruth Milner pointed out, if you're testing this on an OAuth that has been working for the past 6 minor versions, I don't think you'll ever repro, unless you install a previous build such as 78.7.

I also wanted to go diff-hunting to see how GMail's OAuth solved itself, but again, didn't have the time yet…

I looked in the thunderbird (ver 90.0.3) error console (Tools -> Developer Tools -> Error Console, Errors and XHR activated) and found the following output:

09:41:09.600 XHRPOST https://www.googleapis.com/oauth2/v3/token [HTTP/3 200 OK 95ms]
09:41:09.791 XHRPOST https://api.login.yahoo.com/oauth2/get_token [HTTP/1.1 200 OK 166ms]
09:41:18.815 Prompter: internal dialogs not available in this context. Falling back to window prompt. Prompter.jsm:1084

The difference between google and yahoo is the version number of the http request, also the response looks different, maybe this can be another hint where to look?

I am a bit surprised that people say that the problem with Gmail disappeared. I have Thunderbird 78.13 on 64 bit Windows 10. I have two Gmail accounts using Oauth2. I have no Yahoo account. I still have the problem with an IMAP account with normal password authentication. So, I am 100% sure that not only Yahoo accounts are causing the problem and that Gmail accounts have the same problem. It might be, however, that the problem is easier to reproduce with a Yahoo account.

FWIW:
I'm still having the issue with 91.0.3 (64-bit) Windows.
One Yahoo account
Multiple IMAP accounts; all password-losers.

I have been having the issue and documented under comment #71 (O365-OAuth2 + non-OAuth2 account) and comment #80 (GMail-OAuth2 + non-OAuth2 account). One thing which I should mention is that under the SMTP setting for all accounts, it is using SMTP-auth under port 587, with STARTTLS. That is:

O365: OAuth2, SMTP (outlook.office365.com) under port 587, STARTTLS
GMail: OAuth2, SMTP (smtp.gmail.com) under port 587, STARTTLS
non-OAuth2 account, SMTP under port 587, STARTTLS

As this topic has been heat again, I am trying GMail again. But this time, I set the SMTP of GMail so that it is:

GMail: OAuth2, SMTP (smtp.gmail.com) under port 465, SSL/TLS

With this configuration (together with the same non-OAuth2 account), Thunderbird has not asked for password of
my non-OAuth2 account. It has been for almost 24 hours since I have this configuration. Well, maybe I speak too
soon, and I will further conclude on next week (let it run for some days...)

I could not test for O365 as it does not support SMTP under port 465.

I have some sort of hypothesis of this. But probably I could conclude next week...

Could bug 1720657 be involved here, or this be a regression from bug 1682370.

StartTLS does appear to be problematical at the moment when it comes to security.

As mentioned in comment 96 I can confirm that I'm using GMail: OAuth2, SMTP (smtp.gmail.com) under port 465, SSL/TLS with no issues.

Also all other non OAuth accounts use same settings: port 465, SSL/TLS

All 3 of the accounts I tested have OAuth2 and SSL/TLS, StartTLS was not set. The issue is still here. There might be a common cause or be correlated, but mail sending is not the issue at hand (in its entirety, at least). As I said, Thunderbird asks password to send e-Mails, and in all my tests I didn't even try to send any, it fails before.

As data points, my Yahoo account that triggers the problem is using:

  • POP with SSL/TLS on port 995, and an OAuth2 authentication token generated by Yahoo
  • SMTP with SSL/TLS on port 465, same OAuth2 token as for POP access

So this isn't specific to StartTLS.

Again, I encourage any testers/debuggers to use a Yahoo account if at all possible, as there appear to be other complicating factors with Gmail.

The problem is not related to STARTTLS. I have non-OAuth2 accounts, some using STARTTLS while others use SSL/TLS, some even use one way for establishing TLS for sending and the other one for receiving. All are IMAP accounts. None of them have a problem until the OAuth2 account is queried. OAuth2 account is Yahoo and uses SSL/TLS both for IMAP and SMTP.

As Ruth has stated, she's accessing the OAuth2 Yahoo account via POP and sees the password problem also on non-Oauth2 POP accounts. So we can rule out any dependency on the mail retrieval protocol as well.

@Anje: Can you please try wether you can reproduce the bug with a Yahoo OAuth2 account? And is there any way those of us who see the problem can get Thunderbird to write a logfile with appropriate log level? Which config settings that differ in your setup and ours would trigger the OAuth2 code to invalidate the password cache entries?

I do not have a yahoo account, but this is not just Yahoo and it's not just imap non Oauth non saved password accounts. It would also seem to not just effect STARTTLS.

People who have this problem seem to have accounts that were already set up to not use a stored password when they changed some accounts over to OAuth. Or perhaps they were already not saving passwords for some OAuth accounts.

But....as far as I can tell....No one is reporting the problem where they use Password Manager normally for all accounts - all accounts were initially set up to use the Password Manager - and then deliberately removed stored password on non Oauth accounts and suddenly are experiencing same problem. Why is this?
I cannot reproduce. I'm sure if there were other developers who could test it they would be reporting they can suddenly reproduce the problem, but I'm not seeing comments that they can reproduce it. It does not matter if I test on imap or pop because there are reports of it occurring on both and it does not matter if I test with gmail OAuth or other because it is effecting people using only gmail.

So I'm asking those with the problem to create new profile and add two accounts where one is OAuth and at creation set both up to remember password - this is important. So at this point both accounts use Password Manager.
Then run for a couple of sessions, (restarts) abd then remove the stored password for the non OAuth account, so Thunderbird now prompts for password upon restart and then report back on results saying what server used the OAuth and whether accounts were pop or imap.
If it works ok then add a third account, set up to remember password which after a couple of sessions, remove stored password (assumming it is a non OAuth account) and report on results.

So far. I'm not aware that anyone has specifically tested this and reported back. This should be something people can test and results are important.

It's known what set up is not working as there are plenty of reports, but all of those reports involve accounts that were not set up initially to remember password prior to update. So are those people willing to try my setup and perform the test I'm asking people to try to see if you get the same results as me?

It's not just about finding out what is going on which is causing the problem, but also trying to see if there is a method of forcing a fix.

it does not matter if I test with gmail OAuth or other because it is effecting people using only gmail

This conclusion doesn't follow, though. People have reported that Gmail behaves differently depending on build/release/possibly other factors that haven't yet been determined. Therefore, using only Gmail to test for the bug is a flawed design because the bug isn't affecting everyone using Gmail. Evidently your setup is among those not affected, so it isn't really testing the problem.

To be able to reproduce the problem reliably, create a throwaway Yahoo account and set it up to use OAuth2 for authentication. Unlike Gmail, I don't think any of the reports here has mentioned a Yahoo OAuth2 account that doesn't cause the problem.

No one is reporting the problem where they use Password Manager normally for all accounts - all accounts were initially set up to use the Password Manager - and then deliberately removed stored password on non Oauth accounts and suddenly are experiencing same problem. Why is this?

Maybe people aren't reporting it spontaneously because they don't generally remove saved passwords? I just did a simple test of this. Setup:

  • OAuth Yahoo POP account with stored token and password which no longer checks for email automatically because it's the trigger
  • Two non-OAuth accounts with non-stored passwords (one POP, one IMAP), which are the ones affected by the problem
  • A non-OAuth account with stored password (POP)

Then I did this:

  1. Removed the stored password
  2. Restarted TB, entered the (now three) non-stored IMAP/POP passwords manually
  3. Ran for a while as usual
  4. Manually did Get Messages for the Yahoo OAuth account
  5. Did Get Messages on the non-stored accounts. The two POP accounts prompted immediately for their previously-entered password, including the one on which I had removed the saved password. The IMAP one, which is the "default" account in TB, continued to work for a while without prompting (similar to another description in early reports here), but it got slower at loading folders and did eventually ask again.
  6. Repeated steps 2-5 with identical results
  7. Re-saved the password in TB for the non-OAuth account which had been saved originally
  8. Repeated steps 2-5, with the same visible effect on the accounts with manually-entered passwords (1 POP, 1 IMAP) but no prompting on the saved one.

So in the "remove previously-stored password" situation, the bug does indeed appear on that account. Furthermore, going back to storing the password hides the bug's impact on that account. So I strongly suspect that whatever is going wrong inside the program isn't affected by whether or not the account password is stored; the only distinction is whether TB has to prompt the user for it or not.

I hope this is useful.

There does appear to have been some change at yahoo recently as some (Mostly US based users I understand it) have had to reduce the number of connections in advanced server setting to 3 from 5 to prevent yahoo spontaneously refusing to play. Your report that the account functioned slower and slower may indicate it may benefit from reducing the connections.

The account which functioned slower was an IMAP one affected by the bug, not the Yahoo account which triggers the bug. I have not had to make any changes to any TB settings recently, nor have I seen any new issues with Yahoo (yet).

The delay in being re-prompted for that IMAP account password may have been due to the IMAP server retaining the session connection for a longer period of time, so that TB wasn't required to re-authenticate immediately.

Has anyone tried running Thunderbird in safe mode (help menu > troubleshooting mode) while running their operating system in safe mode with networking?

This is a relatively niche issue, but I am yet to see a reliable way to reproduce shown here. There is a possibility other software may be implicated in the failure, and the safe mode tests tend to flush them out.

No one is reporting the problem where they use Password Manager normally for all accounts - all accounts were initially set up to use the Password Manager - and then deliberately removed stored password on non Oauth accounts and suddenly are experiencing same problem. Why is this?

Because if the users are using password manager, Thunderbird automatically refills the request without prompting the user, so the user doesn't even know there's a problem.

I am yet to see a reliable way to reproduce shown here

>_>

Has anyone tried running Thunderbird in safe mode

Yes, same result.

so time to look into the code:

  • what's the name of the function that invalidates the password cache entries?
  • where is it called from the OAuth2 code?
  • does Thunderbird have different OAuth2 code for Gmail and Yahoo?

and I know I asked before but didn't get an answer:

  • can those of us affected by the bug configure Thunderbird to create a meaningful logfile for you?

(In reply to Varstahl from comment #107)

Has anyone tried running Thunderbird in safe mode

Yes, same result.

The statement was "Has anyone tried running Thunderbird in safe mode (help menu > troubleshooting mode) while running their operating system in safe mode with networking?" without the operating system safe mode then the diagnosis is missing an important part.

One of the things I would think would be missing in the operating system safe mode situation is any anti virus or password manager that may be a causative part of this issue.

I'll just update the current situation and I owe an apology to you guys, because under the same conditions I actually failed to repro. Grabbed a fresh copy of Thunderbird, put two accounts in, and they work. Forget the password, put it back in, and they work. Open the other Thunderbird, and it doesn't.

So, my guess is the problem is not really the OAuth2 by itself, but maybe the OAuth2 when multiple accounts are connected to the same inbox. I'm testing this so I should have a 100% repro methodology at some point later today. This gets weirder by the day…

Nevermind, 100% repro, no folder sharing needed:

  1. have an account with POP3 (saved password)
  2. have a yahoo account set up as POP3 (IMAP works), using OAuth2
  3. Everything (seems) to work
  4. Remove the first POP3 account password, that doesn't use OAuth2
  5. Fetch mails for that account, password request, fill it in
  6. Fetch mails from the Yahoo OAuth2 account
  7. Fetch mails from the other account, password is requested again

@varstahl. If you rename the logins and key4 files does it still reproduce. While the data should be in memory, I have to wonder if what is there is corrupted from the start from reading the file.

Yes. Another OAuth2 prompt popped up, and it immediately forgot the other account's password. To be safe and make sure I didn't accidentally triggered a repro through some other custom settings, I intentionally wiped everything and started from scratch, just adding passwords (while remembering) and then manually removing the fist POP3 account password. No extensions, no themes, no nothing.

To be just 3000% sure, I'm going to restart everything in safe mode, but I don't expect the repro status to change.

100% repro with Thunderbird in Safe Mode while Win10 is in Network Safe Mode

I'll echo ras_isaiah's re-query about how those of us living with this problem can tweak TB logging to provide more data to help with debugging. At least with the default settings, there are no entries of any kind in my TB Error Console which have a timestamp anywhere near my tests. So whatever is going on doesn't seem to generate any normally-logged output.

I would like to further answer my comment #96. After staying my Thunderbird session of my PC (Windows 10) for over a weekend
(in fact, from last Thursday to today: Monday), my Thunderbird is still able to keep my non-OAuth2 account password without
asking me to re-type. To summarize:

  1. Thunderbird 91.0.3, win64
  2. non-OAuth2 account: password not saved under Password Manager. SMTP: STARTTLS under 587/tcp
  3. GMail OAuth2 : SMTP: smtp.gmail.com with SSL/TLS under 465/tcp

I do not have a Yahoo account and I am not interested in getting one. So, I am not able to test Yahoo case.

As mentioned in comment #96, I am thinking of 2 possible hypothesis:

  1. The problem is under the SMTP STARTTLS / OAuth2.
    But as some of you have mentioned that they had the same problem even when all accounts are in
    SMTP SSL-TLS / OAuth2, I don't think that this hypothesis is true.

  2. OAuth2 module might mix up with STARTTLS (or SSL/TLS) when 2 or more accounts share with the same
    SMTP connection method. In my case, GMail makes use of SSL/TLS and non-OAuth2 account makes use
    of STARTTLS. It looks like that it works ok so far (for 4 days at least). But if both GMail and non-OAuth2
    accounts are set with using SMTP STARTTLS, the problem comes after a day.
    Unfortunately, I could not test non-OAuth2 account under SMTP SSL/TLS, as the mail server is not under
    my control. Also, I could not test SMTP SSL/TLS under O365 account (in fact, O365 account is exactly
    the issue why I am here) as O365 does not support SMTP SSL/TLS under 465/tcp

Hi.
Bug is still in Linux TB-version 78.13.0 (64-Bit) present.
In this version TB loose the PW when you switch from one folder to the other, so I have to enter the PW for every folder extra.
(OAuth2 -Account I added are @aol.com (IMAP, SSL/TLS/993))
And after a while doing nothing the PW in the other IMAP-Accounts are lost.

Actual I find the error in my productive system Win10 21H1, Thunderbird 91.1.0 (64-bit), OAuth2, POP3, SSL/TLS.
Now I had also access to a Debian GNU/Linux 11 with thunderbird 78.12.0 (32-bit) fresh installed.
I can reproduce the error with my yahoo account with OAUTH2 authentication and another POP3 account with Normal Password authentication and "Use Password Manager to remember this password" unchecked.
The check for new messages interval is set to 10 min for the yahoo account. When I check new messages with the 2nd account within this interval the password is not requested, but as soon as the yahoo account is checked for new messages, the password for the 2nd account is re-requested.

Last month Thunderbird was automatically upgraded to version 91 on my Windows 10 64-bit PC. Since then I have not seen the problem anymore. My IMAP password is requested only once after startup and is remembered several days. I put my PC in hibernation when I switch off power and the next day, when it is powered up, Thunderbird still has access to my IMAP account without asking for the password. During these days I also use my two Gmail Oauth accounts.

The problem is still very much there for non gmail users, and it rubs me off that the status is still unconfirmed after producing 100% repro steps in comment #111. Not that it surprises me too much, there's also confirmed bugs still waiting for a fix after 13 years in this tracker…

I am at 91.3.2 and I STILL have this issue.
One yahoo account
no gmail accounts
5 IMAP/PW accounts

Bug is still present in Build 78.14.0 (64-Bit) for Linux (Ubuntu Mate)
With 1 Account of AOL (whats the same as Yahoo).

Bugs don't go away by themselves. Only when someone fixes them. So yes, of course the bug still exists in the new v91 releases.

(In reply to Fred. Zwarts from comment #119)

Last month Thunderbird was automatically upgraded to version 91 on my Windows 10 64-bit PC. Since then I have not seen the problem anymore. My IMAP password is requested only once after startup and is remembered several days. I put my PC in hibernation when I switch off power and the next day, when it is powered up, Thunderbird still has access to my IMAP account without asking for the password. During these days I also use my two Gmail Oauth accounts.

In fact, I had no issues under version 91, but only under GMail with SMTP auth method which is different from my non-Oauth SMTP auth method.
Specifically, my GMail account is set with SMTP Auth with 465/tcp/ssl, while that of my non-oauth account is 587/tcp/tls. When both accounts are set with 587/tcp/ssl, the same problem comes again.

It still fails under Microsoft 365 mail. I believe that it is because only 587/tcp/tls is available under my non-oauth account, and Microsoft 365 mail does not support 465/tcp/ssl (ie. only 587/tcp/tls is available).

(In reply to Matt from comment #97)

Could bug 1720657 be involved here, or this be a regression from bug 1682370.

StartTLS does appear to be problematical at the moment when it comes to security.

A good next step I think is for one of you who can reproduce this issue to find the regression range using https://mozilla.github.io/mozregression/

Status: UNCONFIRMED → NEW
status-thunderbird_esr91: --- → affected
Ever confirmed: true
OS: Unspecified → All

Because TB seems not to have the will or possibility to fix this Bug, I tried it with "BetterBird", a fork of TB which begins to fix such regressions.
I tried Linux Version 91.4.0-bb22 (64-Bit) but the Bug still appears in this fork.
(with one oauth-Account …@aol.com)
Maybe they will get this problem first.

Just want to add that Seamonkey's Mail has been having this problem too, for quite a while now. Currently on 2.53.10.2.

Which, of course, should not be a big surprise. I'll see if I can do regression testing, but I'd hate to lose my mailboxes.

Seamonkey 2.53.5.1 (17/11/2020) didn't have the problem, 2.53.6 (22/01/2021) did. Time flies like an arrow, fruit flies like a banana.

I went back to tb 60.0 for win64. I installed a yahoo account with imap and pop, and a 2nd account using pop, don't remember password.
My findings:
Yahoo: imap, SSL/TLS, OAuth2 shows the error already
Yahoo: pop3, SSL/TLS, Oauth2 does not work - the OAuth2 authentication popup didn't show up.
Would it make sense to go further back, because in the RN for ver 60.0 it is stated: "new OAuth2 authentication for Yahoo and AOL"

Keywords: dupeme
Whiteboard: [dupme?]

For me this is fixed in tb 102. OAuth2 works with yahoo and gmail and pop3 as expected. There were no further question for passwords.

This is still present for me in TB102, win11. The original problem remains: after having connected to the OAuth2 (e.g., Yahoo) for IMAP, my non-OAuth2 (IMAP) account frequently re-asks for my password for fetching new email.

See Also: → 1762797
See Also: → 1780846

I read the descriptions for bug 1780846 and bug 1762797. I don't think either of them is the same as this one at all. Both have to do with entering a password incorrectly first, and make no mention of OAuth2.

I started experiencing this on TB 102.3.1 (Win10) several days ago , when I had to switch an Office365 IMAP account from using a normal password to using OAuth2. Now, my Fastmail IMAP account has the recurring password request issue, the worst part of which is that when TB forgets the password previously entered, new messages simply don't show up in the message list unless I click on the inbox and reenter the password. Because TB silently fails like this, I often end up thinking there's nothing new in my Fastmail inbox when in fact this is not the case.

(In reply to wintogreen from comment #133)

I started experiencing this on TB 102.3.1 (Win10) several days ago , when I had to switch an Office365 IMAP account from using a normal password to using OAuth2. Now, my Fastmail IMAP account has the recurring password request issue, the worst part of which is that when TB forgets the password previously entered, new messages simply don't show up in the message list unless I click on the inbox and reenter the password. Because TB silently fails like this, I often end up thinking there's nothing new in my Fastmail inbox when in fact this is not the case.

Could be related to https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

You could also see what Error Console is showing for the account having issues via ctrl-shift-J. If there's any error related to this repeated password asking, it will likely show up in red "error" text.

I think Microsoft has a variant of vanilla OAuth2 in production for Exchange / Office 365 account users. I say this because the ticketing system we use at my org allows us to set up and use an authentication connector of type OAuth2 and "Microsoft OAuth2." Apparently not all OAuth2 are created and used equal.

Something you could do to test out whether the OAuth 2 that Microsoft is using versus what other providers might have going is at fault by grabbing Portable TB (https://portableapps.com/apps/internet/thunderbird_portable) and extract it to your desktop or some temp folder and set it up with the account that's giving you issues. It won't affect your current TB install as it's all self contained in it's folder and allows you to play around and compare behaviors. You could do that and compare server settings of your local install and portable TB and look for any subtle differences.

(In reply to Arthur K. (he/him) from comment #134)

Could be related to https://learn.microsoft.com/en-us/exchange/clients-and-mobile-in-exchange-online/deprecation-of-basic-authentication-exchange-online

I seriously doubt that: the problem is not with oauth2, but with accounts using regular password: the password for this regular account
is forgotten as soon as some other account (typically microsoft) uses oath2. This problem remains hidden for people who use a password manager
for the affected account as the password manager will resend the password, thus hiding the bug. All of this is detailed in some of the other posts.

I can confirm that this bug still occurs with version 102.3.2., but I must add that I use the betterbird version (102.3.2-bb19) of thunderbird.
The problem occurred for me in thunderbird versions as well. I run this software on linux (fedora 36)

Of course people start noticing the buig only if they have two or more email accounts, and at the time one of them is swicthed to oauth2.

You could also see what Error Console is showing for the account having issues via ctrl-shift-J. If there's any error related to this repeated password asking, it will likely show up in red "error" text.

Thus produces nothing useful. There is one error regarding "PreferDisplayName", but that is unrelated.

I think Microsoft has a variant of vanilla OAuth2 in production for Exchange / Office 365 account users. I say this because the ticketing system we use at my org allows us to set up and use an authentication connector of type OAuth2 and "Microsoft OAuth2." Apparently not all OAuth2 are created and used equal.

Could be, and microsoft creates many problems for thunderbird users, but this is not related in my opinion.

Something you could do to test out whether the OAuth 2 that Microsoft is using versus what other providers might have going is at fault by grabbing Portable TB (https://portableapps.com/apps/internet/thunderbird_portable) and extract it to your desktop or some temp folder and set it up with the account that's giving you issues. It won't affect your current TB install as it's all self contained in it's folder and allows you to play around and compare behaviors. You could do that and compare server settings of your local install and portable TB and look for any subtle differences.

Sorry to confirm this (still) exists in MacOS/102.3.2

It's been 2+ years, but it's still seemingly being ignored.

Please also see https://support.mozilla.org/en-US/questions/1338306, which refers to this bug.

The main conclusion there is that thunderbird caches passwords in ram, and uses them to
reopen imap connections as needed. If another account uses oauth2, then this cache
is cleared for all accounts instead of being cleared only for the oauth2 accounts.

So the next step would be to find the code that clears the password and fix it...

The problem disappeared for me more than a year ago.
Recently, I upgraded to 102.8.0 (64-bit) Windows and now the problem is back again for my imap account for which I do not save the password. Apart from a few imap accounts using normal password authentication, I have a few Gmail and Microsoft accounts using oauth2. I am not sure when exactly it started again. Maybe after the Thunderbird update, but maybe after a change of my Hotmail account from normal password authentication to oauth2. These two changes happend at approximately the same time.

Just updated to Thunderbird 115. Still same issue.

Someone must have just CC'd me on this bug but I've never seen it before. I've read about half of the above comments and I'm beginning to think that the oauth2 accounts are a red herring. What I see testing with the current daily version is that if I don't store a normal password for an imap account, I sometimes have to re-enter the password. And if I go offline and then back online I also have to re-enter the password. I tested with TB 68 on an old laptop and it remembers the password after coming back online after going offline in the same session. So it looks to me like the password(s) is getting cleared from memory cache somehow.
So I'm thinking that if somehow all the connections to the imap server get dropped for some reason, the cached password also gets cleared and has to be re-entered. This may also be true for POP3 and maybe SMTP, not sure.

Also, FWIW, there is a major problem with the password mgr screen on daily. If you delete a single account line, it looks like you have deleted ALL your security info on all accounts. But when I re-opened the password mgr screen, just the one I had selected was gone. (I thought for a while I was going to have to find and re-enter all my many test account passwords manually again, but didn't have to, which is good.)

(In reply to gene smith from comment #140)
Yes, very true. I recall that quite a while ago (over 2 years ago probably), I used to pull the source code and add some assertion statement to monitor the "password" variable. When Thunderbird asks for password, this password variable is null. When password variable is not NULL (provided that it is a correct password), Thunderbird does not ask for password. It asks for password all a sudden, but it asks when password
variable is empty.

Also, I note that by default, Thunderbird maintains 5 server connections to the IMAP server (on each mail account). It makes 5 IMAP connections to the server. But whenever I send an email, it creates an SMTP connection which seems to count as one connection. This makes one of the IMAP connections to terminate. But when sending another mail, another IMAP connection terminates. When Thunderbird 'feels' like reestablishing a new IMAP connection, it asks for password when it finds the "password variable" is empty.

So, somehow the OAuth2 code can affect the "password" variable value, regardless different mail accounts. It was long time ago and I cannot recall which file I modified to monitor the password variable. I did not go further to trace the codes for further investigation.

Seems that I have regained some memory. I am looking at the source code of 102.5 (not 115 which I haven't downloaded the source).

The file for checking and prompting password is:

comm/mailnews/imap/src/nsImapProtocol.cpp

Under the function "nsImapProtocol::GetPassword()", Thunderbird asks for password whenever password.IsEmpty() is true. When it is empty, Thunderbird attempts to grab the password from the Password Manager (this is why one saves his password under the manager). It does
not prompt for a password when it successfully grabs the password from the manager. Otherwise it prompts for password input.

I understand that when one first logs on via password (if he does not save passwords under the password manager), Thunderbird has to ask the password. But once Thudnerbird is up (before quitting), "password" object should keep the password so that password.IsEmpty() is false. But our case shows that we have 2 accounts, one with non-OAuth IMAP account, another one is Microsoft 365 mail account, after send some emails, Thunderbird asks for password all a sudden, while "password.IsEmpty()" returns true indeed. But shouldn't an OAuth2 account affect the password object under the non-OAuth2 account?

Well, I tested and debugged some more and it DOES appear that the oauth2 account is removing the password on the normal password account after I check for new messages on an oauth2 (tested with yahoo and aol).

A change was done here: bug 1695703 which landed somewhere in v91. I also did some changes in that area for autosync, in 102, but I don't think that is directly causing the problem since this problems was reported much earlier, back at 78.

Looking with debugger and supplemental printf it appears the LoginInfo is always null here https://searchfox.org/comm-central/rev/94cd6266e9ea128d3da8c91ff09f6dbd134f88df/mailnews/base/src/nsMsgIncomingServer.cpp#90 causing the stored password for accounts to get removed when it shouldn't.

The code to check for LoginInfo was added at bug 1695703. But this bug was reported even before that landed in 91.

I've got this problem regulary, using two imap accounts to different ISP's, and never touched oauth2. Just saying.

Out of curiosity (and because I am a daily victim of this bug):

Obviously the nsMsgIncomingServer::Observe call is made by some
automatic process and not by a user editing some password field.
But what value does "aTopic" have when these calls are made?

And,
-since the bug happens when thunderbird is connected to MORE THAN ONE imap server,
with server A using oauth2 and server B not using oauth2
-and assuming each different server (server A and server B) has its own nsMsgIncomingServer data structure:

why does automated processing needed for server A also call nsMsgIncomingServer::Observe
on the data structure of server B?

Assuming "aSubject" is the identifier of a specific server, what is its value during the call?

It would be useful to log all calls to nsMsgIncomingServer::Observe aloing
with the values of "this" (identifies the data structure), "aSubject" and "aTopic" and "loginInfo" (null or not).

Perhaps this sheds some light on the higher level code that makes the calls. For example
is loginInfo==nullptr a symptom or a cause?

Maybe some extra test is needed on aSubject to distinghuish between
-this call is not for this server
and
-this call is for this server, but no login info is available.

The bug also happens when Seamonkey is connected to two imap servers of which none is using oauth2.

(In reply to Gerard H. Pille from comment #146)

The bug also happens when Seamonkey is connected to two imap servers of which none is using oauth2.

I don't see this with Thunderbird. See my comment #20.

I actually use thunderbird with 3 imap servers:

  1. password transmitted insecurely
  2. starttls and normal password
  3. ssl/tls and oauth2 (outlook server)

This creates the problem on servers 1 and 2. So my work around is to set server 2 to "normal password"
This makes email reception impossible for server 3, but and 1 and 2 work fine together.

So when I need server 3, I change it to oauth2. thus loosing server
1 and 2. When I am dome, I change the authentication/method for server 3 again.

In case you ware wondering (and off topic):
-Server 3 is microsoft outlook. I prefer to avoid it completely. and have set a forward of all email, but forwrding on
outlook is unreliable. Most mail is forwarded correctly, but some really important emails are not forwarded.

  • server 1 is connected to over a vpn, so the connection is secure.

So for me, the only problem causing factor is the oauth2 setting.

My case is:

  1. server1: non-OAuth2 - IMAP: 993/tcp, SMTP: 587/STARTTLS
  2. Microsoft 365: OAuth2 - IMAP:993/tcp/OAuth2 SMTP: 587/STARTTLS/OAuth2

With this combination, Thunderbird asks for password of (1) after some operations with both (1) and (2).

Yesterday, I tried to get rid of (2) (not really getting rid, but making use of a third-party addon - owl).
At the same time, I have added a gmail account with OAuth2:

I note that by default, Thunderbird adds the account with SMTP as 465/TLS/OAuth2. That is:

  1. GMail: OAuth2 - IMAP:993/tcp/OAuth2 SMTP: 465/TLS/OAuth2

I have been running this setup for almost a day (21 hours technically) and I do not see Thunderbird
asking for password of (1).

So, is it something special about GMail? I do not think so. But I doubt if it is because GMail setup
makes use of 465/TLS for SMTP, while both non-OAuth2 and Microsoft 365 are using 587/tcp/STARTTLS.
Probably I will have this setup running for few more days. If it works, I will consider switching
GMail setup with SMTP as 587/tcp/STARTTLS to see if the problem comes again.

(note: the non-OAuth2 server "server1" only supports SMTP as 587/tcp/STARTTLS. Also, Microsoft 365
does not seem to support 465/tcp/TLS).

Thanks for the comments. I know why the problem occurs but I haven't found a good fix.
The problem occurs when the login is modified. For unknown reasons (to me), oauth2 signals a change in the login a lot. My guess, not being an oauth2 expert, is that a "token" changes quite often (maybe more often for yahoo). This change is signaled to all the accounts via the Observe() call, and each account is supposed to ignore the info when it is not for that account.

The problem originates here https://searchfox.org/mozilla-central/rev/1cfd3a3bc76a8d09cb4266576f30802753a80c83/toolkit/components/passwordmgr/storage-json.sys.mjs#430 where an "array" of values is passed to the notifiyStorageChanged(), the old and new login info. I can fix the problem by just changing this so that only "newLogin" is passed and not an array. However, this is in mozilla code which TB project normally doesn't change. The problem is also in the TB specific code here, https://searchfox.org/comm-central/rev/f33739060699f381bb6e7dd74e51ee94d9829a6c/mailnews/base/src/nsMsgIncomingServer.cpp#90. The "aSubject" contains the login info and it fails to decode it because it's an array and loginInfo ends up null. This causes the server/account to not ignore the info and it goes ahead and removes its own password incorrectly.

I tried various tricks with query interface and casting to obtain a not-null value for loginInfo but haven't been able to get it to work yet. So what needs to happen is the Observe() call in nsMsgIncomingServer.cpp needs to be able to handle aSubject when the contents is actually an array and it can just use the first array element as the loginInfo since only the hostname is relevant and that should not change.

Note: the Observe() in nsMsgIncomingServer.cpp works fine when you just delete a login line in the password mgr since aSubject is just a single item and not an array.

And whatabout the user names for these 3 accounts? Are they different?

In my caseall 3 accounts have different user names.

3 accounts with different email addresses, on different sites. They are supposed to be independent accounts.

GIven Gene's comment, the question/answer about the accounts ir probably irrelevant now.

Probably. However, with GMail OAuth (instead of Microsoft 365 OAuth), Thunderbird still has not asked for password of my non-OAuth2 email account. But yes, probably GMail does fewer token refresh than Microsoft 365...

Attached patch password-remove-fix.diff (obsolete) — — Splinter Review

This diff shows the changes I've made to fix the bug. With this change a modification of the oauth2 token or normal password for account A shouldn't cause other accounts to have their cached password removed and require re-entry. This now handles the case where an "array" of loginInfo is the "aSubject" parameter of Observe(), as occurs when a login entry is modified. It also still handles the case where the login entry is removed by the user (aSubject not an array in that case).
In addition , it now checks to be sure the oauth2 token has really changed before triggering the Observe() on all accounts, so this should reduce a lot of useless Observer() calls, especially for yahoo.
I need to clean this up and submit a formal patch (it still has several printf() for debugging).
I've only tested it with imap but it should also fix the same problem for pop3.
This avoids changing any mozilla code.

I haven't look at it yet but there may also be a similar issue for SMTP which will cause the removal of passwords. SMTP's Observer() might reject an array for "Subject" too:
https://searchfox.org/comm-central/rev/b9ffc6199f445f6a52c32d8b157f40b3e76d89e8/mailnews/compose/src/SmtpServer.jsm#36
I think several comments above mentioned that sometimes the sending (SMTP) password has to be re-entered.

Assignee: nobody → gds

Gene,
I have applied your patcth (with some changes, did not apply cleanly) to
thunderbird-102.13.0
I had checked out this version to start debugging and I already discovered that nsMsgIncomingServer::Observe
was never called, except at startup, when no Oauth2 account was present.

With your patch, the first test is going well. The problem seems solved.
It does appear that now nsMsgIncomingServer::Observe is now called from time to time
but without adverse effect

I have attached some output from the printfs you have added.

This is with 4 accounts
1 with regular password
2. with regular encrypted password
3 with normal password but account is not working
4 . account with OAuth2
1, 3 and 4 seem to be working, but the test was brief
2 is not expected to work as the provider closed it.

You made my day. Fianlly the most annoying bug in thunderbird has been fixed.

Thanks for testing the patch. The fact that you see the lines with fullName and hostName appearing mean it's working now. I think maybe outlook/micorsoft oauth changes the token/password even more often than yahoo.
:
:
11 days later. Some time has passed because I got diverted to fixing another bug. I'll try to get back on this now

Attached patch password-remove-fix-v2.diff (obsolete) — — Splinter Review

Here's the complete fix but still with printf and dump statements. I'll next remove the printf/dump's and submit the formal patch via moz-phab. But first I want to describe (again?) exactly how to duplicate the problem with release TB (<= 115.0).

It requires a imap/smtp account configured with not-oauth2 security and a Yahoo, AOL or Outlook account configured with oauth2 security. (I haven't seen the problem with gmail oauth2.)

You can't see the problem with a non-oauth2 pop3/smtp account due to bug that I'll describe later.

Assuming you are starting with the default setup that allows TB to store the passwords for you, go to the Password Manager and remove both login lines for the imap/smtp account. Don't touch the lines for the oauth2 account(s). This will then require that you enter the password manually when receiving and when sending messages once per session (ideally) for only the non-oauth2 account.

On the non-oauth2 account enter the imap password when prompted or after clicking "Get Messages", to store the password in ram. Then send a test message with the non-oauth2 account. Be sure after entering the password not to check the box to have TB remember the password.

Now go access a folder in the oauth2 account or do "Get Messages" for the oauth2 account. This should work OK.

Go back to the non-oauth2 account and click "Get Messages". You will be prompted for the password again since it has been removed from ram incorrectly due to the oauth2 access. You can also try to send a message from the non-oauth2 account and it will also prompt for the password for the same reason -- it's also incorrectly gone from ram.

Note: you may spontaneously see a password prompt to receive incoming imap messages depending on if you have timed checked for new mail enabled and, if enabled, the time between checks causes a poll for new messages before you click "Get Messages".

If you try to duplicate the problem with a pop3/smtp account instead of a imap/smtp account you won't be required to re-enter the password for incoming messages,. This is because Observe() in the file nsMsgIncomingServer.cpp now only occurs for imap but previously (before pop3 was ported to JS) it handled imap and pop3. This prevents pop3 password from ever being invalidated (removed from ram) even when it should be. From the point of view of this bug, this seems good, but it's really bad. The fix for this is to add a new Observe() inside MsgIncomingServer.jsm which now handles pop3 password removal from ram when required.

Attachment #9343957 - Attachment is obsolete: true

This is fantastic, Gene, thank you! I look forward to seeing the fix in production. One clarification:

(In reply to gene smith from comment #158)

You can't see the problem with a non-oauth2 pop3/smtp account due to bug that I'll describe later.

I have a non-oauth2 pop3/smtp account which is one of the two with unsaved passwords displaying the re-prompt problem (the other is imap/smtp). See Comment #17 (my own initial report), along with Comment #39 and others in this thread.

I'm not sure how the behavior is being triggered since the code is different, and I don't know anything about the internals of the TB code, but I assure you this does happen with pop3.

Really appreciate your work on this!

I have a non-oauth2 pop3/smtp account which is one of the two with unsaved passwords displaying the re-prompt problem (the other is imap/smtp). See Comment #17 (my own initial report), along with Comment #39 and others in this thread.

It may depend on the TB version you are running. When comment 17 and comment 39 were posted POP3 was probably handled just like IMAP so the problem would occur the same for both protocols. I'm not sure when but maybe some time in 91 or maybe at 102, POP3 was changed to javascript code and I don't see the bug (as described) occurring now for POP3 running daily 116. I noticed POP3 was OK 12 days ago when I was first working on this. Then I noticed it OK again today and it took me a while to remember why POP3 didn't require a password re-entry..

(In reply to gene smith from comment #158)

...
(I haven't seen the problem with gmail oauth2.)
...

I have seen this problem for some years a few years ago. At that time I had only two oauth2 accounts, one Gmail account and one Google account of our university.
Two years ago (see comment 119) the problem disappeared when I upgraded to Thunderbird 91. It did not pop up for about 2 years. A few months ago I added an Outlook oauth2 account and the problem reappeared. Of course in the time frame that Thunderbird 91 was installed, Google may also have changed its oauth2 procedures, so it is not sure whether the problem disappeared due to a change in Thunderbird, or a change in Google's code.

The bug only affects users who manually enter account passwords once per session instead
of letting the password manager store them. This is mostly seen when an oauth2
token (always stored in pwd mgr) changes and would cause normal passwords to be deleted
from ram, requiring the user to re-enter the password again during the session.
This also now avoids notification to all accounts when an oauth2 token seems to change
but really doesn't.
And it also fixes a possible security bug in that a change by the user in pwd mgr of a
stored pwd or username for a POP3 account didn't deleted the password stored in ram like
it should. This appears to be a regression caused by porting of POP3 from C++ to JS.

Comment on attachment 9345706 [details] [diff] [review]
password-remove-fix-v2.diff

This has a significant error that is now fixed in the patch (.diff) found inside the link at comment 162. So marking obsolete.
The direct link to the corrected patch is: https://d2mfgivbiy2fiw.cloudfront.net/file/data/z274ktbh3a2slcbzqe2m/PHID-FILE-koxgj42z7dxoessx3pda/D184660.diff

Attachment #9345706 - Attachment is obsolete: true

I suspect this creates a fair number of support posts.

Severity: -- → S3
Flags: needinfo?(bugzilla2007)
Priority: -- → P1

Yeah. Sorry for the delay, I've been on PTO.

Stoopid question: How will I know when a release will include this fix so I can go back including my yahoo account on T-Bird?

(In reply to stupheyewant from comment #166)

Stoopid question: How will I know when a release will include this fix so I can go back including my yahoo account on T-Bird?

It will probably be mentioned in the release notes: https://www.thunderbird.net/en-US/thunderbird/releases/. Otherwise I don't know of any notification scheme.
Edit:

For thunderbird 115, when the status here in the bug is set to thunderbird_esr115 fixed.

Yes, I just came back here to add to my comment but Magnus beat me to it.

Keywords: checkin-needed-tb

For thunderbird 115, when the status here in the bug is set to thunderbird_esr115 fixed.

Status: NEW → ASSIGNED
status-thunderbird117: --- → affected
status-thunderbird_esr115: --- → affected
tracking-thunderbird_esr115: --- → +
Target Milestone: --- → 118 Branch

Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/0a14f94e47c8
Don't delete ram password when stored login changes on a different account. r=mkmelin

Status: ASSIGNED → RESOLVED
Closed: 2 months ago
Keywords: checkin-needed-tb
Resolution: --- → FIXED
Pushed by mkmelin@iki.fi:
https://hg.mozilla.org/comm-central/rev/5c991219ecf6
follow-up to fix linting. rs=eslint DONTBUILD

Comment on attachment 9345929 [details]
Bug 1673446 - Don't delete ram password when stored login changes on a different account. r=mkmelin

Patch has gone through a full beta and more, with no regressions reported...

[Triage Comment]
Approved for esr115

Attachment #9345929 - Flags: approval-comm-esr115+
You need to log in before you can comment on or make changes to this bug.