Closed Bug 1595169 Opened 5 years ago Closed 5 years ago

IMAP fails login to certain Yahoo! Mail folders till Thunderbird is restarted

Categories

(Thunderbird :: Untriaged, defect)

Product:
Thunderbird
Component:
Untriaged
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

(Not tracked)

Status:
RESOLVED INVALID

People

(Reporter: stevepobox-bugzilla, Unassigned)

References

Details

Attachments

(3 files)

error.png
10.97 KB, image/png
Details
imap_snippet.log
16.41 KB, text/plain
Details
failtoconnecttoyahoo.png
7.22 KB, image/png
Details
Attached image error.png —

User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:70.0) Gecko/20100101 Firefox/70.0

Steps to reproduce:

Using Yahoo! Mail with IMAP; security is "app password".

  1. Use Thunderbird for a while (hours? days?) without restarting it
  2. Open certain folders, most usually Trash or Bulk Mail or both

Actual results:

A login failure message (see attached).

Other folders continue to work fine.

Restarting Thunderbird results in immediate refresh of problem folder contents with no error.

Expected results:

Folder content refreshes with no error and no need to restart Thunderbird.

(Specific Thunderbird version is 60.9.1, 32-bit)

Reporter, when you see the error message, do you click Retry, Enter New Password or Cancel?
I assume you probably did Retry several times with no success.
Does this only occur for Trash and Bulk? If you cancel and try to open another folder that you haven't opened recently, does it work then?
Have you changed the max number of "cached" connection in Server Settings/Advanced ? (It defaults to 5.)
You probably need to record an IMAP:5,timestamp log to see what is happening, see https://wiki.mozilla.org/MailNews:Logging. This will record a lot of data if it takes several days, so you may need to edit out irrelevant data before attaching using the link above.
Thanks.

do you click Retry, Enter New Password or Cancel?

I click Retry, which results in the same message immediately again. Then Retry again, which results in a disconnection popup message in the corner. Retrying the folder results in the same cycle all over again, so instead at that point I just restart Thunderbird so I can get into the folder.

Does this only occur for Trash and Bulk?

I'm not sure, it may have done this to other folders too, but not often.

If you cancel and try to open another folder that you haven't opened recently, does it work then?

Luckily, it just did it again, so I was able to try. No, that also repeats the login failure for the other, non-recent folders.

Have you changed the max number of "cached" connection in Server Settings/Advanced ? (It defaults to 5.)

I was not aware of this setting; it is indeed set to 5.

You probably need to record an IMAP:5,timestamp log

I was able to generate this log; but it looks like I may need to redact it before posting. Is there an easy way I can do this (given competence in, say, Vim)? Or perhaps some guidance as to the the exact snippet I need to post instead of the whole thing?

I don't know what you want to redact but you can use these vim commands (where vim commands start with :):

To make search case insensitive :set ic
To search and replace :.,$s/searchForThis/ReplaceWithThis/gc

Probably need to see where you selected the folder where it failed. You can look for authentication attempts that fail (search for authenticate or login). There will be long lists of tb fetching flags that you can leave out.

I just looked at my yahoo account and selected Bulk Mail as an example. You should see a line that ends in "Bulk%20Mail: = currentUrl", then 2 lines down an "OK [CAPBILITY IMAP4rev1...." response line then then a "try to login" line. Show me what you see on yours when you access the folder that won't allow login and attach the log fragment starting from there until you see something about "OK Authenticate" or probably "BAD Authenticate" or maybe "NO Authenticate" or whatever looks/sounds like a failure.

I just saw a failure to login when I selected INBOX. I tried a few other folders and they failed too. Then it started working again. I never saw the prompt for retry or password, just an error pop-up notification of login failure. However, I let tb store my password so I never have to enter it. I think maybe you are running where the password is entered once at startup for the yahoo account. If the server rejects the login, tb may be refusing to send the password again and wants you to re-enter it, but not sure. Have you tried re-entering the password at the prompt instead of just retry?

I finally noticed above that you are using the app specific password that yahoo generates. I was using the oauth2 that seems to now be supported. I changed my yahoo on tb account back to app specific which just uses TLS and the 16 character password that yahoo generates. I also told tb not to save the password, which seems to be what you are doing too. After selecting folders the prompt appears and retry seems to continuously fail.

You didn't mentioned this but after a couple retries i see a pop-up notification that the connection has been lost. Looking with wireshark, I see that yahoo does drop (RST) the connections.

I don't see that tb is doing anything wrong however. Yahoo seems to quickly drop connections when there is no activity (after about 5m) where most servers will go about 30m before disconnecting. Then when you click on a folder and tb tries to restore the connection, yahoo rejects the login and after a few retires, yahoo drops the connection. When this occurs there are only about 3 active connection.

One way to fix this is to wait a few minutes. This causes yahoo to logout (imap wise) several connection and free resources so new connections and imap logins can occur again. Looking with wireshark, during this few minutes, several connections are closed by yahoo so new ones can occur again. Other non-yahoo imap servers don't seem to have this issue with timing out, dropping connections and allowing new connections and rejecting imap logins.

Another workaround that seems to work for me is to reduce the number of cached connections down to 3 (Server Setting/Advanced). I saw the problems with 4 and 5 but, so far, not with 3. Let me know if this helps for you.

Sorry, one more thing. Since I can now duplicate what you see, probably no need for you to produce and attach an imap log. But if I am missing something and you don't think I have reproduced what you see, go ahead and attach the log.

I also told tb not to save the password, which seems to be what you are doing too.

No, I'm storing the password in TB. When I restart it, it submits the stored password on its own and works fine.

I went ahead and made the snippet you mentioned from my log, see attached.

I'll try reducing my cached connections setting and report back what happens.

Attached file imap_snippet.log —

Yeah, I'm probably wrong about what happens on login failures when you let tb store the password. It probably does ask the user to enter a corrected password. I will try this again later to make sure.
Anyhow, the log you attached looks exactly like what I see here: it tries to use plain authentication by base64 encoding your user id and password. Then when it fails it tries using "old style" login where userid and password are sent unencoded, then it prompts for a user entry. (Of course, this is all sent in an encrypted TLS tunnel so not really plaintext.)
There is no reason I can see why either authentication should fail. Other IMAP servers accept this fine at any time. But, like I suggested, you can do any of three things to maybe workaround this:

  1. Wait a while, at least 5 minutes, so that yahoo times-out all the open connections and allows new logins again. Then click the retry button.
  2. Restart tb. This will cause tb to close any remaining half open connections so on restart yahoo allows new logins. Faster than waiting 5m.
  3. Reduce cached connection to 3. If 3 or less connection to yahoo occur, yahoo remains happy it seems. Not sure why. Tb should work fine with 3 instead of 5 connections.

It's been a while now, and I haven't seen it happen since. Looks like the reduced cache setting has worked around it.

Should this cache setting be made the TB default for Yahoo!-based accounts?

Not sure where the default of 5 connections comes from. Seems to work ok for most servers. Making it 3 for yahoo server might help but some servers like aol, sky.net and maybe verizon all use yahoo as their backend server, from what I have heard. So it might be hard to know exactly when a yahoo server is the "real" server when tb sets up an account.
Also, whatever yahoo is doing that requires tb to only have 3 connection might be a temporary yahoo bug that will be fixed in the future.

Anyhow, thanks for finding this issue which is now somewhat "documented" in this bug report. I am setting this report to INVALID since it can be fixed by changing the default user setting on cached connections down to 3.

Status: UNCONFIRMED → RESOLVED
Closed: 5 years ago
Resolution: --- → INVALID

Well, if it is temporary, it's not a brief temporary — this has been going on for probably all year now, if not longer.

I have a folder, "Bugzilla", where all these bug reports emails wind up. I go to search it, by pasting a bug number into the quick search box, and nothing happens. I go to another folder and come back, and sometimes it will work. Other times, I will have to close Thunderbird completely and paste it in for it to work. Hope this helps. Let me know if there is something I can try in order to help.

(In reply to Worcester12345 from comment #16)

I have a folder, "Bugzilla", where all these bug reports emails wind up. I go to search it, by pasting a bug number into the quick search box, and nothing happens. I go to another folder and come back, and sometimes it will work. Other times, I will have to close Thunderbird completely and paste it in for it to work. Hope this helps. Let me know if there is something I can try in order to help.

I'm not exactly seeing how your comment applies to this bug (currently resolved as invalid). Is your issue Yahoo specific? If so, have you tried the workaround of reducing the number of cached connections from 5 down to 3 in advanced server settings?

(In reply to gene smith from comment #17)

(In reply to Worcester12345 from comment #16)

I have a folder, "Bugzilla", where all these bug reports emails wind up. I go to search it, by pasting a bug number into the quick search box, and nothing happens. I go to another folder and come back, and sometimes it will work. Other times, I will have to close Thunderbird completely and paste it in for it to work. Hope this helps. Let me know if there is something I can try in order to help.

I'm not exactly seeing how your comment applies to this bug (currently resolved as invalid). Is your issue Yahoo specific? If so, have you tried the workaround of reducing the number of cached connections from 5 down to 3 in advanced server settings?

Yes, that inbox is in Yahoo. No, I have not tried this workaround. I must have missed that. That is still no reason to make this bug invalid.

Please reopen.

Flags: needinfo?(gds)

Seems to be a problem for a few users caused by a certain subset of yahoo imap server nodes, not directly by a bug in tb. Since the number of cached connection is adjustable on the advanced server page we deem this bug invalid since it can be fixed by setting the value to 3. But I'll leave the decision up to Wayne on whether to reopen this.

Does the adjustment to 3 work for you?

Flags: needinfo?(gds)

I think we also have to be aware in this connected world that IMAP on idevices has a habit of continuing to connect until it exceeds the number of available connections. Or at least did a couple of years ago. So one iPhone can be enough to lock Thunderbird out of a yahoo server.

I do not use the products with an apple on them, therefore anything I think I know is anecdotal, but I understand there is no option to control connection number on those devices either.

I did not see any settings to change the number of "cached connections".

(In reply to Worcester12345 from comment #22)

I did not see any settings to change the number of "cached connections".

Look under "Account Settings" then "Server Setting" then "Advanced..." and you will see "Maximum number of server connections to cache" which defaults to 5. Reduce it to 3, click OK and restart TB so the up to 5 existing connections are dropped and a max of 3 can then occur after the restart.

See Also: → 1727971
See Also: → 1745205
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: