Thunderbird 38.6.0 stops receiving IMAP emails from charter.net on my Windows 7 PC

RESOLVED DUPLICATE of bug 1231592

Status

Thunderbird
Mail Window Front End
RESOLVED DUPLICATE of bug 1231592
a year ago
5 months ago

People

(Reporter: razorsharpcomm, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

a year ago
I have a Windows 7 PC, receiving emails via Charter.net (IMAP) with Thunderbird 38.6.0.  I can receive one or two emails, and then the software stops receiving emails.  I have to close Thunderbird and restart to receive the newer emails.

What is going on?

Robert A. Saigh
razorsharpcomm@charter.net

Comment 1

a year ago
(In reply to razorsharpcomm from comment #0)
> What is going on?

Frankly, we don't know. But we know this:
Bugzilla is NOT a support forum. It is a tracking system for bugs in Mozilla products.
You have a support issue.
Status: UNCONFIRMED → RESOLVED
Last Resolved: a year ago
Resolution: --- → INVALID
Note
- Charter has numerous issues.
- You can find support and ask questions at https://support.mozilla.org/en-US/products/thunderbird
Summary: Thunderbird 38.6.0 stops receiving IMAP emails on my Windows 7 PC → Thunderbird 38.6.0 stops receiving IMAP emails from charter.net on my Windows 7 PC

Comment 3

11 months ago
Jorg, Wayne  I have need looking at Charter, and their implementation of IMAP is buggy.

The following log extract shows the email server dropping the quotes on the junk email folder and the subsequesnt failure to function.

db48800:mobile.charter.net:A:SendData: 6 STATUS "Junk E-mail" (UIDNEXT MESSAGES
UNSEEN RECENT)
2016-08-27 19:47:51.762000 UTC - 10888[d157680]: ReadNextLine [stream=d039650 nb=66
needmore=0]
2016-08-27 19:47:51.762000 UTC - 10888[d157680]:
db48800:mobile.charter.net:A:CreateNewLineFromSocket: * STATUS Junk E-mail (MESSAGES
0 RECENT 0 UNSEEN 0 UIDNEXT 1000)
2016-08-27 19:47:51.762000 UTC - 10888[d157680]:
db48800:mobile.charter.net:A:PARSER:Internal Syntax Error on line: %s: * STATUS Junk
E-mail (MESSAGES 0 RECENT 0 UNSEEN 0 UIDNEXT 1000)
2016-08-27 19:47:51.762000 UTC - 10888[d157680]:
db48800:mobile.charter.net:A:SendData: 7 logout


The log also identifies the email software in  use as Email Mx.

db48800:mobile.charter.net:A:CreateNewLineFromSocket: * ID ("name" "Email Mx"
"version" "M.9.00.023.01 201-2473-194" "vendor" "Openwave Messaging" "support-url"
"http://support.openwave.com" "date" "Mon Apr 4 21:15:22 PDT 2016")

DO you know of anyone that might have contact with either open wave or charter that we could follow it up?
Flags: needinfo?(vseerror)
I do not know anyone there. But I just chatted with spectrum support, and got a generic email address to which I sent an email asking for their help.  Will see if they respond.
Flags: needinfo?(vseerror)
Duplicate of this bug: 1280816
Duplicate of this bug: 1274833
Duplicate of this bug: 1287735
Just a few examples of reports outside bugzilla:
https://support.mozilla.org/en-US/questions/1120329
https://support.mozilla.org/en-US/questions/1116345
http://www.dslreports.com/forum/r30877934-Charter-net-IMAP-Server-new-messages-fail-to-download
http://forums.mozillazine.org/viewtopic.php?f=39&t=3006467

Comment 9

7 months ago
Any response from open wave Wayne?
Flags: needinfo?(vseerror)
(In reply to Matt from comment #9)
> Any response from open wave Wayne?

I sent to unblock@charter.net on Sept 10  and never heard from them
Flags: needinfo?(vseerror)

Comment 11

6 months ago
I was having this problem with Charter IMAP mail servers for some time, but as of Thunderbird 45.7.1 the situation seems to have improved. When I click on Get Mail, the process status shows at the bottom if there are emails to get and the emails are downloaded. Thanks to those who contributed to this effort.

Comment 12

6 months ago
This explains what causes the problem: https://bugzilla.mozilla.org/show_bug.cgi?id=1231592#c16
I also checked with the imap news group and they pretty much agree that charter's IMAP server is at fault:
http://mailman13.u.washington.edu/pipermail/imap-protocol/2017-February/thread.html
I also observe some improvement with tb 45.7.x. With it, a SELECT occurs to INBOX that causes charter server to flag new emails in the UID FETCH. However this only occurs about every 5 minutes when inbox is polled (either by the timer or by clicking). This SELECT request was not observed with older tb versions I have been mostly using, 45.4.0 and 38.6.0. Also, for all versions, it is necessary to reduce the number of "cached" connections down to 1 (instead of the default 5) for new emails to be detected since if left at 5 a new SELECT never occurs (even with 45.7.x) and no new emails are detected by thunderbird. Setting the max connection to 1 is also necessary for messages to be successfully moved or copied between folders for the same reasons (charter's defective imap server expecting a redundant SELECT to report changes in folders).
See Also: → bug 1231592

Comment 13

6 months ago
The improvements I saw with TB 45.7.1 may only be related to TB on windows vista. After upgrading to 45.7.1 on my windows 10 pc, the situation is as before: Get Messages does not get messages; the only way to get new messages immediately is to restart TB.

Comment 14

6 months ago
The comment from Gene Smith that Charter has a defective IMAP server is confusing, since my Android email, AquaMail, has no problem requesting and receiving new emails on demand from Charter's IMAP server.

Comment 15

6 months ago
(In reply to Jim Cory from comment #14)
> The comment from Gene Smith that Charter has a defective IMAP server is
> confusing, since my Android email, AquaMail, has no problem requesting and
> receiving new emails on demand from Charter's IMAP server.

Yes, some email clients and apps work OK with charter because they do a redundant SELECT command on INBOX before they attempt to FETCH new email. I have observed at least one email client doing this (kmail) while others I have observed don't detect new email and do exactly like thunderbird (evolution and claws-mail). I don't have a way to look at android apps to see how they work but the ones I have seem to work OK with charter like you observed.

Actually, my observation above with tb 45.7.1 is incorrect. It does NOT do a redundant select every 5 minutes. What I observe is that sometimes a redundant (and needed by charter) select occurs on the first "get new messages" event after starting tb. After that, no redundant select occurs (usually). Sometimes this varies. Also, I observe that after a while (sometimes hours) tb logs out and then back into charter. This causes new emails to be detected and received because a new SELECT on INBOX occurs. This times varies a lot and I am not sure what causes this logout/login. Also, I couldn't find anything in the tb CHANGELOG or by looking at the Mercurial commits that indicated any relevant changes in the imap code at or before 45.7.1, so the newer version doesn't matter.

The only reliable workaround is to set max cached connection to one (and restart tb for it to take effect!) and move the selected folder from INBOX to another folder (such as Drafts) and then back to INBOX. Since there is only one connection, this forces a re-SELECT on INBOX so new mails are received. (Waiting for the timed check for new messages, even if set very short like 1 min., or clicking "get new messages" typically does nothing since they don't cause the redundant SELECT to be sent to charter.)

I provided an experimental fix (patch) for the problem to the volunteer developer named "WADA" that I have been running for over a week and tb now "immediately" receives new charter emails with default settings and with no workaround (on clicking "get new message" or using the timer; charter doesn't support actual immediate new mail notification via IDLE). WADA seemed to express some interest in doing a more "correct" fix but I haven't heard anything more from him/her since (s)he posted this: https://bugzilla.mozilla.org/show_bug.cgi?id=1231592#c25

Comment 16

6 months ago
First of all, I apologize for my comment #1 and closing this bug hastily. Sadly, we get too many bugs "cannot receive" and it's almost always a support problem.

I looked at your patch in attachment 8839338 [details] [diff] [review] in bug 1231592. Basically you're introducing a preference "mail.server.useCharterWorkarounds" that tweaks the behaviour slightly.

That is a valid approach. I'd name the preference differently, and I'd need your help to find a better name, maybe "mail.imap.force_select". While looking for a good name, I can across:
https://dxr.mozilla.org/comm-central/rev/a263d7290af0172fbcb1f5ccf4f89e2732e9b714/mailnews/mailnews.js#104
mail.imap.expunge_after_delete

Can you please check whether setting this option fixes the problem from bug 1231592. Then your patch would reduce to the hunk forcing the delete. If setting this option fixes the other problem, can you please attach a new patch here and I'll review it for you.

Of course, you might not know the people involved in the project: Wada is a very experiences bug triager and he has a lot of knowledge of protocols. However, to get a patch approved, you need the attention of a developer, which you have now ;-)
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: INVALID → ---

Comment 17

6 months ago
would it not be appropriate to place the preference in the server(XX)type locations as the user may have many IMAP accounts and only one charter one. OR is the overhead so small that the redundant selects are not an issue on other accounts.

Comment 18

6 months ago
Very good point! But fiddling this option into making it a particular account option is far more difficult. We already have global IMAP preferences, like:
https://dxr.mozilla.org/comm-central/rev/a263d7290af0172fbcb1f5ccf4f89e2732e9b714/mailnews/mailnews.js#93
Yes, the overhead seems to be small.

Comment 19

5 months ago
(In reply to Jorg K (GMT+1) from comment #16)
> First of all, I apologize for my comment #1 and closing this bug hastily.
> Sadly, we get too many bugs "cannot receive" and it's almost always a
> support problem.
> 
> I looked at your patch in attachment 8839338 [details] [diff] [review] in
> bug 1231592. Basically you're introducing a preference
> "mail.server.useCharterWorkarounds" that tweaks the behaviour slightly.
> 
> That is a valid approach. I'd name the preference differently, and I'd need
> your help to find a better name, maybe "mail.imap.force_select". While

Yes, a more general name probably better since any provider using Intermail/Openwave imap server may have this problem.

> looking for a good name, I can across:
> https://dxr.mozilla.org/comm-central/rev/
> a263d7290af0172fbcb1f5ccf4f89e2732e9b714/mailnews/mailnews.js#104
> mail.imap.expunge_after_delete
> 
> Can you please check whether setting this option fixes the problem from bug
> 1231592. 

Just setting expunge_after_delete to TRUE does not fix the problem in bug 1231592 with otherwise default parameters (max cached connection at 5 being the only relevant parmaeter, IDLE is irrelevant since charter imap doens't have that capability). My workaround to fix the problem of bug 1231592 with official released tb is just to set max cached connection to 1 for charter account only. Comment https://bugzilla.mozilla.org/show_bug.cgi?id=1231592#c7 explains why just setting expunge_after_delete alone doesn't work. I also re-verified it today and see that an offline/online transition (or a tb restart) is needed to see the restored email back in INBOX when only expunge after delete is set. 

However, the workaround described above does not fix this bug (bug 1258429: delayed or failure to sometimes receive new charter imap email). The redundant SELECT in the patch fixes only this bug 1258429. However, with the number of cached connection at default of 5 or anything other than 1, the patch does not fix bug 1231592 (failure to move emails between folder) unless expunge_after_delete is set TRUE. Therefore, the patch forces expunge_after_delete parameter to TRUE when the proposed new parameter "useCharterServer" is set TRUE. Not sure if "forcing" a parameter based on another parameter is the right thing to do but it worked for me. I would have no big problem not doing that if the need to sometimes set expunge_after_delete was documented.

> Then your patch would reduce to the hunk forcing the delete. If
> setting this option fixes the other problem, can you please attach a new
> patch here and I'll review it for you.

Not sure what this means. Just to reiterate, the patch gds.diff fixes both bugs (failure to receive new email and problems moving emails between folders).

Comment 20

5 months ago
(In reply to Jorg K (GMT+1) from comment #18)
> Very good point! But fiddling this option into making it a particular
> account option is far more difficult. We already have global IMAP
> preferences, like:
> https://dxr.mozilla.org/comm-central/rev/
> a263d7290af0172fbcb1f5ccf4f89e2732e9b714/mailnews/mailnews.js#93
> Yes, the overhead seems to be small.

When I did the experimental patch I looked around in the code and couldn't figure out how to make it account specific. With the patch I have been running with multiple accounts: charter, yahoo and gmail and haven't seen a problem. However, I haven't done details tests with yahoo and gmail to make certain there is no bad effect with the redundant SELECT and the forced expunge_after_delete.

Comment 21

5 months ago
(In reply to gene smith from comment #20)
> However, I haven't done detailed tests with yahoo and gmail to make
> certain there is no bad effect with the redundant SELECT and the forced
> expunge_after_delete.

Just checked these two accounts with the gds.diff patch and they work as expected.

Comment 22

5 months ago
This will be fixed by bug 1231592. Closing this bug here to avoid two streams of discussion.
Status: REOPENED → RESOLVED
Last Resolved: a year ago5 months ago
Resolution: --- → DUPLICATE
Duplicate of bug: 1231592
You need to log in before you can comment on or make changes to this bug.