Other e-mail appear at the end of e-mail messages

RESOLVED WORKSFORME

Status

RESOLVED WORKSFORME
6 years ago
5 years ago

People

(Reporter: cwiiis, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

6 years ago
When I read a message in the e-mail client, at the end of the message, contents from the e-mail above it appear.

This doesn't happen for all mails - I've observed it in mails from Kickstarter only at the moment, and only on my GoDaddy e-mail account (seems fine in GMail).

Possibly related, after downloading one message on my GoDaddy account, the client seems to get 'stuck' (trying to open a new mail results in a spinner and no activity) until I close it and reopen it.

If it helps, Thunderbird has no issues with this account.
godaddy appears to be running an old version of courier.

Most likely explanation is Courier doing something different (possibly just buggy) from other IMAP servers when we issue our batched requests for parts of message bodies.  We could have a bug in our IMAP parsing code that screws up, or again, it could be Courier's fault.  We've seen wacky behaviour from Courier with its folder list, so neither would be a huge surprise.


More details on the server version:

* OK [CAPABILITY IMAP4rev1 UNSELECT STARTTLS ID CHILDREN NAMESPACE IDLE] Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc.  See COPYING for distribution information.

a1 ID NIL
* ID ("name" "Courier-IMAP" "version" "3.1.0.11")

http://www.courier-mta.org/imap/changelog.html doesn't list a version with that number, but 2004 does seem right.  The current version is 4.13.0

Which is mainly to say that I cast aspersions on the IMAP server in use.  We'd probably want a protocol trace to see what's going on, which means turning on the 'dangerously entrains user data' mode at https://wiki.mozilla.org/Gaia/Email/SecretDebugMode

I don't really recommend doing that for your own account unless you are the only person who is looking at the log.  If you are able to create other accounts for free, if you could create an account and provide credentials, we could do this.  (Or if you want to do that and collect the log yourself, that also works.  Just definitely don't want to see any of your private detail, especially passwords, which will end up in the log at that level of verbosity.)
Status: NEW → UNCONFIRMED
Ever confirmed: false
(Reporter)

Comment 2

6 years ago
(In reply to Andrew Sutherland (:asuth) from comment #1)
> godaddy appears to be running an old version of courier.
> 
> Most likely explanation is Courier doing something different (possibly just
> buggy) from other IMAP servers when we issue our batched requests for parts
> of message bodies.  We could have a bug in our IMAP parsing code that screws
> up, or again, it could be Courier's fault.  We've seen wacky behaviour from
> Courier with its folder list, so neither would be a huge surprise.
> 
> 
> More details on the server version:
> 
> * OK [CAPABILITY IMAP4rev1 UNSELECT STARTTLS ID CHILDREN NAMESPACE IDLE]
> Courier-IMAP ready. Copyright 1998-2004 Double Precision, Inc.  See COPYING
> for distribution information.
> 
> a1 ID NIL
> * ID ("name" "Courier-IMAP" "version" "3.1.0.11")
> 
> http://www.courier-mta.org/imap/changelog.html doesn't list a version with
> that number, but 2004 does seem right.  The current version is 4.13.0
> 
> Which is mainly to say that I cast aspersions on the IMAP server in use. 
> We'd probably want a protocol trace to see what's going on, which means
> turning on the 'dangerously entrains user data' mode at
> https://wiki.mozilla.org/Gaia/Email/SecretDebugMode
> 
> I don't really recommend doing that for your own account unless you are the
> only person who is looking at the log.  If you are able to create other
> accounts for free, if you could create an account and provide credentials,
> we could do this.  (Or if you want to do that and collect the log yourself,
> that also works.  Just definitely don't want to see any of your private
> detail, especially passwords, which will end up in the log at that level of
> verbosity.)

I've created an account for you:

username/email: mozilla-testing@chrislord.net
password: *rules (replace * with the company that makes Firefox in all lower-case)
imap server: imap.europe.secureserver.net (use ssl, default ports)
smtp server: smtpout.europe.secureserver.net (again, ssl + default ports)

Let me know if that's useful. It might not be useful information, but the Android e-mail client, K-9 (another popular Android e-mail client) and Thunderbird all have no such problems with this account (perhaps Thunderbird is useful to see what they do?)

I can forward you some of the mails that have caused me issues if that helps.
Flags: needinfo?(bugmail)
Thanks for creating the account.  I've just gotten around to looking at this.

It appears that very bad things happen to us when we come out of IDLE with the server and the server mistakes our command tag for something else and then decides to treat our command as the tag.  My protocol log traces are showing me something that amounts to the following:

==
C: IDLE IDLE
S: + idling
C: DONE
C: A9 LIST "" "*"
S: IDLE OK IDLE completed
S: LIST NO Error in IMAP command received by server.
==

==
C: IDLE IDLE
S: + idling
C: DONE
C: A7 UID SEARCH NOT DELETED SINCE 1-Jan-1990
S: IDLE OK IDLE completed
S: * SEARCH 1 2 3
* UID OK SEARCH done.
==

The weird thing imap.js is doing is using 'IDLE' as its tag.  Maybe this does something weird to the server's IMAP parser and it thinks our tag is another DONE command or something?  The other possibility is that Courier just strongly dislikes us sending our command before we see it announce that the IDLE is completed.

I'll try changing our tag to use something that's not actually IDLE or maybe even a proper tag.
Flags: needinfo?(bugmail)
Okay, I changed to using "IBLE" as the tag.  That didn't seem to make things any happier, suggesting that it's potentially some type of confusing timing thing.  More investigation is likely merited.  Specifically, I'd be interested if we can insert a bogus NOP command to cause the command-stream to re-synchronize and do it in such a way that we don't have to wait for a network round-trip before sending our actual command.

We can also potentially mitigate by having imap.js just being less aggressive about turning on IDLE.  The server isn't going to close its connection to us super-immediately.


Chris, is using the e-mail app pretty horrible for you when using the account on the server in question?  It seems like connections should be breaking and timing out left and right and message moves between folders would never make it to the server, etc.


Note: I filed bug 897833 related to some very bad UI fallout observed from using the troublesome account.  I think there needs to need to be another bug about us complaining much more loudly and obviously when our IMAP connection encounters an error.  And then we can sever the connection.  Because right now we're just sort of having our connection go into a horrible limbo on errors that we frequently detect as a side-effect of our misunderstanding.
(Reporter)

Comment 5

6 years ago
(In reply to Andrew Sutherland (:asuth) from comment #4)
> Chris, is using the e-mail app pretty horrible for you when using the
> account on the server in question?  It seems like connections should be
> breaking and timing out left and right and message moves between folders
> would never make it to the server, etc.

It is, yes - I haven't really tried to do much more than read messages on it as I usually have limited success just doing that. If I want to read more than one message, I often have to kill the app and restart it.

Would be great if imap.js can be massaged to work with this server (I understand godaddy is a pretty popular host?), but if it's behaviour is particularly egregious, maybe there could be a slower compatibility-mode for blacklisted servers?

Thanks for taking a look at this, it's much appreciated :)
Wonky servers come with the territory; we should definitely try and work with Courier.  I just reserve the right to be shake my fist at Courier and mutter things under my breath occasionally. :)
Going through old bugs here.  My apologies for losing track of this bug previously.  After my regrettably abandoned investigation, we landed 2 potentially relevant imap.js fixes:
- we no longer ever trigger IDLE on any servers
- some parse bugs related to literals were addressed

I think we may have also addressed some frontend-related display bugs that might have corrected the problem when we landed POP3 for v1.3.

Chris, any chance you've risked using the e-mail client again recently or would be willing to and see if you still encounter the same problems?
Status: UNCONFIRMED → NEW
Ever confirmed: true
Flags: needinfo?(chrislord.net)
(Reporter)

Comment 8

5 years ago
This would appear to be fixed, good stuff :) I have issues with Google mail now, but I assume those are being handled/already fixed.
Status: NEW → RESOLVED
Last Resolved: 5 years ago
Flags: needinfo?(chrislord.net)
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.