150010 - Replied to message won't show up in your inbox until you have more than one incoming msgs.

Status: VERIFIED FIXED

(MailNews Core :: Backend, defect)

Description

[esther]

Using build 20020605 on winxp (this has been around for a while, I thought I
found a bug before but I can't find one to comment in it).
If you are reading an incoming message at the bottom or top of the thread pane
and you reply to it cc'ing yourself, the replied to message won't download (I
don't see the biff icon) or display in your inbox until another message has
arrived.  
Doesn't matter if biff is on or off, only happens to IMAP mail accounts

1. Launch app and open messenger
2. Log into an IMAP account, sort by date asscending or descending, doesn't matter.
3. Send that IMAP account a message and click Get Msg to retrieve it. Select it.
 For this test that message needs to be at the top or bottom of the list
depending on your sort. 
4. Click Reply and Send.
5. Click Get Msg to retrieve it.

Result: It won't download the replied message until that IMAP account gets
another message.
Expected: It should download immediately.  

This does not happen when if you Forward instead of Reply

Comment 1

[esther]

Nominating, because this is very annoying if you are replying to a message where
you cc'd yourself so you can have a copy to file elsewhere.  You won't get that
copy of the reply until another message shows up.  If mail correspondence is
slow or non existant I usually send a blank one to myself so I can retrieve the
cc and file it.  Changing sort or folders doesn't push the cc message.  Deleting
the original message and undoing the delete doesn't push the cc message.

Comment 2

[Gayatri Rimola]

Does it show up in the sent folder at the time of sending?

Comment 3

[esther]

My Sent folder is on the same IMAP account and yes I do see a copy of the reply
in that Sent folder almost immediately.

Comment 4

[scottputterman]

I'm able to reproduce this too.  Reassigning to naving.

Comment 5

[scottputterman]

marking nsbeta1+ [adt2 rtm] but we'd like this to be one of the last adt2's
worked on.

Comment 6

[Navin Gupta]

Attachment: imap-log

Comment 7

[Navin Gupta]

Attachment: imap-log

The imap logs shows how repeatedly doing GetMsg() doesn't download that replied
msg
until I sent one more msg and then both msgs were downloaded.

Comment 8

[Navin Gupta]

Attachment: imap-log

The imap logs shows how repeatedly doing GetMsg() doesn't download that replied

msg until I sent one more msg and then both msgs were downloaded.

Comment 9

[Navin Gupta]

I discussed this bug with david and it seems like a server bug because we are
asking for UID x and the response UID is x-1.

John, Can you take a look at the last imap-log ? thx. 

Comment 10

[David :Bienvenu]

also, we're not getting the new message, so we don't think there are any new
messages.

Comment 11

[John G. Myers]

The last IMAP log shows the server behaving as required by the protocol spec.

The client sends the command:

1192[42c3ea0]: judge.mcom.com:S-INBOX:SendData: 21 UID fetch 11003:* (FLAGS)

Per RFC 2060's section 9 definition of sequence_num, "*" is the largest number
in use--in this case 11002.  So that returns the flags for messages between
11003 and 11002 inclusive.  Since the message with UID 11003 doesn't exist but a
message with UID 11002 does, it returns the data for the message with UID 11002.

Comment 12

[David :Bienvenu]

the point is that a message with that uid should exist, because we just sent it.
The append in the log is the appending of the just sent message to the sent mail
folder.

Comment 13

[David :Bienvenu]

here's more detail:

we reply to msg UID 11002 (message-id 3D0FD6DC.4070609) (see where we store the
answered flag for that uid) - we should have sent the reply at this point.

Why we're turning off the SEEN flag for 11002 is a mystery to me. The earlier
log didn't do that, as near as I can tell. I don't know if it's relevant.

 19 uid store 11002 -Flags (\Seen)

then we do a bunch of noops that don't tell us about 11003.

then, it looks like Navin replied to another message, in order to kick start the
process. At that point, we get both 11003 and 11004. Note that 11003 is a reply
to <[e-mail]>.

Now, it's a little bit of a leap of faith that the replies happen as I'm
describing them, and I was not able to reproduce this myself. However, several
other people have been able to. It also seems unlikely that it's a server
problem but I don't see what the client problem could be (other than some sort
of goofy smtp send hanging problem...) 

Comment 14

[John G. Myers]

I checked nsmail-2 and found that a

tag uid fetch * flags

was indeed sufficient to get the server to report new mail.

One hypothesis is that the SMTP server tried to deliver the message to the
user's mailbox while the IMAP server had the mailbox locked (during the store of
the answered flag, for instance).  That caused the SMTP server to defer
delivery.  It's possible the second message triggered an early retry of all mail
queueud for that user--I don't remember the retry behavior of that particular
SMTP server anymore.

Comment 15

[David :Bienvenu]

that would explain it. That would also explain why it's replies that have this
problem. If that's the case, I believe we would have to say this is an invalid
bug from the client's pov. If there's a retry on the smtp server, and we knew
what the retry interval was, we could probably verify that that's what's going on.

Comment 16

[esther]

yes it does show up in the Sent folder and this still happens with released 7.0
and trunk build 20020905.  

Comment 17

[David :Bienvenu]

Esther, to reiterate, we believe this is a side effect of sharing the imap and
smtp servers on the same server - if you set your smtp server to a different
server, I bet you can't get this to happen. It's something going on on the server.

Comment 18

[Cavin Song]

Reassign.

Comment 19

[Cavin Song]

From the imap log here is the timeline that various events occurred:

14 noop
* 1002 EXISTS
* 2 RECENT
14 OK Completed  // We see 2 new msgs here.
<<<--- A reply msg is sent here (with "Subject: Re: ...").
18 uid store 11002 +Flags (\Answered)  // Store reply flag.
18 OK Completed
20 noop         // Hit Get Msg button
20 OK Completed	// No new msgs seen by the server at this point.
.  .  .
28 noop         // Hit Get Msg button again
28 OK Completed // Still no new msgs seen by the server yet.
<<<--- Sent another new msg here (with "Subject: ...").
30 noop
* 1004 EXISTS
* 4 RECENT
30 OK Completed // Now the server sees the new msgs!!!

I think that if the servers don't tell the clients that there are new msgs 
waiting there's nothing the clients can do about it. So from the log looks like 
this is the case (ie, it's a server problem) and I think John's comment in 
Comment #14 makes a lot of sense.

Comment 20

[Cavin Song]

Marking fix. Please reopen it if you think/know it's caused by other things.

Comment 21

[esther]

Verified, this is a server issue.  This now happens on 4.7 too.