Closed Bug 40701 Opened 24 years ago Closed 23 years ago

IMAP Error when using two imap clients against the same folder

Categories

(MailNews Core :: Networking: IMAP, defect, P2)

Product:
MailNews Core
Component:
Networking: IMAP
Platform:
x86
Windows 2000
Type:
defect

Tracking

(Not tracked)

Status:
VERIFIED FIXED
Milestone:
Future

People

(Reporter: bugzilla, Assigned: Bienvenu)

References

Details

Attachments

(3 files)

IMAP log with "UID fetch -1,12180" in it!
85.11 KB, text/plain
Details
bandaid fix
714 bytes, patch
Details | Diff | Splinter Review
don't put the -1 keys in the uid array at all
2.57 KB, patch
Details | Diff | Splinter Review 
I get a popup error saying "Expected SPACE not found" when using both Netscape 
and Mozilla to query the same IMAP account.
I'm not sure if' anything can be done, but I never expericenced the same 
problems when using Netscape and Outlook and the same account.

To reproduce.
- Setup Mozilla and Netscape to use query the same IMAP account.
- Go to your Netscape and send a mail to yourself.
- Check mail and download the new account with Netscape
- Read the mail in Netscape
- Go to Mozilla and press "Get Msg"
Now you get a error. The IMAP log says:
1160[1f37bf0]: tarkin.ost.tele.dk:S-INBOX:SendData: 17 UID fetch 9505,9504-1 
(UID RFC822.SIZE FLAGS ENVELOPE BODY.PEEK[HEADER.FIELDS (Priority X-Priority 
References Newsgroups)])
1160[1f37bf0]: tarkin.ost.tele.dk:S-INBOX:CreateNewLineFromSocket: 17 BAD 
Protokolfejl: "Expected SPACE not found"
Forgot using build 2000052520

BTW another bug in the popup error dialog that Mozilla shows me is the ugly [] 
append to the text (bug 26527)
strange.  I use both all the time (4.7 and seamonkey)
QA Contact: lchiang → huang
Summary: Error when using both Mozilla and Netscape → IMAP Error when using both Mozilla and Netscape
Cant you reproduce the error as I described?
Gemal, this could be a good catch on your part. I think we are issuing bad
protocol here. According to the IMAP4rev1 RFC, we are supposed to have a space
after the ']' enclosing the BODY.PEEK field.

I can attach a patch? are you running debug builds where you could apply a patch
or do you only have a release build? It would be ideal if we could get you to
confirm that this patch would fix it on your system. I believe you are using a
MSExchange server.

Nominating for beta2. I want to get this in before mail connect next week.
Keywords: interop, nsbeta2
I'm using the moz-win32-installer so no chance there. Yes, I'm using a Exchange 
server.
Please ignore my comment about where we needed to add the space. I mis-
interpreted the BNF notation in the RFC. Adding the space is not correct. I'm
still trying to figure out what the exchange server is complaining about.

Hmm, I've gone through the BNF notation for the imap protocol to reconstruct the
syntax for the FETCH command we are issuing. According to the spec, it looks
correct to me. I don't understand why the exchange server is not liking the
command. cc'ing John Meyers in case he can quickly spot a syntax problem in the
command our client is issuing:

1160[1f37bf0]: tarkin.ost.tele.dk:S-INBOX:SendData: 17 UID fetch 9505,9504-1
(UID RFC822.SIZE FLAGS ENVELOPE BODY.PEEK[HEADER.FIELDS (Priority X-Priority
References Newsgroups)])
1160[1f37bf0]: tarkin.ost.tele.dk:S-INBOX:CreateNewLineFromSocket: 17 BAD
Protokolfejl: "Expected SPACE not found"

After turning on logging in Outlook Express this is what I get:

Outlook Express:
FETCH (BODY[HEADER.FIELDS (References X-Ref X-Priority X-MSMail-Priority 
X-MSOESRec Newsgroups)] {53}

Mozilla:
UID fetch 9505,9504-1 (UID RFC822.SIZE FLAGS ENVELOPE BODY.PEEK[HEADER.FIELDS 
(Priority X-Priority References Newsgroups)])

The same thing actually happens the other way around. Try this:
- Setup Mozilla and Netscape to use query the same IMAP account.
- Select another folder than the INBOX with your Netscape 4.73
- Send a mail from Mozilla to yourself that lands in you INBOX
- Read the mail in Mozilla
- Go to you Netscape and select INBOX

After doing so I'm getting this error with Netscape 4.73:
"The current command did not succeed. The mail server responded: Protokolfejl: 
Invalid message set in FETCH command"

The "-" in "9505,9504-1" is not permitted by the IMAP syntax.

Thank You John. I'm glad I cc'ed you on this =).

Target nominated bugs to M16.  Please update as necessary.
Target Milestone: --- → M16
Putting on [nsbeta2+] radar for beta2 fix.
Whiteboard: [nsbeta2+]
I have not been able to find a code path that gives us a '-'  character in the
message range. It's a number everywhere in our code. I wonder if memory is
getting corrupted or something......

Since 4.7 also has the problem I wonder if your mail server is giving a
corrupted id for this particular message?

I'll take this.
Assignee: mscott → bienvenu
M16 has been out for a while now, these bugs target milestones need to be 
updated.
moving to m18, but will probably fix sooner. I've more or less figured out 
what's going on.
Status: NEW → ASSIGNED
Target Milestone: M16 → M18
Basically, what seems to be going on is that we're getting an unsolicited flags
change response for a header we haven't downloaded yet (which is what Henrik's
multi-client steps to reproduce can cause - on Netscape's IMAP server, it helps
to toggle yourself offline to make the server flush the inbox state after
reading the message with 4.x, and before getting messages with 6.0). When we get
this response, it seems to mess up the nsImapFlagAndUidState, such that the last
couple UID's are out of order (a big no-no, since the code assumes that array is
sorted) and the array size is out of sync with the number of UID's we think we
have in the array. This causes the code that generates the range numbers to
download to put a -1 in the range because it gets back an nsMsgKey_None (-1)
when asking for the last uid. So that part is easy to fix, but I need to figure
out how the array got out of sync in the first place. It's not surprising if 4.x
and 6.0 both have this problem since all that code is the same.
You probably want to ignore FLAGS notifications for headers you haven't 
downloaded yet.

yes, since we're just going to ask for the flags when we download the headers. 
It still doesn't explain why the array gets horked, though.
the 4.5 code that was inserting new uid's into the array did not work if the new 
uid wasn't the largest uid it had seen. Charming. I've got a fix for that.
fix has been reviewed - removing interop keyword - it's not an interoperability bug.
Keywords: interop
hmm, that commit worked (sorry, cc'list people - I'm using this bug to test stuff)
fix checked in.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago
Resolution: --- → FIXED
Verified 07-17-09-M17 Windows build for WinNT commercial build.
No error is displaying when using both Mozilla and Netscape now.
Henrik, since this bug is reported from you & it's from windows 2000, would you 
please help for verify on your windows 2000 for us? Thanks.
Please mark as verify if this problem has been fixed.
Verified on build 2000071708 on Win2k with Exchange
Status: RESOLVED → VERIFIED
Thanks for double checking for this bug!!
Hmmm... this is now happening again with another error:
BAD Protokolfejl: "Invalid message set in FETCH command"

To reproduce:
- Setup Mozilla and Netscape to use query the same IMAP account.
- Go to your Netscape and send a mail to yourself.
- Check mail and download the new account with Netscape
- Read the mail in Netscape
- Go to Mozilla and press "Get Msg"

You you get an error!

This is on a exchange server, but I dont think this matters.

The problem is that Mozilla sends this IMAP command:
UID fetch -1,12180 (UID RFC822.SIZE FLAGS ENVELOPE BODY.PEEK[HEADER.FIELDS 
(Priority X-Priority References Newsgroups)])

is -1 valid??????
Status: VERIFIED → REOPENED
Resolution: FIXED → ---
No, -1 is not valid. I don't know why this broke again. It was fixed. I think
this is a different but similar bug. I'll try to reproduce it later.
Status: REOPENED → ASSIGNED
Keywords: nsbeta2 → nsbeta3
Whiteboard: [nsbeta2+]
I cannot reproduce this on NMS 4.15 IMAP mail account by using 08-01-05-M17 
commercial build. I will try on MS Exchange server later.
I still cannot reproduce this problem on MS Exchange server for the same build
by following Henrik steps. I am wondering to know whether this problem is
occurring for some specific message.......
A little more detailed walk through to reproduce. Hope you can reproduce it:

1) Start Mozilla and Netscape
2) Point both to the same Exchange Inbox and click Get Msg in both
3) Compose a mail from Netscape and send it to your exchange account.
4) Press Get Msg in Netscape and read the message.
5) Go to Mozilla and see that the message hasn't arrived yet.
6) Press Get Msg in Mozilla and you should get an error

My result is the same....
So no error? What version of Exchange are you connecting to?
Mine is "Microsoft Exchange IMAP4rev1-server version 5.5.2232.11"
Yes. No error.
Mine is "Microsoft Exchange IMAP4rev1 server version 5.5.1960.6".
The IMAP log shows an unsolicited FETCH FLAGS response for message 32 between 
the time the server gets the * 32 EXISTS and the time it can do a UID FETCH 
FLAGS for that message.
Keywords: mail2
- per mail triage
Whiteboard: [nsbeta3-]
This bug is driving me nuts...
My log says: "12 UID fetch -1,14484"

100% way to reproduce:
0) Shutdown Mozilla
1) Start Mozilla Mail and Netscape Mail
2) Point both to the same Exchange Inbox and click Get Msg in both
3) Compose a mail from Netscape Mail and send it to your exchange account.
4) Press Get Msg in Netscape and read the message.
5) Go to Mozilla and see that the message hasn't arrived yet.
6) Press Get Msg in Mozilla and you will get the error

build 2000091108
Severity: normal → major
Cant the code just be modified so that it never generates a "-1"

I know it's not a perfect patch but it's a start!
removing mail2 keyword.
Keywords: mail2
adding mail3 keyword
Keywords: nsbeta3 → mail3
Whiteboard: [nsbeta3-]
changing priorities
Priority: P3 → P2
marking nsbeta1- due to the fact that it won't affect many users and it looks
like we are having trouble reproducing.
Keywords: nsbeta1-
Target Milestone: M18 → Future
Should I open a new bug regarding on the "UID fetch -1,2514" command that
Mozilla sends, since this bug seems to contain more than one problem.
This "UID fetch -1,2514" bug is really annoying...
No, this bug is about the -1 problem.
*** Bug 85814 has been marked as a duplicate of this bug. ***
Are y'all still having trouble reproducing this? It is a significant problem for
me - and happens 100% reproducibly, all you have to do is use two separate mail
clients against the same server and make a change to a folder that mozilla has
used, such as Sent Items. (in my case, it happens every time I: 1. send a
message with mozilla (linux) followed by 2: clean up/alter/etc. contents of Sent
Items using Outlook in another window.)

It seems like a simple fix/workaround would be to have mozilla completely forget
about a folder (i.e. go back to 'just started mozilla today' state) if it gets
an error accessing that folder that is non-data-loss in nature. I.e. if the
client is trying to update it's list of messages, and it fails to access the
folder or messages properly, just completely blank all in-memory status for that
folder. 
*** Bug 91911 has been marked as a duplicate of this bug. ***
OK, I'll try to fix this today, if I can recreate it (the problem is that our
server makes this hard to recreate).
If you are unable to reproduce, contact me privately, and I'll get you 
temporary account on our exchange server (where problem is very easy to 
reproduce).
Keywords: interop
Summary: IMAP Error when using both Mozilla and Netscape → Exchange IMAP: IMAP Error when using both Mozilla and Netscape
Attached patch bandaid fix — — Splinter Review 

    
    

    
  
rs=mscott. Let's get this in for .9.3

    
    

    
  
I still can't reproduce this, but I saw a flaw with my previous attempt to fix
this, a fix for which I've checked in. I'm trying to get a test account, but if
someone who can reproduce this problem can try with tonight/tomorrow's build,
I'd appreciate it.

    
    

    
  
This is not only Exchange. Reproduces on my Software.com IMAP server.

My IMAP log shows:
SendData: 12 UID fetch -1,1057 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS 
(From To Cc Subject Date Message-ID Priority X-Priority References Newsgroups)])

Note the -1 in the fetch command...

build 2001072403

I can provide you with an account if you want...
Summary: Exchange IMAP: IMAP Error when using both Mozilla and Netscape → IMAP Error when using both Mozilla and Netscape

    
    

    
  
The subject isn't really correct. Can be reproduces when using Mozilla and any 
tool that can access the inbox like Outlook Express, Netscape, webmail, etc.

The problem is the second fetch command Mozilla sends.

The setup:
1) I have 1 mail in my inbox and checks mail with IMAP in Mozilla. Mozilla sends 
"S-INBOX:SendData: 7 UID fetch 1:* (FLAGS)" which is correct
2) Now I use a webmail on the same inbox to send a mail to myself and marks the 
mail as read. Now I have two read mails in my inbox, but in Mozilla there is 
still only one. Which is correct since I havn't press the Get new msgs.
3) Now I press the Get new msgs in Mozilla. Mozilla now incorrectly sends the 
"SendData: 15 UID fetch -1,1057" plus some more about the flags, etc. But the 
main problem is the -1

    
    

    
  
On Linux, Build 2001072504 still has this problem. 

    
    

    
  
changing summary, removing interop keyword - it's a general problem with several
different servers.
Keywords: interop
Summary: IMAP Error when using both Mozilla and Netscape → IMAP Error when using two imap clients against the same folder

    
    

    
  
Well, here's a big part of the problem. The exchange server (and perhaps other
servers) returns unsolicited flag responses without the uids, e.g.,

804[4f228d8]: fep2.sprawl.dk:S-INBOX:CreateNewLineFromSocket: * 16 FETCH (FLAGS
(\Recent \Seen \Deleted))

804[4f228d8]: fep2.sprawl.dk:S-INBOX:CreateNewLineFromSocket: * 17 FETCH (FLAGS
(\Recent \Seen \Deleted))

and we're not handling that correctly. bullet proofing fix for that coming up.


    
    

    
  
Can I get reviews from Cavin and Seth, please? The fix is simply to ignore flag
responses for which we don't have the UID. I also cleaned up a little bit of the
braces style, and used an nsMsgKeyArray method to Remove an element instead of
doing it by hand. Thanks.

    
    

    
  
I should add that the only way I was able to reproduce the problem was with the
following steps:

1. Have one or two message in your inbox
2. Open the inbox with the two mail clients 
3. With one of the clients, copy four or five messages into your inbox.
4. Delete those same messages
5. Do a get new mail with that same client.
6. Do a get new mail with the second client.


    
    

    
  
fix checked in.
Status: ASSIGNED → RESOLVED
Closed: 24 years ago → 23 years ago
Resolution: --- → FIXED

    
    

    
  
Henrik, since this bug cannot reproduce from our internal MS Exchnage server.
Do you have test accounts for me to verify this bug? Can you email the test 
account info to me huang@netscape.com?
Or can you help for verifying this bug? (Please mark as verify if this problem 
has been fixed). Thanks.

    
    

    
  
I was the guy who got y'all test accounts a while back, and I'm pretty sure this
has been resolved. I know I never see any more problems with IMAP fetch errors
since that fix was applied.

    
    

    
  
seems fixed. Tested using Outlook Express, Mozilla and webmail all using the 
same IMAP account...
Status: RESOLVED → VERIFIED

    
    

    
  
damn!
Just got this one in my IMAP log with build 20011010. I was using my Netscape 
4.78 and Mozilla on the same MS Exchange IMAP mailbox:

116[2ab4c08]: tarkin.int.tele.dk:S-Sent:SendData: 22 UID fetch 
0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 (UID RFC822.SIZE 
FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Subject Date Message-ID Priority 
X-Priority References Newsgroups)])
Status: VERIFIED → REOPENED
Resolution: FIXED → ---

    
    

    
  
I got the same error 3 times. Everytime it was a UID fetch just after fetching 
a folder with 2001 messages in it.

    
    

    
  
This seems to have been regressed.  I moved from 0.9.3 to 0.9.5 on Windows 2K,
and from 0.9.4 to 0.9.5 on Linux (RH 7.1) and the error showed up again on both
platforms....

    
    

    
  
perhaps this is really fixed and what we're seeing is bug 94738 ?

    
    

    
  
Is anyone still seeing this? I suspect it was fixed 10 days ago or more.

    
    

    
  
seems ok....

    
    

    
  
Actually, I cannot reproduce this problem originally. But since bug 97497 & bug 
94738 fixed and according to reporter's (Henrik) verification. David, can you 
mark as fix for this bug, so I can close this bug? Thanks.
Status: REOPENED → RESOLVED
Closed: 23 years ago → 23 years ago
Resolution: --- → FIXED

    
    

    
  
Marking as verified based on above reporter's(Henrik) verification.
Status: RESOLVED → VERIFIED
Product: MailNews → Core
Product: Core → MailNews Core

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: