User-Agent: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.7.3) Gecko/20040922 MultiZilla/220.127.116.11b Mnenhy/0.6.0.104 Build Identifier: Mozilla/5.0 (X11; U; Linux i686; de-AT; rv:1.7.3) Gecko/20040922 MultiZilla/18.104.22.168b Mnenhy/0.6.0.104 It's possible to easily construct FETCH or STORE queries with mozilla that crash IMAP servers. The same problem, reported by a different user can be found here: http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=23846 The reason here: http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=23849 Cite: "Yes, it happens when you send a query that's to long. I.e. a very large msgs_id string in combination with FETCH or STORE. The imap-client you are using should split up those imap-queries and use smaller message sets." Reproducible: Always Steps to Reproduce: 1. Select all Mails (Ctrl-A) in a large folder, e.g. "Sent". 2. Mark all Mails at once as "Not Junk" using the menu. Actual Results: Mozilla displays error window, that the connection to the IMAP server was lost. I could find the reason in my imapd.log: Oct 11 10:06:24 gaia imapd: open: user boris opened Sent Oct 11 10:06:53 gaia imapd: Fatal error: word too long Oct 11 10:06:53 gaia master: process 23161 exited, status 75 Expected Results: According to Marc Groot Koerkamp (see links above) the query has to be splitted. I don't know if that is an requirement of the IMAP RFC or a cyrus-specific issue. Please fix it anyway. I use cyrus-imapd 2.1.16 and mozilla 1.7.3.
It doesn't seem to be very clear in the RFC what the maximum length is, but for compatibility reason, this should be fixed. Can you log what Mozilla sends exactly, and make sure what is the maximum viable length ? Have a look at http://www.mozilla.org/quality/mailnews/mail-troubleshoot.html#imap to see how to activate IMAP log in mozilla.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Created attachment 161762 [details] Gzipped IMAP log file from mozilla I removed "private" parts from the start and end of the log file. Search for "too long" to find the error message.
Looking at the sources it seems to me that 8K is the right size to go. At least cyrus and dovecot IMAPD use both 8192 bytes.
After more verifcation, I see we should probably close as a duplicate of bug 91269 (yes, it's an old problem ...). Can you do it if you don't see a reason against that ? It would be good to raise up the issue, but bugzilla is not the right place for advocacy ...
Did you try a newer build? I fixed some instances of this problem in 1.8a builds (in particular some of the Store commands, and some fetch commands). bug 91269 is more about the COPY command, which I don't believe I've fixed, though it's on my list of things to do. There is a method that knows how to deal with the uid ranges to shorten the lines, but I haven't applied it to the COPY command.
Currently using Thunderbird "version 22.214.171.124 (20070326)" I am having the same problem where tbird sends a command that is too long and my mail server disconnects me. 1736[672f9b0]: 254a218:imap.[private].com:S-INBOX:SendData: 13 UID fetch 311261:311262,311268,311325,311336,311345,311351 ,311356,311398,311417,311441,311457,311527,311538,311572,311591,311596,311762,311778,311785,311843,312062,312168,312172, 312307:312308,312793,312807,312812,312816,312819,313054,313414,313536,313645,313647,313723,313957,314106,314673,314922,3 15016,315441,315550,315819,315840,316062,316471,316484,316490,316509,316585,316597,316708,316734,31 1736[672f9b0]: ReadNextLine [stream=24a84d0 nb=79 needmore=0] 1736[672f9b0]: 254a218:imap.[private].com:S-INBOX:CreateNewLineFromSocket: * BYE [ALERT] Fatal error: max atom size too s mall: No such file or directory 1736[672f9b0]: 254a218:imap.[private].com:NA:TellThreadToDie: close socket connection 1736[672f9b0]: ImapThreadMainLoop leaving [this=254a218] As reported elsewhere, this is a message returned when the command being sent is too large for a buffer. Is there a workaround presently?
I cannot presently build, and I'm still trying to figure out how to contribute properly. Looking in nsImapProtocol.cpp, under nsImapProtocol::Store, I find this line: PRUint32 msgsToHandle = msgCountLeft; It also appears in ::Fetch and ::Copy. All three methods take a list of messages, divide them into an array of message ids, then step through a loop until all the messages have been processed with the command. The size of the batch of messages to process is msgsToHandle. Unfortunately, this line makes it appear as though it always executes exactly one batch of messages, by setting it on the first run to be equivalent to the number remaining (all of them). Perhaps what we meant was something like, msgsToHandle = 50; //arbitrary batch size if(msgsToHandle > msgCountLeft) msgsToHandle = msgCountLeft; //within the loop There's some other strange interaction in here too: each of the methods takes an idsAreUid which seems to affect how the messageList string would be parsed. But msgKeys (the array into which it is parsed) looks like it would never be filled, ever, if !idsAreUid. Also, there's an nsCString messageIDList declared in each message that appears to be unused? I'm going to try and get my own build working and see if I can fix this. What confuses me most is that logs seem to indicate that there IS some splitting into batches for things, but that perhaps it's being done before the store/fetch methods are called?
I tried my "Not Junk" test case on a folder with more than 9000 mails, and it worked perfectly. In the meanwhile I use Seamonkey 1.0.8 and cyrus-imapd-2.2.12-3.RHEL4.1.
Brian, any news? Also, what is exact error message that displays in status/UI?
Summary: IMAP Fetch-Query too long → IMAP Fetch-Query too long - long FETCH or STORE command may cause mail server disconnect
OS: Linux → All
QA Contact: grylchan → networking.imap
Hardware: PC → All
(In reply to comment #7) > I'm going to try and get my own build working and see if I can fix this. What > confuses me most is that logs seem to indicate that there IS some splitting > into batches for things, but that perhaps it's being done before the > store/fetch methods are called? Any luck in building ?
ludo, Brian's address bounces. so we need someone to run with the draft patch bienvenu, question from comment #7 > What confuses me most is that logs seem to indicate that there IS some splitting > into batches for things, but that perhaps it's being done before the > store/fetch methods are called? (note, bug 91269 mentioned in comment 5 is fixed, per bienvenu)
Whiteboard: [patchlove][has draft patch][needs owner][batch operation]
(In reply to comment #11) > ludo, Brian's address bounces. so we need someone to run with the draft patch shall we try http://psinet.cjb.net/ ?
(In reply to comment #12) > (In reply to comment #11) > > ludo, Brian's address bounces. so we need someone to run with the draft patch > > shall we try http://psinet.cjb.net/ ? ludo, any feedback?
(In reply to comment #13) > (In reply to comment #12) > > (In reply to comment #11) > > > ludo, Brian's address bounces. so we need someone to run with the draft patch > > > > shall we try http://psinet.cjb.net/ ? > > ludo, any feedback? No :-(
Nikolay not taking? :(
Whiteboard: [patchlove][has draft patch][needs owner][batch operation] → [bulkoperations]
I just duped bug 239951 and in bug 239951 comment 5 I attempt a history about command length and state that I found no support forum reports and find nothing in bug reports, except this bug 263821. Comment 6 above claims there was still a problem in version 2. Comment 7 is noteworthy - it's not clear the code around http://hg.mozilla.org/comm-central/annotate/12e6710c1ed6/mailnews/imap/src/nsImapProtocol.cpp#l5135 entirely does the job. ref also http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/mailnews/imap/src/nsImapProtocol.cpp&rev=1.688&mark=4781#4781 I also wonder whether bienvenu's comment 5 has been fixed. (I didn't dig)
Severity: normal → major
Summary: IMAP Fetch-Query too long - long FETCH or STORE command may cause mail server disconnect → IMAP Fetch-Query too long - long FETCH or STORE command may cause mail server disconnect (Too long argument)
Removing myslef on all the bugs I'm cced on. Please NI me if you need something on MailNews Core bugs from me.
You need to log in before you can comment on or make changes to this bug.