Closed Bug 83396 Opened 24 years ago Closed 16 years ago

Alert "message contains bare newlines" displays and lost messages when copying msgs from Local(or other IMAP account)folder to an IMAP account folder

Categories

(MailNews Core :: Networking: IMAP, defect, P2)

x86
All
defect

Tracking

(Not tracked)

RESOLVED WORKSFORME
Thunderbird 3.0rc1

People

(Reporter: huang, Assigned: Bienvenu)

References

Details

(5 keywords)

Attachments

(7 files, 2 obsolete files)

Used 05-30-06-trunk build Got an abnormal alert when trying to copy local msgs to an IMAP folder 1) Copy about 200 msgs from local to an IMAP folder 2) Actual Results: didn't copy completely and got an abnormal alert: "message contains bare newlines"(see following attach screen shot) Expected results: Should copy successfully & completely without this alert
Attached image Attach this alert info
This again is not reproducible always. cc sheela, who has seen this come and disappear for the same message.
Nominating nsbeta1 since it seems that many peoples is hitting this problem.....
Keywords: nsbeta1
*** Bug 83831 has been marked as a duplicate of this bug. ***
changing summary as David mentioned on bug 83831 - this only happens on some messages....
Summary: Got an abnormal alert when trying to copy local msgs to an IMAP folder → Can't copy some messages from local mail to imap folder
so this isn't a true dup of the other bug - the error message I was seeing had to do with an invalid message header because we're copying the From - envelope header. This is a similar bug, however, and I've seen it as well on different messages than the ones that generate the above error message.
Looks like there are different problems occurring for copying local msgs to IMAP folder now....restoring this summary back based on David's comments.
Summary: Can't copy some messages from local mail to imap folder → Alert "message contains bare newlines" displays when copying local msgs to IMAP folder
reassigning to naving. It sounds like we need to find the message that causes this. Sheela or Karen since you've both seen this, can you try finding a message?
Assignee: mscott → naving
Whiteboard: [nsbeta1+]
Priority: -- → P2
Target Milestone: --- → mozilla0.9.2
From 200 msgs??....I need to investigate which one!!....
But, I did have the IMAP log saved at that time when copying the 200 msgs. I will attach here, hope it help....
adding pdt+
Whiteboard: [nsbeta1+] → [nsbeta1+][PDT+]
It seems that I losted the messages duing copying these 200 messages. I only have 54 messages left and I am still couldn't finding the specific message which causing this alert and losting messages. Updating summary to include losting messages.
Summary: Alert "message contains bare newlines" displays when copying local msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local msgs to IMAP folder
Try deleting the msf file. They may come back. If you are copying messages, how can you lose them ?
I used drag/drop for copying these messages, it probably moving the messages instead of copying messages...I just couldn't find them...but I willl keep finding them.... :-(
I cannot find any such message. Also only that message would not be copied but the rest of messages (multiple copy) should be copied after bug 83831 gets checked in.
Depends on: 83381
Depends on: 83831
No longer depends on: 83381
moving to 0.9.3
Target Milestone: mozilla0.9.2 → mozilla0.9.3
Does the message generating this alert disappear, causing data loss?
Whiteboard: [nsbeta1+][PDT+] → [nsbeta1+]
I have been able to reproduce this problem on linux machine with yesterday's trunk build. I have this message that causes the bare lines alert dialog trying to move it back to an imap folder. The following are the steps to produce this problem: create a new message with a jpg attachment, some text and send it to yourself After receiving it in the imap account Drag and drop it to pop account folder Go to your pop account folder where you moved the message select that message and click on the file button and try to move it back to an imap folder Results in the alert dialog where it fails to move the message back to imap folder Navin, If you are not able to reproduce using steps above stop by I can reproduce this on my machine.
There is no dataloss. The only bad thing is the messages does not copied.
Navin, I deleted the .msf file of the local folder where I was trying to move that message and relaunched the application and tried to move again and resulted with the same error still. So deleting .msf file did not do any good to get rid of the problem.
That's right, I was not able to just move the message from the local folder back to an imap folder.
Target Milestone: mozilla0.9.3 → mozilla0.9.4
Mail news triage meeting --> .9.5
Target Milestone: mozilla0.9.4 → mozilla0.9.5
I can reproduce this probelm by using the attach msg as following.
moving to 0.9.6
Target Milestone: mozilla0.9.5 → mozilla0.9.6
Blocks: 104166
moving to 0.9.7
Whiteboard: [nsbeta1+]
Target Milestone: mozilla0.9.6 → mozilla0.9.7
I migrated several users from POP to IMAP accounts and noticed that copying from local to IMAP failed on two cases: - message had only headers and no body. Didn't check was there any empty lines or was the message body just empty. I guess that the error message was "message contains bare newlines" - message header had first line as "From - ..." used as e-mail separator at some mailboxes. There was another error thrown out, unfortunately I don't remember exact wording. I assumed that this was Cyrus IMAP server issue, but since you are talking about it here....
Keywords: nsbeta1+
Target Milestone: mozilla0.9.7 → mozilla0.9.9
Priority: P2 → P3
moving to 1.0.1
Target Milestone: mozilla0.9.9 → mozilla1.0.1
Blocks: 122274
Status: NEW → ASSIGNED
Keywords: nsbeta1+nsbeta1-
Target Milestone: mozilla1.0.1 → mozilla1.2
*** Bug 142620 has been marked as a duplicate of this bug. ***
Olga seeing this problem when copying messages from other IMAP account to NMS 6.0 (tintin.mcom.com) account. So, this seems is also occurring for copying IMAP account messages to IMAP account messages.....
Summary: Alert "message contains bare newlines" displays and lost messages when copying local msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local or IMAP msgs to IMAP folder
Summary: Alert "message contains bare newlines" displays and lost messages when copying local or IMAP msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local or IMAP folder msgs to IMAP folder
It was Web Mail Account.
Olga, did you copy from Webmail IMAP to NMS 6.0 (tintin.mcom.com)IMAP? Can you attach that problem message here? Thanks.
Olga, did you lost this message during the copy? We may need to nominating nsbeta1 again if msgs have been lost during the copy......
Summary: Alert "message contains bare newlines" displays and lost messages when copying local or IMAP folder msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying msgs from Local(or other IMAP account)folder to an IMAP account folder
This morning after re-starting NO problem with copy/move neither from Web Mail nor from another POP, IMAP. Yesterday I saw the same alert mentioned in comment #1.
I can now get this 100% when trying to move a message that contains text and a attached jpg image from a Exchange IMAP server to a Cyrus IMAP server...:(
*** Bug 159323 has been marked as a duplicate of this bug. ***
I'm seeing this also copying IMAP -> IMAP on Linux.
OS: Windows NT → All
In particular, I'm copying from a UW IMAP server to a Netscape Mail Server.
cross-imap server move/copy copies the msg down to a local tmp folder and then up to the destination imap server. The problem is that some servers (like our NMS server) are very picky about the format of the messages they allow to be appended to a folder.
I have the same problem with mozilla (1.0 and 1.1) on linux when trying to move some local mails to my Cyrus IMAPD server. I hope to see a quick fix rsn so that mozilla skips the mails with bare newlines and continues the copy/move operation. Is ist now clear if mozilla has problems or if the imap servers have problems? Reading this thread I'm not sure about that.
*** Bug 133377 has been marked as a duplicate of this bug. ***
Check http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=5629 for a statement about this bug/feature/enforcement in Cyrus IMAPD.
QA Contact: huang → gchan
mass re-assign.
Assignee: naving → sspitzer
Status: ASSIGNED → NEW
I try to set up a Cyrus server, and the problem is still there in mozilla 1.5 Linux. A google search tend to say that it's a problem with mozilla and not Cyrus. The problem is always reproductible when I create a mail with big attachment, I save it, and I try to move it to the Imap server. Hope it will be solve soon...
Is this problem more easily reproducible on Linux?
Assignee: sspitzer → bienvenu
I used to see this from time to time on Linux, and I don't think I've ever seen it on Windows, so I suspect it may in fact be platform-specific.
It happened to me in Windows when I copied a large number of messages from one IMAP folder to another folder on the same server. This was back in the 1.3 days though, now I usually just use a web interface for my IMAP provider to move a large number of messages.
from one imap folder to another on the same server does not involve the client, or at least its line endings - it's simply a copy command sent to the server, just like the web client would do.
My mistake, you are correct. The message copies I made were from one IMAP server to another, MyRealBox -> Fastmail I believe. I remember being frustrated with the "bare newlines" error and came across a "Migrate IMAP" feature within Fastmail's website and used that instead of Mozilla.
I would like to add that I'm seeing this bug in Mozilla 1.5 and Thunderbird 0.7 on a windows XP home ed machine when moving mails from the local folders to an Cyrus-IMAPD account. Any Ideas? Tarjei
... this bug is around for a really long time and annoying quite some people. migrating from an archive of local pop-messages to an imap solution is not so uncommon today, i guess. however, nobody seems to have even looked at any code that could cause this problem... can we increase the priority of this an get somebody to take a look at it? imho this is one of the most visible mail bugs!
I fear that's the classic problem that the person knowing the code can't reproduce the problem and vice versa. David, would a tcpdump log help you more than a Mozilla log?
Severity: normal → major
Target Milestone: mozilla1.2alpha → ---
sorry for being a bit nervous in my last post... i did some research on this, with the following results: 1.) AFAIK, this problem happens only with the Cyrus IMAP server. The Cyrus folks follow a zero-tolerance policy concerning RFC violations, and reject malformed mails submitted with the IMAP APPEND command. 2.) The actual bug is found in the e-mail clients sending non-RFC-compliant mails (e.g. Outlook up to version 5). If such a mail is received by mozilla through a POP connection, it is stored on disk (as is) and displayed correctly. If such a mail is received by Cyrus IMAP by a local sendmail daemon, it is (AFAIK) corrected and stored on disk in the correct format. 3.) The problem occurs when you try to move such a mail stored by mozilla locally to the Cyrus IMAP server - mozilla delivers the mail in the original (non-compliant) format, and Cyrus thinks it is talking to a non-compliant client and rejects the mail. the problem is that we have a deadlock situation here - IMHO mozilla is right to store the mails in the original form, and cyrus is right to reject non-RFC-compliant mails. and it makes no sense to blame "the other" clients, since even if they fix this problem in future releases, this won't change the messages already received by mozilla. in addition, since sendmail usually corrects malformed mails, this problem is not very visible to end-users of such buggy email clients - usually their messages are delivered without problems (even to Cyrus). this problem only occurs when such a malformed message is "replayed" by mozilla. the possible fixes to this problems that i can see are: 1.) talk to the cyrus folks about theirt policy. i am not sure if they are aware of this scenario when they argue in favour of their zero-tolerance policy. 2.) make mozilla correct the stored emails when sending to an IMAP server. this is possibly not trivial(?). the three main causes for these kinds of problems are: LF without CR in the mail body, NUL characters in the mail body, or malformed headers. (the relevant RFCs are RFC821 (SMTP), RFC2060 (IMAP4), and RFC1939 (POP3), i guess) 3.) offer end-users a link to an external program, that is able to fix their mailboxes when needed. unfortunately, i haven't found such a software by now... or build a hack into mozilla, to either re-send these mails via SMTP to the account (with the original date), or something like that (probably least preferred by all) fact is, that i and other people are completely stuck in the process of migrating our mail archives to IMAP. some people just deleted the messages, but i have a large archive reaching back to 1996 and i would loose 1/3rd of my historic messages - completely unacceptable for me. if anyone can think of a hack to allow me to move my mails, i would be very happy for now.
Flo, to follow strictly the RFC is good in my opinion. I know about the "be tolerant when reading/recieving" theory, but I don't agree with it. The problem that made me look into this bug can indeed be caused by imported Outlook mails. But 1. this would mean a bug in the import code, and 2. doesn't absolve the code that writes to IMAP from responsibility. To this mails on Unix/Linux are also stored with LF only line endings in the mbox (and CR only on MAC). In result 2.) you don't write about importet mails from non-RFC-compliant clients but received through POP. That confuses me - do you know a mail with LF is written with LF (on not-Unix-systems) or do you just think so? AFAIK the code in RetrResponse() and ReadNextLine(), line endings are stripped away uppon receiving a line and the gets a new one depending on the platform. So we end up with mboxes with different line endings on different operating systems. I think your conclusion 2.) is the right one. David, don't we really convert the line endings when appending to IMAP?
FYI: i am sitting on a windows XP box. and now that you mention it: i couldn't actually find the LF characters that Cyrus complains about in my mbox files... that was my first try, to fix the mbox files manually. so maybe it is really just a bug in the mozilla IMAP code. but sorry i can't help you here, i am not a mozilla hacker (and don't currently plan to become one, due to time constraints) the reason why i thought it is the original source of the mail that couses the troubles, is, that this bug occurs only for some mails. and for me, there are some senders (using for example outlook) that are more frequently the source of such mails than others. but, you are right, this is just speculation... of course strictness with respect to RFCs is good. i just thought that this was a scenario that the RFCs didn't fully consider.
Flo, that was a very accurate summary. This problem also happens with the Netscape IMAP server, since it was developed by the same person as Cyrus. We could fix the imap append code to repair naked newlines, and we could replace embedded nulls with spaces - malformed headers are a bit trickier. I'll try to look into it today.
just one more comment: it would be great if mozilla would, if you move messages, at least delete those messages that have been moved succesfully. at the moment, if you move multiple messages at once, and the error occurs in one message, the messages that have been copied successfully are not deleted from the original folder, resulting in duplicate messages that are very hard to sort out. one has no clue which messages have been copied and which not, and which message caused the error. this is also what the reporter of this bug pointed out, i think. or maybe open a new bug "remove successfully moved messages from original folder even if an error occurs"? i haven't looked if this happens in other cases too...
Flo, the duplicate mail problem is indeed annoying. Shouldn't it be possible to use formail to remove the duplicate mails from the local folder?
unfortunately, the duplicate mails are now on my imap server. is it possible to use formail on the mailboxes on the server? i could theoretically have access to the server to fix my mails. could formail be used to fix invalid headers and the other problems mentioned above? i have never used this tool...
The formail command is part of the procmail package found on most Linux systems. It can split mailboxfiles into seperate mails and can also manipulate singe mails. But it may break something if you directly let it work on cyrus' mail files. Even when you turn cyrus off. May be the db files of cyrus get out of sync. Surely they will, if you delete some mails with a simple rm. On the other hand formail is perfect for header manipulation. Check man formail.
one tricky thing about fixing this is that we need to fix it when we create the temp file who's contents we append to the imap folder, because the append command requires us to specify the message size up front, and we use the size of the temp file for that. Thus, we can't be inserting extra cr's or lf's to fix bare newlines at append time; we have to do it when we generate the temp file.
But that alone wouldn't be so hard. Or is it that this temp file generation is also used for other purposes where this conversion would be wrong?
yes, I think the code is used for other purposes - I'll probably need a flag that says to sanitize the copied message.
actually, the code looks to me like it handles this. Apparently it doesn't work, but when creating the temp file, it definitely scans for a CR or LF when parsing for lines, and writes a CRLF instead of a single CR or LF terminator. I'll have to try the test case.
Attached patch proposed fix (obsolete) — Splinter Review
the code was only figuring out the linebreak length once, which is wrong - it needs to figure it out for every line. This causes a slight amount of code duplication, but I couldn't figure out an easy way to just have one instance of the code that determines the linebreak length
Attachment #145760 - Flags: superreview?(mscott)
Attachment #145760 - Flags: superreview?(mscott) → superreview+
fix checked in.
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
Blocks: 240476
i had only now time to test this with a current nightly.. and unfortunately, i have to say, the bug seems to be not fully fixed. i was able to move approx. 2/3rds of the massages that would trigger the error message to my IMAP server - however, there are still messages remaining that trigger the error as before. nearly all of these messages have large attachments >200k in size. there is only a single message that triggers the error for me that is does not have an attachment, but is a 15k multipart html message. i will attach the source of this message to the bug. sorry for the bad news, otherwise thanks and respect for the quick help! using Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a) Gecko/20040421
see previous comment. this is an example for a message still not working for me.
sorry it's not completely fixed for you. I don't think that test case helps me, because the line endings are correct in it, and I can copy it to a cyrus server w/o a problem. Would it be possible for you to save off the actual message into a text file from your mailbox, using a text editor that maintains the line endings, and e-mail that to me? And maybe one of the other messages with the big attachment as well? thx...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
after a little debugging, I think the original message is OK, but we're messing up the line ending in the process of saving the message to the temp file, prior to uploading the message. When data is streamed to us, the stream might break on a CRLF boundary, and in that case, we'll write an extra blank line, I think...
Attached patch proposed fix (obsolete) — Splinter Review
handle case where crlf are split by block boundary. Thx very much for the test cases, Flo.
Attachment #146879 - Flags: superreview?(mscott)
Attachment #146879 - Flags: superreview?(mscott) → superreview+
fixed
Status: REOPENED → RESOLVED
Closed: 21 years ago21 years ago
Resolution: --- → FIXED
Sounds great! Will the fix be in release 1.7?
amazing! tried it out in the current nightly, and everything works! thanks a LOT for fixing this so fast. there are two bugs that should be spun off from this one: invalid headers in mail messages (same problem as with this bug, but probably harder to fix as indicated in comment #58), and the general problem that if a problem occurs during moving of multiple mails, you end up with duplicate mails since the mails that have been copied successfully are not removed from the source folder. how should we proceed with these two bugs? i didn't query yet if they are already there...
Comment on attachment 146879 [details] [diff] [review] proposed fix this change affects the copying of local messages to imap folders (or messages from other imap servers) - it actually fixes a data corruption problem, so it's a good one to get in. Usually, an extra blank line doesn't cause any harm, but it can be a bad thing...
Attachment #146879 - Flags: approval1.7?
Comment on attachment 146879 [details] [diff] [review] proposed fix a=mkaply for 1.7
Attachment #146879 - Flags: approval1.7? → approval1.7+
Attachment #145760 - Attachment is obsolete: true
Attachment #146879 - Attachment is obsolete: true
Keywords: nsbeta1-fixed1.7
FYI, David fixed this on the tbird 0.6 branch too
A user I've contact to still experiences this problem with TB 2004-04-28-00-trunk. I've a log here that shows this and also the mail that generates this. Unfortunately I can't reproduce the problem with my own Cyrus IMAPd and the messages, so it's difficult to help here. David, do you think there's any chance to find another bug in the move-code without being able to reproduce it yourself? I could supply you with log and mails too.
FYI: unfortunately i cannot assist you with this bug anymore, because my organisation has migrated to a different imap server. for me everything worked fine, but i was told only afterwards that the server software had changed ;)
Erm, you mean your comment #76 "everything works" was made after testing on the new server (without your knowledge)? So the bug you discovered might also still be active.
Christian, sure, send me the mail and the logs
i _think_ i verified the bug while the cyrus server was still active. but i am not sure. sorry about this, i only was told about the switch two days ago, but it happened earlier...
I'm reasonably confident that the last fix actually fixed your problem, at least for the sample messages you sent me. But I'll have a better idea when I get the mail messages from Christian.
By using the original msg that I attached on #25: http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25 I am not able to reproduce this problem when copy the Verified as fix
QA Contact: grylchan → karenlhuang
By using the original msg that I attached on #25: http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25 I am not able to reproduce this problem anymore when copy this message from Local to IMAP on latest 1.7 branch 06-24 build. Changing keywords from fixed1.7 to verified1.7. Leave this bug status "as is" until this bug be verified on trunk again...
Keywords: fixed1.7verified1.7
By using the original msg that I attached on #25: http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25 I am not able to reproduce this problem anymore when copy this message from Local to IMAP on latest 1.7 branch 06-24 build. Changing keywords from fixed1.7 to verified1.7. Leave this bug status "as is" until this bug be verified on trunk again...
Ok, I upraded to the mozilla-1.7-0.3.2 rpm from the FC2 devel-branch. Copying mails from local to cyrus-imapd that previously triggered this bare lines error seems to be better now. But I came across an other problem, mozilla refuses to copy a certain email with this error message: current command not successful: mail-server responded: message contains invalid header. I translated this back from the german locale to english. I'm going to attach the mail for you.
hi i also came across this problem and hopefully can give some additional infos. i dont think this "bug" is resolved yet. i tried converting my mails having of old pop times by saving them via "the bat's" export function saving them to mbox files. after your post, that some programs just save the mails "as is" without converting them, i thought i should save some mail coming from an cyrus imap server. so i did this by using the bat, copied this mail to thunderbird version 0.9 (20041103) and tried to move this mail over to another cyrus mail server. this ended with the "Message contains invalid headers" dialog. hope this can be fixed. thx :)
additional information: when you try to put mails, imported from the same mbox files to the imap server with evolution it works very well.
I see the problem frequently when moving messages from my local Sent folder to my Imap sent-messages folder on Mozilla 1.7.1 running on Linux to the Sun ONE / iPlanet Messaging Server V5.2. To get around it I take move each of the messages which won't copy to a local folder, then load the folder into emacs, then save it back out. Whatever line-endings Mozilla and/or the IMAP server are complaining about get worked out when emacs resaves the file.
my guess is that it's unhappy about the received-by header(s). I'll see if I can find a cyrus server to test against...
Product: MailNews → Core
AFAIK this bug is not fixed. I am seeing the "bare newlines" bug on TB 1.0 (20041206). I have observed this bug since the very early releases of TB as well as Mozilla Mail on Linux and Windows. Most recently it occurs when moving messages from one Cyrus IMAP server to another Cyrus IMAP server. I agree with Flo's description that this bug seems to occur more often with certain senders and the offending messages usually include attachment(s). I can move the problematic message(s) to different folders within the original IMAP server, but when I try to move them to a different IMAP server it complains about bare newlines. Some times the same message will not trigger the error and will be moved correctly. Usually I keep trying to move the problematic message(s) for a few days and eventually the move succeeds. The server I'm trying to move mail to is running cyrus-imapd-2.2.8-1 built from an SRPM. I cannot (or at least I don't know how to) get version information from the other IMAP server. I can attach the Cyrus logs (from the server to which I'm moving mail), a sample problematic message, and a tcpdump if necessary.
a sample problematic message would be useful (zipped, perhaps, to save line endings)
I just tried attaching a message I was originally unable to move, but even when zipped it is >300k so I'm prevented from uploading it. However, after ~24 hours and a reboot or two I was able to move said message. Usually I just read my messages from the preview pane, but immediately before the message was successfully moved, I double-clicked the attached message to view it independently and also viewed its source. Maybe this had something to do with it?
(In reply to comment #98) > Usually I just read my messages from the preview pane, but immediately before > the message was successfully moved, I double-clicked the attached message to > view it independently and also viewed its source. Maybe this had something to > do with it? Apologies. The above is less than clear. I will try to rephrase: Immediately before I was able to move the message I opened it by double-clicking on it in my inbox. Once the message window was open I viewed its source. I closed the message window (but left the source window open) and was able to successfully move the message from one IMAP server to the other IMAP server via drag and drop. Perhaps by opening the message the newlines felt they needed to get properly dressed (or a parsing function was activated that cleaned them up).
Eudora (and I think Outlook) import can also cause us to get corrupt messages that can't be copied to a picky imap server. This fixes that - the problem happened when a CRLF pair spanned 4K blocks - we would write a gargage char in between the CRLF because we weren't checking that the block ended in CRLF as opposed to just CR. There are probably bugs out there about Eudora and Outlook messages w/ attachments getting corrupted when imported.
Attachment #209013 - Flags: superreview?(mscott)
Attachment #209013 - Flags: superreview?(mscott) → superreview+
Flags: blocking-thunderbird2+
Comment on attachment 209013 [details] [diff] [review] fix for eudora and outlook import causing these messages approving for the 1.8.1 branch
Attachment #209013 - Flags: approval1.8.1+
Attachment #209013 - Flags: approval1.8.0.2?
the import bug is fixed on the 1.8.1 branch for tb 2.0
Keywords: fixed1.8
Keywords: fixed1.8fixed1.8.1
Keywords: fixed1.8.0.2
Attachment #209013 - Flags: approval1.8.0.2? → approval1.8.0.2+
I am still having this problem when I am trying to transfer a local folder (imported from Eudora) to the IMAP "sent" folder. Initially I was getting the "message contains bare newlines" as originally reported with this bug. Then I copied the local folder to a NEW local folder. The error dialog disappeared but now TBird just ignores the transfer if a troublesome email is included in the bunch I'm trying to copy. I was using 1.5.0.2 but even with 3.0 alpha still get the error (or ignoring the copy if funny email is there).
I dont know if this is part of my problem, I was running 1.5.0.2 of tb and was migrating from local boxes to imap/cyrus. I am getting bare new line errors, so I update to daily build 20060528 I think 3.0a1 and still cant move this msg. Does this fix not resolve bare newlines that are already in the the msg database....
Here is a hex dump from tcpdump of the mesg it is trying to send. http://pastebin.com/744441
* Work Around * I am new to this group, so please forgive me if I sound stupid :-) I too was facing this problem even in 1.5.0.4 (for Solaris/sparc), apparently it was being cased by "^M" characters, and here is how I got over it (of course manually): 1) Stop thunderbird client 2) Go to the Mail files under <HOME>/.thunderbird/<profile>/Mail/Local Folders 3) Perform the following steps on the file WITHOUT *.msf or *.sbd, for example on Inbox: cat Inbox | sed "s/^From Sun/From - Sun/g" |sed "s/^From Tue/From - Tue/g" | sed "s/^From Mon/From - Mon/g" | sed "s/^From Wed/From - Wed/g" | sed "s/^From Thu/From - Thu/g" | sed "s/^From Fri/From - Fri/g" | sed "s/^From Sat/From - Sat/g" | sed "s/>From -/From -/g" | sed "s/^M//g" > Inbox.new This command will also take care of a few other corruptions which occur when you download mail from IMAP to your Local Folders, and later when you try to put them back on IMAP account, it gives errors. 4) Now, move this Inbox.new to Inbox (of course you better take backup of Inbox before doing step 1 itself, in case things to go wrong.). 5) Start Thunderbird again, and start moving the mails, hopefully it'll get you through this time. I hope it helps.
I´m using TB ver. 3.0a2 2008070803. IMAP server Cyrus When I Send message with attachment then copying message to Sent folder fail. Error: The current command did not succeed. The mail server responded : Message contains bare newlines
I was not facing this issue in TB2, but it started appearing when I switched to TB3 Alpha 1 and Alpha 2 (version 3.0a2 (2008072418)) and try to send/reply messages with attachments or other mime data. IMAP server is Cyrus.
Reopening based on comment 108; added qawanted keyword. Assuming this can be reproduced, it should block tb3, since it's dataloss.
Status: RESOLVED → REOPENED
Flags: blocking-thunderbird3?
Keywords: qawanted
Resolution: FIXED → ---
(In reply to comment #109) > Reopening based on comment 108; added qawanted keyword. Assuming this can be > reproduced, it should block tb3, since it's dataloss. > Normally we'd deal with these in new bugs... Anyway think I know the regression, my mistake in bug 429891, taking.
Assignee: bienvenu → bugzilla
Status: REOPENED → NEW
So now I look at it, I don't think bug 429891 is the culprit (as its change would not have caused a regression such as this, and the bug which made a change before that was checked in in 2007), but its possibly not helping. I'm going to deal with my issue in a separate bug. David, if you've got any ideas, please help. At the moment I can't reproduce this locally - but its interesting the two reports we have got both use cyrus IMAP.
Cyrus is the only server I know that really complains about this. I can try with my test account on MoMo's Cyrus server.
Assignee: bugzilla → bienvenu
I tried creating a local message with barenewlines, and copying it to a cyrus imap server worked fine. So at this point, I really need a reproducible test case. I'll try to verify that our Cyrus server complains about this, by disabling the code that fixes barenewlines before appending to an imap folder.
This seems to happen mostly only when replying (to the bare newline messages). After the I get the error popup saying copying to the IMAP Sent folder failed, if I return to the composition window and save the message in a local Drafts folder and then copy manually from local Drafts to the IMAP Sent folder, it copies fine! So can you try to reproduce this by replying to a bare newline message instead of manual copy?
If necessary for any tests you can contact me to get an test account on a working cyrus-imapd-2.2.12-8.1.RHEL4 installation on one of my servers.
I often triggered this bug when copying a bunch of messages from a local folder to an Imap folder - that might be a quick way to try to catch the bug. Note that I haven't seen it in many months (I'm running on a Sun IMAP server).
I think we need to fix this.
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Priority: P3 → P2
see also bug 301010
Severity: major → critical
Keywords: dataloss
QA Contact: karenlhuang → networking.imap
Product: Core → MailNews Core
I wonder if the remaining issue is bug 381472 ? If anyone has a local folder and imap server that shows this problem, and can share it with me, please contact me.
There was a case in which Tb is culprit, Bug 460636(fixed on 2008-10-22. see also Bug 466892). > Bug 460636 nsMsgSaveAsListener sometimes inserts extra LF characters If the extra LF is inserted between consecutive [CRLF]s by Tb himself, "bare newline" issue can occur.
raal, are you seeing this with TB 3.0 b2, or nightly 3.0 trunk builds?
Target Milestone: --- → Thunderbird 3.0rc1
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090331 Lightning/1.0pre Shredder/3.0b3pre This bug does not occur.
ok, thx, I'm going to mark this WFM. If there still exists a reproducible bug in 3.0 trunk builds, let's open a new bug for it.
Status: NEW → RESOLVED
Closed: 21 years ago16 years ago
Resolution: --- → WORKSFORME
(In reply to comment #123) David Bienvenu, "LF without CR" and "NUL character" related issue can still occur, if such data is passed to Tb by POP3/IMAP server, or if such mail data is imported by Tb. >(examples) > Original issue of this bug : "LF without CR" is imported > Bug 301010 : MS Exchange server passes "LF without CR" to Tb > Bug 477799 : NUL is imported from Outlook family's mailbox So, protection from such data is still required. (A) A way to protect Tb from contaminated mail data (== be tolerant with RFC violation by server or other mail client) e.g. Reject contaminated data, replace contaminated data (B) A way to avoid passing contaminated data to server (== avoid RFC violation by Tb) e.g. Remove contaminated data, replace contaminated data Note: When NUL case(Bug 477799), Tb doesn't violate RFC fortunately, because Tb currently stops appending mail data at NUL.
we do try to fix CRLF issues when appending to the imap server. NULL characters are a bit different, but I think much rarer.
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: