Closed
Bug 83396
Opened 24 years ago
Closed 16 years ago
Alert "message contains bare newlines" displays and lost messages when copying msgs from Local(or other IMAP account)folder to an IMAP account folder
Categories
(MailNews Core :: Networking: IMAP, defect, P2)
Tracking
(Not tracked)
RESOLVED
WORKSFORME
Thunderbird 3.0rc1
People
(Reporter: huang, Assigned: Bienvenu)
References
Details
(5 keywords)
Attachments
(7 files, 2 obsolete files)
6.44 KB,
image/gif
|
Details | |
307.92 KB,
text/plain
|
Details | |
5.43 KB,
text/plain
|
Details | |
15.22 KB,
text/plain
|
Details | |
2.18 KB,
patch
|
Details | Diff | Splinter Review | |
7.41 KB,
text/plain
|
Details | |
1.23 KB,
patch
|
mscott
:
superreview+
mscott
:
approval1.8.0.2+
mscott
:
approval1.8.1+
|
Details | Diff | Splinter Review |
Used 05-30-06-trunk build
Got an abnormal alert when trying to copy local msgs to an IMAP folder
1) Copy about 200 msgs from local to an IMAP folder
2) Actual Results: didn't copy completely and got an abnormal alert: "message
contains bare newlines"(see following attach screen shot)
Expected results: Should copy successfully & completely without this alert
Reporter | ||
Comment 1•24 years ago
|
||
Comment 2•24 years ago
|
||
This again is not reproducible always. cc sheela, who has seen this
come and disappear for the same message.
Reporter | ||
Comment 3•24 years ago
|
||
Nominating nsbeta1 since it seems that many peoples is hitting this problem.....
Keywords: nsbeta1
Reporter | ||
Comment 5•24 years ago
|
||
changing summary as David mentioned on bug 83831 - this only happens on some
messages....
Summary: Got an abnormal alert when trying to copy local msgs to an IMAP folder → Can't copy some messages from local mail to imap folder
Assignee | ||
Comment 6•24 years ago
|
||
so this isn't a true dup of the other bug - the error message I was seeing had
to do with an invalid message header because we're copying the From - envelope
header. This is a similar bug, however, and I've seen it as well on different
messages than the ones that generate the above error message.
Reporter | ||
Comment 7•24 years ago
|
||
Looks like there are different problems occurring for copying local msgs to IMAP
folder now....restoring this summary back based on David's comments.
Summary: Can't copy some messages from local mail to imap folder → Alert "message contains bare newlines" displays when copying local msgs to IMAP folder
Comment 8•23 years ago
|
||
reassigning to naving. It sounds like we need to find the message that causes
this. Sheela or Karen since you've both seen this, can you try finding a message?
Assignee: mscott → naving
Whiteboard: [nsbeta1+]
Updated•23 years ago
|
Priority: -- → P2
Target Milestone: --- → mozilla0.9.2
Reporter | ||
Comment 9•23 years ago
|
||
From 200 msgs??....I need to investigate which one!!....
Reporter | ||
Comment 10•23 years ago
|
||
But, I did have the IMAP log saved at that time when copying the 200 msgs.
I will attach here, hope it help....
Reporter | ||
Comment 11•23 years ago
|
||
Reporter | ||
Comment 13•23 years ago
|
||
It seems that I losted the messages duing copying these 200 messages.
I only have 54 messages left and I am still couldn't finding the specific
message which causing this alert and losting messages. Updating summary to
include losting messages.
Summary: Alert "message contains bare newlines" displays when copying local msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local msgs to IMAP folder
Comment 14•23 years ago
|
||
Try deleting the msf file. They may come back. If you are copying messages, how
can you lose them ?
Reporter | ||
Comment 15•23 years ago
|
||
I used drag/drop for copying these messages, it probably moving the messages
instead of copying messages...I just couldn't find them...but I willl keep
finding them.... :-(
Comment 16•23 years ago
|
||
I cannot find any such message. Also only that message would not be copied but
the rest of messages (multiple copy) should be copied after bug 83831 gets
checked in.
Updated•23 years ago
|
Comment 18•23 years ago
|
||
Does the message generating this alert disappear, causing data loss?
Updated•23 years ago
|
Whiteboard: [nsbeta1+][PDT+] → [nsbeta1+]
Comment 19•23 years ago
|
||
I have been able to reproduce this problem on linux machine with yesterday's
trunk build. I have this message that causes the bare lines alert dialog trying
to move it back to an imap folder.
The following are the steps to produce this problem:
create a new message with a jpg attachment, some text and send it to yourself
After receiving it in the imap account
Drag and drop it to pop account folder
Go to your pop account folder where you moved the message
select that message and click on the file button and try to move it back to an
imap folder
Results in the alert dialog where it fails to move the message back to imap folder
Navin,
If you are not able to reproduce using steps above stop by I can reproduce this
on my machine.
Comment 20•23 years ago
|
||
There is no dataloss. The only bad thing is the messages does not copied.
Comment 21•23 years ago
|
||
Navin,
I deleted the .msf file of the local folder where I was trying to move that
message and relaunched the application and tried to move again and resulted with
the same error still. So deleting .msf file did not do any good to get rid of
the problem.
Comment 22•23 years ago
|
||
That's right, I was not able to just move the message from the local folder back
to an imap folder.
Updated•23 years ago
|
Target Milestone: mozilla0.9.3 → mozilla0.9.4
Comment 23•23 years ago
|
||
Mail news triage meeting --> .9.5
Target Milestone: mozilla0.9.4 → mozilla0.9.5
Reporter | ||
Comment 24•23 years ago
|
||
I can reproduce this probelm by using the attach msg as following.
Reporter | ||
Comment 25•23 years ago
|
||
Comment 27•23 years ago
|
||
moving to 0.9.7
Whiteboard: [nsbeta1+]
Target Milestone: mozilla0.9.6 → mozilla0.9.7
Comment 28•23 years ago
|
||
I migrated several users from POP to IMAP accounts and noticed that copying from
local to IMAP failed on two cases:
- message had only headers and no body. Didn't check was there any empty lines
or was the message body just empty. I guess that the error message was "message
contains bare newlines"
- message header had first line as "From - ..." used as e-mail separator at some
mailboxes. There was another error thrown out, unfortunately I don't remember
exact wording.
I assumed that this was Cyrus IMAP server issue, but since you are talking about
it here....
Updated•23 years ago
|
Priority: P2 → P3
Updated•23 years ago
|
Reporter | ||
Comment 30•23 years ago
|
||
*** Bug 142620 has been marked as a duplicate of this bug. ***
Reporter | ||
Comment 31•23 years ago
|
||
Olga seeing this problem when copying messages from other IMAP account to NMS
6.0 (tintin.mcom.com) account. So, this seems is also occurring for copying IMAP
account messages to IMAP account messages.....
Reporter | ||
Updated•23 years ago
|
Summary: Alert "message contains bare newlines" displays and lost messages when copying local msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local or IMAP msgs to IMAP folder
Reporter | ||
Updated•23 years ago
|
Summary: Alert "message contains bare newlines" displays and lost messages when copying local or IMAP msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying local or IMAP folder msgs to IMAP folder
Comment 32•23 years ago
|
||
It was Web Mail Account.
Reporter | ||
Comment 33•23 years ago
|
||
Olga, did you copy from Webmail IMAP to NMS 6.0 (tintin.mcom.com)IMAP?
Can you attach that problem message here? Thanks.
Reporter | ||
Comment 34•23 years ago
|
||
Olga, did you lost this message during the copy?
We may need to nominating nsbeta1 again if msgs have been lost during the
copy......
Summary: Alert "message contains bare newlines" displays and lost messages when copying local or IMAP folder msgs to IMAP folder → Alert "message contains bare newlines" displays and lost messages when copying msgs from Local(or other IMAP account)folder to an IMAP account folder
Comment 35•23 years ago
|
||
This morning after re-starting NO problem with copy/move neither from Web Mail
nor from another POP, IMAP. Yesterday I saw the same alert mentioned in comment #1.
Comment 36•22 years ago
|
||
I can now get this 100% when trying to move a message that contains text and a
attached jpg image from a Exchange IMAP server to a Cyrus IMAP server...:(
Reporter | ||
Comment 37•22 years ago
|
||
*** Bug 159323 has been marked as a duplicate of this bug. ***
Comment 39•22 years ago
|
||
In particular, I'm copying from a UW IMAP server to a Netscape Mail Server.
Assignee | ||
Comment 40•22 years ago
|
||
cross-imap server move/copy copies the msg down to a local tmp folder and then
up to the destination imap server. The problem is that some servers (like our
NMS server) are very picky about the format of the messages they allow to be
appended to a folder.
Comment 41•22 years ago
|
||
I have the same problem with mozilla (1.0 and 1.1) on linux when trying to move
some local mails to my Cyrus IMAPD server. I hope to see a quick fix rsn so that
mozilla skips the mails with bare newlines and continues the copy/move operation.
Is ist now clear if mozilla has problems or if the imap servers have problems?
Reading this thread I'm not sure about that.
Comment 42•22 years ago
|
||
*** Bug 133377 has been marked as a duplicate of this bug. ***
Comment 43•22 years ago
|
||
Check
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=5629
for a statement about this bug/feature/enforcement in Cyrus IMAPD.
Comment 45•21 years ago
|
||
I try to set up a Cyrus server, and the problem is still there in mozilla 1.5
Linux. A google search tend to say that it's a problem with mozilla and not Cyrus.
The problem is always reproductible when I create a mail with big attachment, I
save it, and I try to move it to the Imap server.
Hope it will be solve soon...
Assignee | ||
Comment 46•21 years ago
|
||
Is this problem more easily reproducible on Linux?
Assignee: sspitzer → bienvenu
Comment 47•21 years ago
|
||
I used to see this from time to time on Linux, and I don't think I've ever seen
it on Windows, so I suspect it may in fact be platform-specific.
Comment 48•21 years ago
|
||
It happened to me in Windows when I copied a large number of messages from one
IMAP folder to another folder on the same server. This was back in the 1.3 days
though, now I usually just use a web interface for my IMAP provider to move a
large number of messages.
Assignee | ||
Comment 49•21 years ago
|
||
from one imap folder to another on the same server does not involve the client,
or at least its line endings - it's simply a copy command sent to the server,
just like the web client would do.
Comment 50•21 years ago
|
||
My mistake, you are correct. The message copies I made were from one IMAP
server to another, MyRealBox -> Fastmail I believe. I remember being frustrated
with the "bare newlines" error and came across a "Migrate IMAP" feature within
Fastmail's website and used that instead of Mozilla.
Comment 51•21 years ago
|
||
I would like to add that I'm seeing this bug in Mozilla 1.5 and Thunderbird 0.7
on a windows XP home ed machine when moving mails from the local folders to an
Cyrus-IMAPD account. Any Ideas?
Tarjei
Comment 52•21 years ago
|
||
... this bug is around for a really long time and annoying quite some people.
migrating from an archive of local pop-messages to an imap solution is not so
uncommon today, i guess. however, nobody seems to have even looked at any code
that could cause this problem...
can we increase the priority of this an get somebody to take a look at it? imho
this is one of the most visible mail bugs!
Comment 53•21 years ago
|
||
I fear that's the classic problem that the person knowing the code can't
reproduce the problem and vice versa.
David, would a tcpdump log help you more than a Mozilla log?
Severity: normal → major
Target Milestone: mozilla1.2alpha → ---
Comment 54•21 years ago
|
||
sorry for being a bit nervous in my last post... i did some research on this,
with the following results:
1.) AFAIK, this problem happens only with the Cyrus IMAP server. The Cyrus folks
follow a zero-tolerance policy concerning RFC violations, and reject malformed
mails submitted with the IMAP APPEND command.
2.) The actual bug is found in the e-mail clients sending non-RFC-compliant
mails (e.g. Outlook up to version 5). If such a mail is received by mozilla
through a POP connection, it is stored on disk (as is) and displayed correctly.
If such a mail is received by Cyrus IMAP by a local sendmail daemon, it is
(AFAIK) corrected and stored on disk in the correct format.
3.) The problem occurs when you try to move such a mail stored by mozilla
locally to the Cyrus IMAP server - mozilla delivers the mail in the original
(non-compliant) format, and Cyrus thinks it is talking to a non-compliant client
and rejects the mail.
the problem is that we have a deadlock situation here - IMHO mozilla is right to
store the mails in the original form, and cyrus is right to reject
non-RFC-compliant mails. and it makes no sense to blame "the other" clients,
since even if they fix this problem in future releases, this won't change the
messages already received by mozilla.
in addition, since sendmail usually corrects malformed mails, this problem is
not very visible to end-users of such buggy email clients - usually their
messages are delivered without problems (even to Cyrus). this problem only
occurs when such a malformed message is "replayed" by mozilla.
the possible fixes to this problems that i can see are:
1.) talk to the cyrus folks about theirt policy. i am not sure if they are aware
of this scenario when they argue in favour of their zero-tolerance policy.
2.) make mozilla correct the stored emails when sending to an IMAP server. this
is possibly not trivial(?). the three main causes for these kinds of problems
are: LF without CR in the mail body, NUL characters in the mail body, or
malformed headers. (the relevant RFCs are RFC821 (SMTP), RFC2060 (IMAP4), and
RFC1939 (POP3), i guess)
3.) offer end-users a link to an external program, that is able to fix their
mailboxes when needed. unfortunately, i haven't found such a software by now...
or build a hack into mozilla, to either re-send these mails via SMTP to the
account (with the original date), or something like that (probably least
preferred by all)
fact is, that i and other people are completely stuck in the process of
migrating our mail archives to IMAP. some people just deleted the messages, but
i have a large archive reaching back to 1996 and i would loose 1/3rd of my
historic messages - completely unacceptable for me.
if anyone can think of a hack to allow me to move my mails, i would be very
happy for now.
Comment 55•21 years ago
|
||
p.s. the most relevant post from the cyrus mailing list concerning this issue:
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=5639
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=5641
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=7038
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=21465
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=21835
http://asg.web.cmu.edu/archive/message.php?mailbox=archive.info-cyrus&msg=21877
Comment 56•21 years ago
|
||
Flo, to follow strictly the RFC is good in my opinion. I know about the "be
tolerant when reading/recieving" theory, but I don't agree with it.
The problem that made me look into this bug can indeed be caused by imported
Outlook mails. But 1. this would mean a bug in the import code, and 2. doesn't
absolve the code that writes to IMAP from responsibility.
To this mails on Unix/Linux are also stored with LF only line endings in the
mbox (and CR only on MAC).
In result 2.) you don't write about importet mails from non-RFC-compliant
clients but received through POP.
That confuses me - do you know a mail with LF is written with LF (on
not-Unix-systems) or do you just think so?
AFAIK the code in RetrResponse() and ReadNextLine(), line endings are stripped
away uppon receiving a line and the gets a new one depending on the platform. So
we end up with mboxes with different line endings on different operating systems.
I think your conclusion 2.) is the right one.
David, don't we really convert the line endings when appending to IMAP?
Comment 57•21 years ago
|
||
FYI: i am sitting on a windows XP box.
and now that you mention it: i couldn't actually find the LF characters that
Cyrus complains about in my mbox files... that was my first try, to fix the mbox
files manually. so maybe it is really just a bug in the mozilla IMAP code. but
sorry i can't help you here, i am not a mozilla hacker (and don't currently plan
to become one, due to time constraints)
the reason why i thought it is the original source of the mail that couses the
troubles, is, that this bug occurs only for some mails. and for me, there are
some senders (using for example outlook) that are more frequently the source of
such mails than others. but, you are right, this is just speculation...
of course strictness with respect to RFCs is good. i just thought that this was
a scenario that the RFCs didn't fully consider.
Assignee | ||
Comment 58•21 years ago
|
||
Flo, that was a very accurate summary. This problem also happens with the
Netscape IMAP server, since it was developed by the same person as Cyrus.
We could fix the imap append code to repair naked newlines, and we could replace
embedded nulls with spaces - malformed headers are a bit trickier.
I'll try to look into it today.
Comment 59•21 years ago
|
||
just one more comment: it would be great if mozilla would, if you move messages,
at least delete those messages that have been moved succesfully. at the moment,
if you move multiple messages at once, and the error occurs in one message, the
messages that have been copied successfully are not deleted from the original
folder, resulting in duplicate messages that are very hard to sort out. one has
no clue which messages have been copied and which not, and which message caused
the error. this is also what the reporter of this bug pointed out, i think.
or maybe open a new bug "remove successfully moved messages from original folder
even if an error occurs"? i haven't looked if this happens in other cases too...
Comment 60•21 years ago
|
||
Flo, the duplicate mail problem is indeed annoying. Shouldn't it be possible to
use formail to remove the duplicate mails from the local folder?
Comment 61•21 years ago
|
||
unfortunately, the duplicate mails are now on my imap server. is it possible to
use formail on the mailboxes on the server? i could theoretically have access to
the server to fix my mails.
could formail be used to fix invalid headers and the other problems mentioned
above? i have never used this tool...
Comment 62•21 years ago
|
||
The formail command is part of the procmail package found on most Linux systems.
It can split mailboxfiles into seperate mails and can also manipulate singe mails.
But it may break something if you directly let it work on cyrus' mail files.
Even when you turn cyrus off. May be the db files of cyrus get out of sync.
Surely they will, if you delete some mails with a simple rm.
On the other hand formail is perfect for header manipulation. Check man formail.
Assignee | ||
Comment 63•21 years ago
|
||
one tricky thing about fixing this is that we need to fix it when we create the
temp file who's contents we append to the imap folder, because the append
command requires us to specify the message size up front, and we use the size of
the temp file for that. Thus, we can't be inserting extra cr's or lf's to fix
bare newlines at append time; we have to do it when we generate the temp file.
Comment 64•21 years ago
|
||
But that alone wouldn't be so hard. Or is it that this temp file generation is
also used for other purposes where this conversion would be wrong?
Assignee | ||
Comment 65•21 years ago
|
||
yes, I think the code is used for other purposes - I'll probably need a flag
that says to sanitize the copied message.
Assignee | ||
Comment 66•21 years ago
|
||
actually, the code looks to me like it handles this. Apparently it doesn't work,
but when creating the temp file, it definitely scans for a CR or LF when parsing
for lines, and writes a CRLF instead of a single CR or LF terminator. I'll have
to try the test case.
Assignee | ||
Comment 67•21 years ago
|
||
the code was only figuring out the linebreak length once, which is wrong - it
needs to figure it out for every line. This causes a slight amount of code
duplication, but I couldn't figure out an easy way to just have one instance of
the code that determines the linebreak length
Assignee | ||
Updated•21 years ago
|
Attachment #145760 -
Flags: superreview?(mscott)
Updated•21 years ago
|
Attachment #145760 -
Flags: superreview?(mscott) → superreview+
Assignee | ||
Comment 68•21 years ago
|
||
fix checked in.
Status: NEW → RESOLVED
Closed: 21 years ago
Resolution: --- → FIXED
Comment 69•21 years ago
|
||
i had only now time to test this with a current nightly.. and unfortunately, i
have to say, the bug seems to be not fully fixed.
i was able to move approx. 2/3rds of the massages that would trigger the error
message to my IMAP server - however, there are still messages remaining that
trigger the error as before.
nearly all of these messages have large attachments >200k in size. there is only
a single message that triggers the error for me that is does not have an
attachment, but is a 15k multipart html message. i will attach the source of
this message to the bug.
sorry for the bad news, otherwise thanks and respect for the quick help!
using Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8a) Gecko/20040421
Comment 70•21 years ago
|
||
see previous comment. this is an example for a message still not working for
me.
Assignee | ||
Comment 71•21 years ago
|
||
sorry it's not completely fixed for you. I don't think that test case helps me,
because the line endings are correct in it, and I can copy it to a cyrus server
w/o a problem. Would it be possible for you to save off the actual message into
a text file from your mailbox, using a text editor that maintains the line
endings, and e-mail that to me? And maybe one of the other messages with the big
attachment as well? thx...
Status: RESOLVED → REOPENED
Resolution: FIXED → ---
Assignee | ||
Comment 72•21 years ago
|
||
after a little debugging, I think the original message is OK, but we're messing
up the line ending in the process of saving the message to the temp file, prior
to uploading the message. When data is streamed to us, the stream might break on
a CRLF boundary, and in that case, we'll write an extra blank line, I think...
Assignee | ||
Comment 73•21 years ago
|
||
handle case where crlf are split by block boundary. Thx very much for the test
cases, Flo.
Assignee | ||
Updated•21 years ago
|
Attachment #146879 -
Flags: superreview?(mscott)
Updated•21 years ago
|
Attachment #146879 -
Flags: superreview?(mscott) → superreview+
Assignee | ||
Comment 74•21 years ago
|
||
fixed
Status: REOPENED → RESOLVED
Closed: 21 years ago → 21 years ago
Resolution: --- → FIXED
Comment 75•21 years ago
|
||
Sounds great! Will the fix be in release 1.7?
Comment 76•21 years ago
|
||
amazing! tried it out in the current nightly, and everything works! thanks a LOT
for fixing this so fast.
there are two bugs that should be spun off from this one: invalid headers in
mail messages (same problem as with this bug, but probably harder to fix as
indicated in comment #58), and the general problem that if a problem occurs
during moving of multiple mails, you end up with duplicate mails since the mails
that have been copied successfully are not removed from the source folder.
how should we proceed with these two bugs? i didn't query yet if they are
already there...
Assignee | ||
Comment 77•21 years ago
|
||
Comment on attachment 146879 [details] [diff] [review]
proposed fix
this change affects the copying of local messages to imap folders (or messages
from other imap servers) - it actually fixes a data corruption problem, so it's
a good one to get in. Usually, an extra blank line doesn't cause any harm, but
it can be a bad thing...
Attachment #146879 -
Flags: approval1.7?
Comment 78•21 years ago
|
||
Comment on attachment 146879 [details] [diff] [review]
proposed fix
a=mkaply for 1.7
Attachment #146879 -
Flags: approval1.7? → approval1.7+
Assignee | ||
Comment 79•21 years ago
|
||
Attachment #145760 -
Attachment is obsolete: true
Attachment #146879 -
Attachment is obsolete: true
Assignee | ||
Updated•21 years ago
|
Comment 80•21 years ago
|
||
FYI, David fixed this on the tbird 0.6 branch too
Comment 81•21 years ago
|
||
A user I've contact to still experiences this problem with TB 2004-04-28-00-trunk.
I've a log here that shows this and also the mail that generates this.
Unfortunately I can't reproduce the problem with my own Cyrus IMAPd and the
messages, so it's difficult to help here.
David, do you think there's any chance to find another bug in the move-code
without being able to reproduce it yourself? I could supply you with log and
mails too.
Comment 82•21 years ago
|
||
FYI: unfortunately i cannot assist you with this bug anymore, because my
organisation has migrated to a different imap server. for me everything worked
fine, but i was told only afterwards that the server software had changed ;)
Comment 83•21 years ago
|
||
Erm, you mean your comment #76 "everything works" was made after testing on the
new server (without your knowledge)? So the bug you discovered might also still
be active.
Assignee | ||
Comment 84•21 years ago
|
||
Christian, sure, send me the mail and the logs
Comment 85•21 years ago
|
||
i _think_ i verified the bug while the cyrus server was still active. but i am
not sure. sorry about this, i only was told about the switch two days ago, but
it happened earlier...
Assignee | ||
Comment 86•21 years ago
|
||
I'm reasonably confident that the last fix actually fixed your problem, at least
for the sample messages you sent me. But I'll have a better idea when I get the
mail messages from Christian.
Comment 87•20 years ago
|
||
By using the original msg that I attached on #25:
http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25
I am not able to reproduce this problem when copy the
Verified as fix
QA Contact: grylchan → karenlhuang
Comment 88•20 years ago
|
||
By using the original msg that I attached on #25:
http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25
I am not able to reproduce this problem anymore when copy this message from
Local to IMAP on latest 1.7 branch 06-24 build.
Changing keywords from fixed1.7 to verified1.7.
Leave this bug status "as is" until this bug be verified on trunk again...
Keywords: fixed1.7 → verified1.7
Comment 89•20 years ago
|
||
By using the original msg that I attached on #25:
http://bugzilla.mozilla.org/show_bug.cgi?id=83396#c25
I am not able to reproduce this problem anymore when copy this message from
Local to IMAP on latest 1.7 branch 06-24 build.
Changing keywords from fixed1.7 to verified1.7.
Leave this bug status "as is" until this bug be verified on trunk again...
Comment 90•20 years ago
|
||
Ok, I upraded to the mozilla-1.7-0.3.2 rpm from the FC2 devel-branch. Copying
mails from local to cyrus-imapd that previously triggered this bare lines error
seems to be better now. But I came across an other problem, mozilla refuses to
copy a certain email with this error message:
current command not successful: mail-server responded: message contains
invalid header.
I translated this back from the german locale to english. I'm going to attach
the mail for you.
Comment 91•20 years ago
|
||
Comment 92•20 years ago
|
||
hi
i also came across this problem and hopefully can give some additional infos.
i dont think this "bug" is resolved yet.
i tried converting my mails having of old pop times by saving them via "the
bat's" export function saving them to mbox files.
after your post, that some programs just save the mails "as is" without
converting them, i thought i should save some mail coming from an cyrus imap
server.
so i did this by using the bat, copied this mail to thunderbird version 0.9
(20041103) and tried to move this mail over to another cyrus mail server.
this ended with the "Message contains invalid headers" dialog.
hope this can be fixed. thx :)
Comment 93•20 years ago
|
||
additional information:
when you try to put mails, imported from the same mbox files to the imap server
with evolution it works very well.
Comment 94•20 years ago
|
||
I see the problem frequently when moving messages from my local Sent folder to
my Imap sent-messages folder on Mozilla 1.7.1 running on Linux to the Sun ONE /
iPlanet Messaging Server V5.2. To get around it I take move each of the
messages which won't copy to a local folder, then load the folder into emacs,
then save it back out. Whatever line-endings Mozilla and/or the IMAP server are
complaining about get worked out when emacs resaves the file.
Assignee | ||
Comment 95•20 years ago
|
||
my guess is that it's unhappy about the received-by header(s). I'll see if I can
find a cyrus server to test against...
Updated•20 years ago
|
Product: MailNews → Core
Comment 96•20 years ago
|
||
AFAIK this bug is not fixed.
I am seeing the "bare newlines" bug on TB 1.0 (20041206). I have observed this
bug since the very early releases of TB as well as Mozilla Mail on Linux and
Windows. Most recently it occurs when moving messages from one Cyrus IMAP
server to another Cyrus IMAP server.
I agree with Flo's description that this bug seems to occur more often with
certain senders and the offending messages usually include attachment(s).
I can move the problematic message(s) to different folders within the original
IMAP server, but when I try to move them to a different IMAP server it complains
about bare newlines.
Some times the same message will not trigger the error and will be moved
correctly. Usually I keep trying to move the problematic message(s) for a few
days and eventually the move succeeds.
The server I'm trying to move mail to is running cyrus-imapd-2.2.8-1 built from
an SRPM. I cannot (or at least I don't know how to) get version information
from the other IMAP server.
I can attach the Cyrus logs (from the server to which I'm moving mail), a sample
problematic message, and a tcpdump if necessary.
Assignee | ||
Comment 97•20 years ago
|
||
a sample problematic message would be useful (zipped, perhaps, to save line endings)
Comment 98•20 years ago
|
||
I just tried attaching a message I was originally unable to move, but even when
zipped it is >300k so I'm prevented from uploading it.
However, after ~24 hours and a reboot or two I was able to move said message.
Usually I just read my messages from the preview pane, but immediately before
the message was successfully moved, I double-clicked the attached message to
view it independently and also viewed its source. Maybe this had something to
do with it?
Comment 99•20 years ago
|
||
(In reply to comment #98)
> Usually I just read my messages from the preview pane, but immediately before
> the message was successfully moved, I double-clicked the attached message to
> view it independently and also viewed its source. Maybe this had something to
> do with it?
Apologies. The above is less than clear. I will try to rephrase:
Immediately before I was able to move the message I opened it by double-clicking
on it in my inbox. Once the message window was open I viewed its source. I
closed the message window (but left the source window open) and was able to
successfully move the message from one IMAP server to the other IMAP server via
drag and drop.
Perhaps by opening the message the newlines felt they needed to get properly
dressed (or a parsing function was activated that cleaned them up).
Assignee | ||
Comment 100•19 years ago
|
||
Eudora (and I think Outlook) import can also cause us to get corrupt messages that can't be copied to a picky imap server. This fixes that - the problem happened when a CRLF pair spanned 4K blocks - we would write a gargage char in between the CRLF because we weren't checking that the block ended in CRLF as opposed to just CR.
There are probably bugs out there about Eudora and Outlook messages w/ attachments getting corrupted when imported.
Attachment #209013 -
Flags: superreview?(mscott)
Updated•19 years ago
|
Attachment #209013 -
Flags: superreview?(mscott) → superreview+
Updated•19 years ago
|
Flags: blocking-thunderbird2+
Comment 101•19 years ago
|
||
Comment on attachment 209013 [details] [diff] [review]
fix for eudora and outlook import causing these messages
approving for the 1.8.1 branch
Attachment #209013 -
Flags: approval1.8.1+
Attachment #209013 -
Flags: approval1.8.0.2?
Assignee | ||
Comment 102•19 years ago
|
||
the import bug is fixed on the 1.8.1 branch for tb 2.0
Keywords: fixed1.8
Assignee | ||
Updated•19 years ago
|
Keywords: fixed1.8 → fixed1.8.1
Updated•19 years ago
|
Keywords: fixed1.8.0.2
Updated•19 years ago
|
Attachment #209013 -
Flags: approval1.8.0.2? → approval1.8.0.2+
Comment 103•19 years ago
|
||
I am still having this problem when I am trying to transfer a local folder (imported from Eudora) to the IMAP "sent" folder.
Initially I was getting the "message contains bare newlines" as originally reported with this bug. Then I copied the local folder to a NEW local folder. The error dialog disappeared but now TBird just ignores the transfer if a troublesome email is included in the bunch I'm trying to copy.
I was using 1.5.0.2 but even with 3.0 alpha still get the error (or ignoring the copy if funny email is there).
Comment 104•19 years ago
|
||
I dont know if this is part of my problem, I was running 1.5.0.2 of tb and was migrating from local boxes to imap/cyrus. I am getting bare new line errors, so I update to daily build 20060528 I think 3.0a1 and still cant move this msg. Does this fix not resolve bare newlines that are already in the the msg database....
Comment 105•19 years ago
|
||
Here is a hex dump from tcpdump of the mesg it is trying to send. http://pastebin.com/744441
Comment 106•18 years ago
|
||
* Work Around *
I am new to this group, so please forgive me if I sound stupid :-)
I too was facing this problem even in 1.5.0.4 (for Solaris/sparc), apparently it was being cased by "^M" characters, and here is how I got over it (of course manually):
1) Stop thunderbird client
2) Go to the Mail files under <HOME>/.thunderbird/<profile>/Mail/Local Folders
3) Perform the following steps on the file WITHOUT *.msf or *.sbd, for example on Inbox:
cat Inbox | sed "s/^From Sun/From - Sun/g" |sed "s/^From Tue/From - Tue/g" | sed "s/^From Mon/From - Mon/g" | sed "s/^From Wed/From - Wed/g" | sed "s/^From Thu/From - Thu/g" | sed "s/^From Fri/From - Fri/g" | sed "s/^From Sat/From - Sat/g" | sed "s/>From -/From -/g" | sed "s/^M//g" > Inbox.new
This command will also take care of a few other corruptions which occur when you download mail from IMAP to your Local Folders, and later when you try to put them back on IMAP account, it gives errors.
4) Now, move this Inbox.new to Inbox (of course you better take backup of Inbox before doing step 1 itself, in case things to go wrong.).
5) Start Thunderbird again, and start moving the mails, hopefully it'll get you through this time.
I hope it helps.
Comment 107•16 years ago
|
||
I´m using TB ver. 3.0a2 2008070803.
IMAP server Cyrus
When I Send message with attachment then copying message to Sent folder fail.
Error: The current command did not succeed. The mail server responded : Message contains bare newlines
Comment 108•16 years ago
|
||
I was not facing this issue in TB2, but it started appearing when I switched to TB3 Alpha 1 and Alpha 2 (version 3.0a2 (2008072418)) and try to send/reply messages with attachments or other mime data.
IMAP server is Cyrus.
Comment 109•16 years ago
|
||
Reopening based on comment 108; added qawanted keyword. Assuming this can be reproduced, it should block tb3, since it's dataloss.
Comment 110•16 years ago
|
||
(In reply to comment #109)
> Reopening based on comment 108; added qawanted keyword. Assuming this can be
> reproduced, it should block tb3, since it's dataloss.
>
Normally we'd deal with these in new bugs...
Anyway think I know the regression, my mistake in bug 429891, taking.
Assignee: bienvenu → bugzilla
Status: REOPENED → NEW
Comment 111•16 years ago
|
||
So now I look at it, I don't think bug 429891 is the culprit (as its change would not have caused a regression such as this, and the bug which made a change before that was checked in in 2007), but its possibly not helping.
I'm going to deal with my issue in a separate bug.
David, if you've got any ideas, please help. At the moment I can't reproduce this locally - but its interesting the two reports we have got both use cyrus IMAP.
Assignee | ||
Comment 112•16 years ago
|
||
Cyrus is the only server I know that really complains about this. I can try with my test account on MoMo's Cyrus server.
Assignee: bugzilla → bienvenu
Assignee | ||
Comment 113•16 years ago
|
||
I tried creating a local message with barenewlines, and copying it to a cyrus imap server worked fine. So at this point, I really need a reproducible test case. I'll try to verify that our Cyrus server complains about this, by disabling the code that fixes barenewlines before appending to an imap folder.
Comment 114•16 years ago
|
||
This seems to happen mostly only when replying (to the bare newline messages). After the I get the error popup saying copying to the IMAP Sent folder failed, if I return to the composition window and save the message in a local Drafts folder and then copy manually from local Drafts to the IMAP Sent folder, it copies fine!
So can you try to reproduce this by replying to a bare newline message instead of manual copy?
Comment 115•16 years ago
|
||
If necessary for any tests you can contact me to get an test account on a working cyrus-imapd-2.2.12-8.1.RHEL4 installation on one of my servers.
Comment 116•16 years ago
|
||
I often triggered this bug when copying a bunch of messages from a local folder to an Imap folder - that might be a quick way to try to catch the bug. Note that I haven't seen it in many months (I'm running on a Sun IMAP server).
Comment 117•16 years ago
|
||
I think we need to fix this.
Flags: blocking-thunderbird3? → blocking-thunderbird3+
Priority: P3 → P2
Comment 118•16 years ago
|
||
see also bug 301010
Updated•16 years ago
|
Product: Core → MailNews Core
Assignee | ||
Comment 119•16 years ago
|
||
I wonder if the remaining issue is bug 381472 ? If anyone has a local folder and imap server that shows this problem, and can share it with me, please contact me.
Comment 120•16 years ago
|
||
There was a case in which Tb is culprit, Bug 460636(fixed on 2008-10-22. see also Bug 466892).
> Bug 460636 nsMsgSaveAsListener sometimes inserts extra LF characters
If the extra LF is inserted between consecutive [CRLF]s by Tb himself, "bare newline" issue can occur.
Assignee | ||
Comment 121•16 years ago
|
||
raal, are you seeing this with TB 3.0 b2, or nightly 3.0 trunk builds?
Target Milestone: --- → Thunderbird 3.0rc1
Comment 122•16 years ago
|
||
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1b4pre) Gecko/20090331 Lightning/1.0pre Shredder/3.0b3pre
This bug does not occur.
Assignee | ||
Comment 123•16 years ago
|
||
ok, thx, I'm going to mark this WFM. If there still exists a reproducible bug in 3.0 trunk builds, let's open a new bug for it.
Status: NEW → RESOLVED
Closed: 21 years ago → 16 years ago
Resolution: --- → WORKSFORME
Comment 124•16 years ago
|
||
(In reply to comment #123)
David Bienvenu, "LF without CR" and "NUL character" related issue can still occur, if such data is passed to Tb by POP3/IMAP server, or if such mail data is imported by Tb.
>(examples)
> Original issue of this bug : "LF without CR" is imported
> Bug 301010 : MS Exchange server passes "LF without CR" to Tb
> Bug 477799 : NUL is imported from Outlook family's mailbox
So, protection from such data is still required.
(A) A way to protect Tb from contaminated mail data
(== be tolerant with RFC violation by server or other mail client)
e.g. Reject contaminated data, replace contaminated data
(B) A way to avoid passing contaminated data to server
(== avoid RFC violation by Tb)
e.g. Remove contaminated data, replace contaminated data
Note:
When NUL case(Bug 477799), Tb doesn't violate RFC fortunately, because Tb currently stops appending mail data at NUL.
Assignee | ||
Comment 125•16 years ago
|
||
we do try to fix CRLF issues when appending to the imap server. NULL characters are a bit different, but I think much rarer.
You need to log in
before you can comment on or make changes to this bug.
Description
•