309965 - RSS entries with long subjects become corrupted when copied into an IMAP folder.

Status: RESOLVED INVALID

(MailNews Core :: Feed Reader, defect)

Description

[David Simmons]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050909 Fedora/1.0.6-1.2.fc4 Firefox/1.0.6
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050720 Thunderbird/1.0.6 Fedora/1.0.6-1.1.fc4

(Note: I also tried this on Thunderbird 1.5 Beta 1, and it seems to have the
same problem.)

There is a bug in Thunderbird that causes the RSS/Atom->RFC822 conversion to
introduce a spurious CTRL-M (0x0D, ASCII CR) into the Subject header when it is
long enough to warrant folding (72 characters).  This CR/LF newline at the
folding point is inconsistent with the LF newlines used elsewhere in the
message.  If you try to copy the resulting message into an IMAP folder, a
malformed header is transmitted to the IMAP server resulting in a corrupt message.

I'm not a Mozilla hacker, but it looks like the fold CR/LF is ultimately
inserted by the generate_encodedwords() function in
mailnews/mime/src/comi18n.cpp.  Perhaps this is the correct behavior, and either
1) the message is supposed to be "cleaned up" elsewhere, or 2) the IMAP
transmission code is not gracefully handling the CR/LF as it perhaps should.

When I manually create a message with a long subject using the Compose window,
there is no spurious CR and the message can be copied into an IMAP folder with
no problems.

I'm using the Dovecot IMAP server, although I've confirmed with a tcpdump that
Thunderbird is sending a corrupt message to the server.  The IMAP 'append'
command is followed immediately by the second line of the two-line (folded)
Subject header and the remainder of the headers.  The first line of the Subject
header and all preceding headers are missing.


Reproducible: Always

Steps to Reproduce:
1.  Subscribe to an RSS or Atom feed with an entry containing a long <title/>
(say, over 72 characters), such as this one:
http://davidsimmons.com/misc/tbbug-atom.xml

2. Transfer the feed entry to an IMAP folder by right-clicking the feed entry,
selecting "Copy To...", "[e-mail]", "myfolder".

3. Select the IMAP folder in the folder pane, and observe the mangled message
that has been added.  Press CTRL-U (view source) to see how the first several
headers are missing.

Actual Results:  
The corrupt message has been appended to the IMAP folder.  The first several
headers, up to and including the first line of the multi-line (folded) Subject:
header, are missing.

Expected Results:  
The message should have been properly appended to the IMAP folder in its
entirety, without corruption.

Comment 1

[David Simmons]

For my own uses, I worked around this bug with the following small change.  I'm
not saying that this is a proper solution, but it does allow me to move
rfc822ified RSS entries into IMAP folders without corruption.

--- FeedItem.js.orig    2005-09-25 17:55:58.000000000 -0600
+++ FeedItem.js 2005-09-25 17:55:17.000000000 -0600
@@ -354,6 +354,9 @@
 
   this.title = this.mimeEncodeSubject(title, this.characterSet);
 
+// SIMMONS-HACK: fix title by removing spurious 0x0D's
+this.title = this.title.replace(/\r/g, "");
+
   // If the date looks like it's in W3C-DTF format, convert it into
   // an IETF standard date.  Otherwise assume it's in IETF format.
   if (this.date.search(/^\d\d\d\d/) != -1)

Comment 2

[Magnus Melin [:mkmelin]]

-> new using
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8b5) Gecko/20051022 Thunderbird/1.5 ID:2005102204

Comment 3

[alta88]

this looks no longer valid as the code replaces \r with a space, reopen if not.  a similar issue for just the right length titles ending in space is patched in bug 547543.