306856 - threading does not always work with MIME-encoded subject field

Status: NEW

(MailNews Core :: Backend, defect)

Description

[Ben Hetland]

User-Agent:       Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050728
Build Identifier: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.11) Gecko/20050728

When a reply to a message with MIME-encoded subject is listed in the mail folder
view, the "Re: " prefix (or "RE: " with Outlook, sic) of the subject text
sometimes are placed inside the encoded part in the reply. (I use IMAP with an
Exchange server, and I see this happens when other users with Outlook reply to
mails.) Mozilla's mail client doesn't see this, but simply lists these messages
as two separate threads in threaded view.

Example
-------
Headers in first user's message seen as:

Subject: =?iso-8859-1?Q?F=F8rste_rapport?=
Thread-Topic: =?iso-8859-1?Q?F=F8rste_rapport?=
thread-index: AcWvxH5rfy3wzw42SvejI7GnMKPKRQ==

Another user's answer contains:

Subject: =?iso-8859-1?Q?RE=3A_F=F8rste_rapport?=
Thread-Topic: =?iso-8859-1?Q?F=F8rste_rapport?=
thread-index: AcWvxH5rfy3wzw42SvejI7GnMKPKRQAAZrxg

Result:
In this case, the "RE: " inside the encoded part goes undetected by Mozilla, so
it thinks the messages are not related... (The "Thread-Topic" could have been
used as an additional clue to get the threading right in this case, but I don't
know how standard that header really is.)

Reproducible: Always

Steps to Reproduce:
1. List mail folder in threaded mode
2.
3.




Discovered with Mozilla 1.7.3, still present in 1.7.11 after upgrade.

Comment 1

[Mike Cowperthwaite]

Hm.  I just tested this using TB 1.5rc2, with a few messages in 
netscape.public.test, and I'm not able to reproduce.

Comment 2

[Ben Hetland]

Yes, I see this works OK both with the M1.7.11 that I reported the bug for, and in my TB (older than yours). I don't know if the threading logic is the same for both NNTP and IMAP hosts, but one difference I notice is that your test messages contain a "References" and an "In-Reply-To" field. I don't think Outlook produces those (for mail), at least I don't see them using IMAP with the Exch. server.

Comment 3

[Mike Cowperthwaite]

(In reply to comment #2)
> one difference I notice is that your test
> messages contain a "References" and an "In-Reply-To" field.

No, not all of them.  The fourth message in that thread was composed as a new message, but it still threads.  Note that I had to fudge to apply the 2047-encoding to the Re: string -- when Moz makes its replies (at least with my testing, composing as ISO-8859-1), the encoding doesn't get applied until later in the string.

Normally, Moz threads by the In-Reply-To header, but it falls back on 
threading by subject (xref bug 164115).

Comment 4

[Ben Hetland]

(In reply to comment #3)
> No, not all of them.  The fourth message in that thread was composed as a new
> message, but it still threads.

Please try now, I've added a new series of test messages, the one that says "A reply which is not actuall a reply" in the body (Message-ID: <dpu7f4$8as$[e-mail]>) doesn't thread correctly with my client (TB_1.0.6)

Comment 5

[Mike Cowperthwaite]

(In reply to comment #4)
> (In reply to comment #3)
> > No, not all of them.  The fourth message in that thread was composed as a
> > new message, but it still threads.
> 
> Please try now, I've added a new series of test messages, the one that says
> "A reply which is not actuall a reply" in the body (Message-ID:
> <dpu7f4$8as$[e-mail]>) doesn't thread correctly with my client
> (TB_1.0.6)

I see those five messages in two threads.  The only thing that occurs to me might be causing the issue is that the "RE" in the message that breaks the thread is all uppercase (altho displayed "Re" in the thread pane, it shows as "RE" in the envelope panel).  HOWEVER, you did the same uppercase thing with 
the message you added to my thread, and that threads as I expect.

Reproduced with TB 1.5RC2 (1201) and 1.6a1-0105, Win2K.  Attachment follows.

Comment 6

[Mike Cowperthwaite]

Attachment: mbox file with the messages in question

Simply copy this file into the Local Folders directory of your system, start TB or Moz, and shift to the folder to see the threading as discussed in the previous comments.

Comment 7

[Ben Hetland]

(In reply to comment #5)
>     The only thing that occurs to me
> might be causing the issue is that the "RE" in the message that breaks the
> thread is all uppercase (altho displayed "Re" in the thread pane, it shows as
> "RE" in the envelope panel).

Yes, I did this intentionally, since that is the typical output from (misbehaving) Outlook, and I had a feeling that this could have something to do with the problem.

But another intentional change I did was to remove the : after Olé, leaving only one colon left in the replies. I thought that maybe the double-colons in the subject could hide the bug in the first test.

BTW, also changed "ISO-8859-1" to "iso-8859-1" to comply with my original report. Not that I really thought this should matter, but ...

Comment 8

[buovjaga]

(In reply to Mike Cowperthwaite from comment #6)
> Created attachment 208010 [details]
> mbox file with the messages in question
> 
> Simply copy this file into the Local Folders directory of your system, start
> TB or Moz, and shift to the folder to see the threading as discussed in the
> previous comments.

Still confirmed.

Thunderbird 52.1.0 (32-bit)
Windows 7 64-bit