310189 - Multiple "Subject:" lines defeat filtering (mailformed mail. problem on any header defined as max number=1, for example From:. first header should be used)

Status: RESOLVED FIXED

(Thunderbird :: Mail Window Front End, defect)

Description

[Eric E. Johnson]

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.10) Gecko/20050916 Firefox/1.0.6
Build Identifier: version 1.0.6 (20050725) - (Gentoo Linux mozilla-thunderbird 1.0.6-r2)

My company has a spam filter installed that marks all incoming email that it
thinks is spam with a "[SPAM]" prefix on the subject.  Naturally, I put a filter
in Thunderbird so that it moves all of the email with "[SPAM]" in the subject. 
I was surprised, then, to open an email (two pane view, double-clicked message
to open in a window) from my inbox this morning, and see that the subject line
read "[SPAM]", when the listing in my inbox did not show that subject line.

Two issues here:
* Multiple subject lines displayed inconsistently.  Used the first one in the
separate window, and the second one in the inbox listing.
* Mail filtering based on subject only looked at one of the subject headers.  It
probably needs to look at both.


Reproducible: Always

Steps to Reproduce:
Part of the source fo the email.

Received: (qmail 4273 invoked from network); Tue, 27 Sep 2005 04:17:17 -0300
Message-ID: <5210352242.44427.09853.sendUpdate@pamper>
XDSB-Id: 8353095::22406713::PlaxoUpdateRequest
Subject: [SPAM] Disc0unted S0ftware
Mime-Version: 1.0
From: "Jasper House" <[e-mail]>
Subject: Disc0unted S0ftware
Date: Tue, 27 Sep 2005 04:17:17 -0300
X-Virus-Scanned: by amavisd-new at amelia
Content-Type: multipart/related;
Return-Path: [e-mail]


Actual Results:  
In the inbox listing, this shows "Disc0unted S0ftware".

In the individual mail window, it shows as "[SPAM] Disc0unted S0ftware".

My subject based filter looking for "[SPAM]" to shunt the email to my spam
folder did not redirect the email.


This looks like it is related to bug 172104 reported against the Mozilla suite.

Comment 1

[Eric E. Johnson]

Updated the version number...

Comment 2

[Ali Ebrahim]

I see this bug too on Windows. Confirming bug.

Comment 3

[Mike Cowperthwaite]

Seamonkey bug 172104; this is probably a backend bug.

Comment 5

[Magnus Melin [:mkmelin]]

Since bug 321887 now got duped here, note that it doesn't have to be the full header. Su: xx will be taken as Subject: xx, Da: yy as Date: yy etc  - at least in the thread pane.

Comment 6

[Tobias]

Please fix this ugly bug, please!

i get following header, which does not work for K9-filtering with thunderbird:

From - Thu Sep 23 13:19:20 2010
X-Account-Key: account7
X-UIDL: 0Lk22x-1OMvx240B0-00bWti
X-Mozilla-Status: 0001
X-Mozilla-Status2: 00000000
X-Mozilla-Keys:                                                                                 
Return-Path: <[e-mail]>
Delivered-To: GMX delivery to [e-mail]
Received: (qmail invoked by alias); 23 Sep 2010 11:13:45 -0000
Received: from mail.casavekhatib.com (EHLO mail.casavekhatib.com) [206.217.137.220]
  by mx0.gmx.net (mx024) with SMTP; 23 Sep 2010 13:13:45 +0200
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; s=default; d=casavekhatib.com;
 h=To:Message-ID:Date:From:Mime-Version:Subject:Content-Type:Content-Transfer-Encoding; i=[e-mail];
 bh=9A5s5s52qxGMXAiqwmEhTv3kERM=;
 b=SDxLQfTRUgKZg6bf2MX0Ql2nMf0kc1gU7o9O8xFuwISNEBCvGPlJc0Cen5lFBSKpBE4/rP83oLpH
   JElPnd3yjfvy7ZbLDkQhJC8MY2xeZgmf782+dHdn5tiN1En3GEFib2R1tQAMYrYU8eePgamZYRsB
   5PMVGKuyWJGA4ichN1c=
DomainKey-Signature: a=rsa-sha1; c=nofws; q=dns; s=default; d=casavekhatib.com;
 b=xuOEm+I88I25xcwZ+XX76PN34R/nze43L19fxQsirmkI4LMH/vfyZ1YO64s/ycg1/AxHO5yFFS3u
   W+SrwmTLxJ729Jddd6V5tqGxC+MXbIfmhIDu7y8dUcJQmgFdnnXoVyOidlZ/bqHcsFz/0gmCnz3p
   9yGzNFzvDhxxqDKTm6I=;
To: <n_____@_____t>
Message-ID: <[e-mail]>
Date: Thu, 23 Sep 2010 04:10:49 -0700
From: "Dermology" <[e-mail]>
Mime-Version: 1.0
Subject: Reduce Existing Stretch Marks
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 8bit
Content-Disposition: inline
X-GMX-Antivirus: 0 (no virus found)
X-GMX-Antispam: 0 (Mail was not recognized as spam);
 Detail=5D7Q89H36p5xOp9NadjzZxxziP5/kfXJ5KpNrvVVDT8k/3xBNwpR2D4gX/5vpX9rVASzz
 x0tqqd6iqBr8IiheizrUBud+qH6a9H32iWRgE0eiz/pxBTiupI6+i/uoWJp6f01OANiITCazhY8C
 BTqx6jVIonG2InjV1;
Subject: *****[Spam]*****[83.2%] (No Subject)

Comment 7

[Neal Poole]

Attachment: A screeenshot of the behavior

I've just run into this behavior with From: lines in Thunderbird 6. I'm attaching a screenshot.

The first From line ([e-mail]) is displayed when viewing the email in full. The second From line ([e-mail]) is displayed in the list of messages.

Comment 8

[WADA:World Anti-bad-Duping Agency]

Removing "display inconsistently" part from bug summary to avoid confusion, because it's processed by bug 172104.

Comment 9

[WADA:World Anti-bad-Duping Agency]

Bug 678322 exists for filter of leagal "multiple same message headers" case.
When illegal multiple headers case like this bug(multiple Subject:, From:, etc.), filtering of wrongly added headers may produce problems. Only fist header should be used in both thread pane/message pane display and message filtering.

(In reply to Neal Poole from comment #7)
> The first From line ([e-mail]) is displayed when viewing the email in full.
> The second From line ([e-mail]) is displayed in the list of messages.

This is From: version of bug 172104 for Subject:.
As seen in patch proposed to bug 172104, "use first header" is done on Subject: header only by fix for bug 172104.
Neal Poole, can ypu open separate bug for display of From: case?

Comment 10

[WADA:World Anti-bad-Duping Agency]

Following is headers defined as "Max number=1" in RFC 5322.
> http://tools.ietf.org/html/rfc5322#section-3.6
>    +----------------+--------+------------+
>    | Field          | Min    | Max number |
>    |                | number |            |
>    +----------------+--------+------------+
>    | orig-date      | 1      | 1          |
>    | from           | 1      | 1          |
>    | sender         | 0*     | 1          |
>    | reply-to       | 0      | 1          |
>    | to             | 0      | 1          |
>    | cc             | 0      | 1          |
>    | bcc            | 0      | 1          |
>    | message-id     | 0*     | 1          |
>    | in-reply-to    | 0*     | 1          |
>    | references     | 0*     | 1          |
>    | subject        | 0      | 1          |
>    +----------------+--------+------------+

If Max number=1, "multiple headers" means malformed mail, and first header should be used for thread pane display & message pane display, and in message filtering & search too. This should be consistent among headers defined as "Max number=1".

Comment 12

[Magnus Melin [:mkmelin]]

Attachment: Bug 310189 - consistently use first header for the headers that have max-number of 1. r=kaie

Comment 13

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/comm-central/rev/6aed229a3bcb
consistently use first header for the headers that have max-number of 1. r=kaie

Comment 15

[Magnus Melin [:mkmelin]]

For anyone testing (or bothered with old messages where it's wrong), if you already have the message in the folder, it won't get re-parsed until you do repair folder.

Comment 16

[Magnus Melin [:mkmelin]]

Comment on attachment 9261520 [details]
Bug 310189 - consistently use first header for the headers that have max-number of 1. r=kaie

[Approval Request Comment]
Regression caused by (bug #): ancient problem
User impact if declined: problems due to bad headers
Testing completed (on c-c, etc.): just landed on c-c
Risk to taking this patch (and alternatives if risky): it's well understood. The same fix was applied to Subject years ago, but was not handled for other headers for some reason. Has automated test now.

Comment 17

[Rob Lemley [:rjl]]

Comment on attachment 9261520 [details]
Bug 310189 - consistently use first header for the headers that have max-number of 1. r=kaie

Merge picked this up on beta.

Comment 18

[Wayne Mery (:wsmwk)]

Comment on attachment 9261520 [details]
Bug 310189 - consistently use first header for the headers that have max-number of 1. r=kaie

[Triage Comment]
Approved for esr91

Comment 19

[Rob Lemley [:rjl]]

Thunderbird 91.7.0:
https://hg.mozilla.org/releases/comm-esr91/rev/b30848bcf0a8