369269 - show raw/source header values in header pane (don't unfold long headers)

Status: NEW

(MailNews Core :: Backend, enhancement)

Description

[snowcrash+mozilla]

User-Agent:       Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2pre) Gecko/20070203 BonEcho/2.0.0.2pre
Build Identifier: Mozilla/5.0 (Macintosh; U; PPC Mac OS X Mach-O; en-US; rv:1.8.1.2pre) Gecko/20070203 Thunderbird/2.0pre Mnenhy/0.7.4.10005

when i receive a message that's, e.g., passed through SpamAssassin, if i
view the Message Source in any client, i see a correctly/expected
formatted report in a long, folded-whitespace header, e.g:

----------------------------------------------------------
X-Spam-Checker-Version: SpamAssassin 3.1.8-r454679 (2006-10-10)
X-Spam-Level: !!!!!!!!!!!!!!!!!
X-Spam-Status: score=17.5/4.0 autolearn=spam
X-Spam-Report:
      *  1.1 EXTRA_MPART_TYPE Header has extraneous Content-type:...type= entry
      *  0.0 DK_POLICY_SIGNSOME Domain Keys: policy says domain signs
some mails
      *  5.0 BOTNET Relay might be a spambot or virusbot
      *      [botnet0.7,ip=208.103.1.19,hostname=208.103.0.19.etczone.com,baddns,client,ipinhostname]
      *  0.1 TW_CX BODY: Odd Letter Triples with CX
      *  0.1 TW_GW BODY: Odd Letter Triples with GW
      *  0.1 TW_MK BODY: Odd Letter Triples with MK
      *  0.1 TW_BJ BODY: Odd Letter Triples with BJ
      *  0.1 TW_JM BODY: Odd Letter Triples with JM
      *  0.1 TW_UW BODY: Odd Letter Triples with UW
      *  0.1 TW_PW BODY: Odd Letter Triples with PW
      *  0.1 TW_IU BODY: Odd Letter Triples with IU
      *  0.1 TW_YJ BODY: Odd Letter Triples with YJ
      *  0.1 TW_DB BODY: Odd Letter Triples with DB
      *  0.0 HTML_MESSAGE BODY: HTML included in message
      *  3.1 HTML_IMAGE_ONLY_08 BODY: HTML: images with 400-800 bytes of words
      *  1.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
      *      [score: 0.5531]
      *  6.0 FUZZY_OCR BODY: Img with common spam text inside
      *      Words found:
      *      "cialis" in 1 lines
      *      "viagra" in 1 lines
      *      "cialis" in 1 lines
      *      "viagra" in 1 lines
      *      (4 word occurrences found)
----------------------------------------------------------

if i open the message in "another MUA", (e.g. Mulberry, KMail, Pine, Evolution, Apple Mail, Euroda), and view 'all' headers, i see a similarly, correctly formatted:

----------------------------------------------------------
X-Spam-Level: !!!!!!!!!!!!!!!!!
X-Spam-Status: score=17.5/4.0 autolearn=spam
X-Spam-Report:
      *  1.1 EXTRA_MPART_TYPE Header has extraneous Content-type:...type= entry
      *  0.0 DK_POLICY_SIGNSOME Domain Keys: policy says domain signs
some mails
      *  5.0 BOTNET Relay might be a spambot or virusbot
      *      [botnet0.7,ip=208.103.1.19,hostname=208.103.0.19.etczone.com,baddns,client,ipinhostname]
      *  0.1 TW_CX BODY: Odd Letter Triples with CX
      *  0.1 TW_GW BODY: Odd Letter Triples with GW
      *  0.1 TW_MK BODY: Odd Letter Triples with MK
      *  0.1 TW_BJ BODY: Odd Letter Triples with BJ
      *  0.1 TW_JM BODY: Odd Letter Triples with JM
      *  0.1 TW_UW BODY: Odd Letter Triples with UW
      *  0.1 TW_PW BODY: Odd Letter Triples with PW
      *  0.1 TW_IU BODY: Odd Letter Triples with IU
      *  0.1 TW_YJ BODY: Odd Letter Triples with YJ
      *  0.1 TW_DB BODY: Odd Letter Triples with DB
      *  0.0 HTML_MESSAGE BODY: HTML included in message
      *  3.1 HTML_IMAGE_ONLY_08 BODY: HTML: images with 400-800 bytes of words
      *  1.5 BAYES_50 BODY: Bayesian spam probability is 40 to 60%
      *      [score: 0.5531]
      *  6.0 FUZZY_OCR BODY: Img with common spam text inside
      *      Words found:
      *      "cialis" in 1 lines
      *      "viagra" in 1 lines
      *      "cialis" in 1 lines
      *      "viagra" in 1 lines
      *      (4 word occurrences found)
----------------------------------------------------------

BUT, if i open the message in Thunderbird2, the line-breaks in the
header are apparently stripped off; here's what it looks like.

      http://img100.imageshack.us/img100/278/mnenhyallheaderswh1.jpg

the image is also attached to this post ...

In troubleshooting this, i was informed about the Mozilla MailNews
backend, that TBird is using,

 > As per RfC (2)822, header _values_ are always just *one* line.
 > To get around the (server) restriction of 998 usable characters per
 > line, it is allowed to split the value into multiple lines. But these
 > line breaks are *not* part of the actual value and recipients have to
 > remove the line breaks when decoding the message to get back the real
 > value. If the the value should contain line breaks, these have to be
 > encoded before, eg. as =0A in the Quoted Printable encoding.
 >
 > The X-Spam-Result header value is not encoded, thus the line breaks used
 > as a formatting in the source are *not* part of the value and *must* be
 > stripped before passing the value to the frontend.
 >
 > The MailNews backend handling is correct.

Since this is the same message, retrieved from the same mail server, and, therefore, having been processed by the same instance of SA, i'm guessing that MailNews backend d-- & what it does with whitepace folded, long headers -- *is* the issue here.

Reproducible: Always

Steps to Reproduce:
1.
2.
3.

Comment 1

[snowcrash+mozilla]

Attachment: picture of UN-folded headers in tbird client

this un-folding occurs whether, or not, mnenhy is installed

Comment 2

[Tuukka Tolvanen (sp3000)]

stock thunderbird doesn't have a multiline header value view, afaict, and unfolding the value per rfc 2822 2.2.3 is clearly necessary in the single line output case. I suppose it might be nice for mnenhy's multiline value view to avoid unfolding (and for the backend to enable it to do so if it currently doesn't) although that combined with soft linebreaking would likely lead to bad results in less intentionally formatted long heder values in a narrow space.

Comment 3

[snowcrash+mozilla]

again, this is also a problem WITHOUT the presence of mnenhy.  i.e, with 'stock' tbird ...  in that the long header is displayed as a loooon, single line ... which one cannnot scroll to see.

tbh, i don't understand your "would likely lead to bad results" arguments, as no other client that i'd mentioned has any troubles whatsoever with any long headers.  _they_ all consistently handle this correctly.

tbird (well, mozilla clients ...) are unique in their inability to display a properly formatted header


are you suggesting that 'eeveybody else' is wrong/incorrect in how they handle this?

thanks.

Comment 4

[Tuukka Tolvanen (sp3000)]

The header values are scrollable with mouse selection and the usual keyboard text navigation. The "bad results" referred to the case of a non-unfolding multiline view that was soft-wrapped, in addition to the hard breaks due to not doing unfolding, to a smaller width than those of the header lines. Also, headers that aren't intended to contain formatting but are long and folded, and would fit into the view unfolded, would take up excess vertical space and leave useless horz whitespace if shown folded. (example: bugmail from this bug)

I might have misunderstood your request -- I take it this is for a mnenhy-like multiline header value view but without unfolding, rather than just doing away with unfolding. That's kind of reasonable, but due to the above caveats it sounds like it would need special handling (per header name I guess) to not make some headers appear worse.

Comment 5

[Tuukka Tolvanen (sp3000)]

> useless horz whitespace if shown folded. (example: [subject header of] bugmail from this bug)

Comment 6

[snowcrash+mozilla]

> The header values are scrollable with mouse selection and the usual keyboard
> text navigation.

i did not know that.  it does work, but for such a long header, that's completely useless, imho.

> The "bad results" referred to
<snip>
> I take it this is for a mnenhy-like
> multiline header value view but without unfolding, rather than just doing away
> with unfolding. That's kind of reasonable,

i should be careful as to what i'm asking for here, as i'm aware that "mnenhy-like", "multiline" and "unfolding" have _specific_ meaning(s) to you (and others?), and i'm not at all sure *I* am using/differentiating the terms correctly.

so, instead, let's try this,

<unreasonable & demanding end-user hat>
i want my headers to look like "nicely formatted" this:

  http://img299.imageshack.us/img299/781/mulberryheadersoo8.png

NOT "run together" like (in tbird) this:

  http://img299.imageshack.us/img299/2159/tbirdheaderwithmnenhyhm6.png

</unreasonable & demanding end-user hat>

> but due to the above caveats it
> sounds like it would need special handling (per header name I guess) to not
> make some headers appear worse.

i certainly don't know the limitations of the mailnews code, but it seems odd to me that tbird/mozilla is the only case that seems to need "special handling", as, none of the other mentional MUAs have any header display problems/issues.

i suppose it _could_ the case that _all_ of the others have _implemented_ "special handling", and that tbird simply lacks it ... i'd be surprised, but if true, perhaps "do what the others do" might not be a bad place to start.

Comment 7

[Karsten Düsterloh]

Actually, this bugs summary is just wrong: the unfolding is correct (I did say that before in comment #0, did I? ;-) )

OTOH, I think one could rephrase your request to "allow source view of header values in the header pane", which IMO is a valid enhancement request - albeit not very likely to be fixed soon as it is no real bug...

Comment 8

[snowcrash+mozilla]

i'm 'resigning' from the sparring back and forth about 'proper language'.

again, my only goal here is, again, to see this

  http://img299.imageshack.us/img299/781/mulberryheadersoo8.png

NOT this,

  http://img299.imageshack.us/img299/2159/tbirdheaderwithmnenhyhm6.png

call it a bug, or not, it really doesn't matter to me.

i honestly don't understand the defense_of/insistence_on formatting that looks shoddy, and behaves in a fashion that is, clearly, not how the 'senders' intended ... especially since it behaves demonstrably correctly in so many other
clients.

if folks here don't want to fix it, then please just say so, and close the bug -- there are alternatives.

Comment 9

[Karsten Düsterloh]

Don't get me wrong, I'm all for fixing this. Just don't hold your breath until someone writes a patch that does it...