77903 - If Body MIME present, no Headers MIME and folder MIME override is not on -> use body MIME to display Headers.

Status: RESOLVED WONTFIX

(MailNews Core :: Internationalization, defect)

Description

[Alexey Chernyak]

This is spawned from bug 75506 to mirror the same fix for other areas of message
composition that need it.

I believe same character encoding logic as in bug 75506 should be used in:

1.
"Author wrote:" line in message body when you "Reply" to an e-mail.
At the moment folder character coding is used for Author's name there.

2.
Reply-To: field in message Composition when you "Edit Message As New".
I'm not sure what encoding it is using there at the moment.

These are just 2 examples I noticed, maybe there are more places like this.

this can be tested on the same testcase as before:
http://bugzilla.mozilla.org/showattachment.cgi?attach_id=30702

Great work on bug 75506! Mozilla is now a bit more pleasurable to use. :o)

Comment 1

[Alexey Chernyak]

Oh and how could I forget!
Same should be applied to all fields in Header display in message view.
At the moment it uses folder encoding when none is specified in message header
fields.
This is for both, "Normal" header view and minimised header view. And for "All"
whenever it will be implemented.

Comment 2

[ji]

For problem 2, bug 41009 has been filed.

Comment 3

[marina]

Send mail charset problem is reported in bug #53634

Comment 4

[nhottanscp]

We don't guess (i.e. auto detect) for headers. It's too expensive and may not
get correct result anyway with a small amout of characters used in headers.
Currently, it's only used for non labeled attachments.

Comment 5

[Alexey Chernyak]

nhotta, I didn't mean guessing as in implementing AI for it. What I meant is
putting the same algorithm you used for bug 75506 in a few other similar places
that need it.

I certainly didn't mean guessing encoding for message body when it is absent.
What I meant was, if encoding is given for the body, but not for Header fields
like Subject: From: To: Reply-To: and so on.

bug 53634 and bug 41009 are absolutely different from this. If I'm not mistaken,
they are dealing with auto-detecting encoding of the body. This bug is about
guessing encoding of other message headers when body encoding is known. Exactly
the same as bug 75506, but for rest of the places that need it.

Sorry, for not making it clear enough. I will attach a few screenshots to show
what I am talking about.

Comment 6

[Alexey Chernyak]

Attachment: testcase

Comment 7

[Alexey Chernyak]

Attachment: messageview Normal header

Comment 8

[Alexey Chernyak]

Attachment: messageview compact header

Comment 9

[Alexey Chernyak]

Attachment: Reply to Sender

Comment 10

[Alexey Chernyak]

Attachment: Edit Message as New

Comment 11

[Alexey Chernyak]

In "Reply to Sender" screenshot you can see the fix for bug 75506 working inside
To: field.
The fields marked with red need the same kind of treatment.

Comment 12

[ji]

Thanks for the screenshots, now we understand when you talked out. We need this
bug to address the problems in Normal header view, compact header and author in
reply mail body. Naoki, do you need seperate bugs for those?
The problem for "Edit as New" is already in 41009.

Comment 13

[ji]

I can reproduce the problem in compact header view, but I can't reproduce it on 
Normal header view and the author problem in reply mail body. After I set the 
folder charset to KOI8-R, those places are displayed correctly.

Comment 14

[Alexey Chernyak]

I've read though bug 41009 and I don't see it addressing this problem.
It addresses cases when Header MIME is correct, but Body MIME is missing or
incorrect.

This bug is opposite of that. Correct Body MIME with missing Header MIME.

In bug 41009:
* See screenshot attached.
* Read Katsuhiko Momoi's comments from 2000-08-10
* Read Katsuhiko Momoi's description of attachments on 2000-08-13




I used Western ISO-8859-1 folder encoding to demonstrate Headers and Author
reply effect.
I just think when displaying that, message body encoding should have priority
over folder encoding.
See comments nhotta made in bug 75506 on 2001-04-13
I liked that algorithm very much. :o)

I do know this can be fixed with setting folder encoding.
But I must admit I have used Mozilla for half a year without even knowing about
that feature. I am coming from a user point of view here. It's better when it
just works, than when you have to do somethting for it to work, especially when
you are unaware what can be done to fix that.

Comment 15

[Alexey Chernyak]

In previous comment I was referring to nhotta's comments in bug 75506 from:
2001-04-13 13:58

Comment 16

[ji]

Yes, that's right. 41009 is about mail body with no or wrong charset lable.
I remember there is a bug about header somewhere.

Comment 17

[ji]

Bug 66098 is filed to use folder charset for headers which has no or wrong 
charset label, although it's originally filed for reply msg.

Comment 18

[Alexey Chernyak]

Changing Summary to be more specific:
Old Summary: Need to guess right character encoding for message Header fields
when not specified
New Summary: If Body MIME present, no Headers MIME and folder MIME override is
not on -> use body MIME to display Headers.

At the moment, if Headers don't have MIME, folder MIME is used even if Body has
MIME specified.

To reproduce:
1. In "View -> Folder Character Coding" Set folder encoding to Western
(ISO-8859-1) and disable "Apply default to all messages (ignore charset
specified by MIME header)"
2. View attached testcase. You will see that message body is correctly displayed
using KOI8-R encoding, but Headers are displayed using ISO-8859-1.
3. Go to "View -> Character Coding", notice that KOI8-R is selected in there.
4. Click on KOI8-R anyways. You will now see Header display change to KOI8-R and
display messages correctly.

I think, if the message Body uses encoding for display, Headers with no MIME
should be displayed using the same encoding, unless folder encoding override is
not on.
I think the behaviour shown in above 1-4 steps should be used only when folder
encoding override is set.

See previously provided screenshots for areas that are affected by this.
Adding one more screenshot.

I think bug 66098 is probably a subset of this bug.

Comment 19

[Alexey Chernyak]

oops, in previous comment I meant:
"if the message Body uses encoding for display, Headers with no MIME
should be displayed using the same encoding, unless folder encoding override is
 ON."

Comment 20

[Alexey Chernyak]

Attachment: Threadpane

Comment 21

[Alexey Chernyak]

bug 80942 seems to be related.

Comment 22

[Alexey Chernyak]

oops sorry pasted wrong bug number.
I meant bug 71541.

Comment 23

[Alexey Chernyak]

*** Bug 97314 has been marked as a duplicate of this bug. ***

Comment 24

[ji]

The fix should also consider the case when the mail doesn't contain charset info 
either in mail body or header. In this case, when auto-detector is turned on, 
the charset sniffed by auto-detector should be passed to the header display.
For the details of the discussion, please see bug 97314.

Comment 25

[[e-mail]]

When trying to forward a malformed message, Message Compose succeeded in
displaying the subject correctly, despite failing to set the correct default
character set...

Comment 26

[nhottanscp]

*** Bug 109867 has been marked as a duplicate of this bug. ***

Comment 27

[Boris Zbarsky [:bzbarsky]]

*** Bug 168488 has been marked as a duplicate of this bug. ***

Comment 28

[Simon Bohlin]

Using Thunderbird version 1.0.7 (20050923) in slovene version of Windows 98, the folder character set is not applied to messaged with UNKNOWN.
Only when I check to override message settings can I get the folder character set on subject and from-columns in the folder list, but I need several charsets so I'd be very happy if this bug could be solved.
I agree with Alexey and I will vote for this bug.

Comment 29

[Mike Cowperthwaite]

The problem in comment 1 & (especially) comment 18 (which is where this bug morphed away from the symptoms in comment 0) is bug 90584.  I recommend duping this one to that one.  Don't mark a dupe in the other direction, as that bug has a little bit of traction towards gettings solved.

Comment 30

[Joshua Cranmer [:jcranmer]]

So the rules for JSMime are that the body charset is used as fallback for the headers if UTF-8 doesn't work, which is AFAICT what this bug asks. I see no reason to change libmime (which I think already largely supports this feature, except in cases where the body charset is overridden by late information, i.e., <meta charset=> or charset autodetection, both of which should hopefully be dieing out anyways).