28908 - Need a warning dialog for non-matching charset in headers and text body

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect, P3)

Description

[Katsuhiko Momoi]

This is an RFE.
Mozilla should be designed to get average users out of having
to meddle/deal with charset issues or charset menu manipulations.

Mime charset honoring, Mail send charset charset feedback menchanism,
auto-detection, and deafult default settings are all ways to make 
it so that the user will have to do minimum about charset to use mail 
correctly.

There is one area where we can do better and that is when the
the input charset does not match the chosen encoding for mail send.

It could easily happen because Mozilla's layout will display just
about any language input without complaint or having to change
the charset menu. For these cases, we should issue an warning like
OE5 does for body text and encoding mismatch. Here are 2 specific
types:

1. Input in the body contains characters outside of the chosen 
   encoding or HTML 4 CERs.
2. Input in the subject (including the case when the subject 
   is automatically read in when engaging Page Send on a page
   which has a meta charset tag or HTTP charset) does not 
   match the chosen encoding. 

We should not expect average users to be savvy with charset feedback mechanism
or the charset menu manipulation. Including these 2 types of warnings
should take care of most of user mishaps.

Comment 1

[Katsuhiko Momoi]

*** Bug 28865 has been marked as a duplicate of this bug. ***

Comment 2

[nhottanscp]

Let me explain the current charset conversion fallback behavior (when a 
character cannot be mapped to the target encoding).

text/html - search for CER if not found then use NCR
text/plain - transliterate (e.g., EUR) then fallback to '?'

What kind of additional fallback to be provided to the user by the proposed 
warning dialog?

Comment 3

[Katsuhiko Momoi]

I would like to addrsss this quesiton in the overall context
of what else we are doing in the charset related UI.

I believe that there are cases when it would be good to
warn the user they have chosen an encoding in which we may
have to send all the data in NCRs unless the encoding is
changed. This is the of case whihc might arise because our
layout can take just about any language input and display
them correctly. In 4.x, unreasonable encoding for input data 
usually are noticed because of font switching and data looking
like garbage. Under Mozilla, this is no longer true and thus 
it would be best if we put in some warning dialogs in these
mismatched cases. The other reason is that there are many 
different user scenarios and I don't think we can exhaust them. 

Let's put this item on the agenda for a while and review them
later in the overall context of Messenger Intl UI. (You already
set this to M16 and so that is good.)

Comment 4

[nhottanscp]

My question was what other options we can provide. Do we ask the user to change 
charset by the menu?

Comment 5

[Katsuhiko Momoi]

Yes, I think in some cases the user should be given an option to
change charset or not include multilingual data.
 

Comment 6

[Frank Tang]

Should we do the same thing for 
1. Save as Plain Text 
2. HTML form posting 

Comment 7

[Frank Tang]

file form posting warning as 29271, save as text in 29272

Comment 8

[nhottanscp]

> the user should be given an option to change charset 
> or not include multilingual data.
What UI do we need? Is it okay to use an alert with ok and cancel button?

Comment 9

[Katsuhiko Momoi]

Yes. I think an "alert" would be fine. Let's work on proper wording.

Comment 10

[nhottanscp]

Here is a list of things need to be done.

1) in charset conversion, check the error code NS_ERROR_UENC_NOMAPPING
2) if error pop up an alert
3) if ok proceed mail send
4) if cancel abort the mail send

Question to Jean-Francois:
Is there a generic way to use alert in C++?
Is there a way to abort in message send? There are two places (plain and html), 
nsMsgCompose::SendMsg and nsMsgComposeAndSend::GetBodyFromEditor.

Comment 11

[Jean-Francois Ducarroz]

we have several Alert utility functions in msg compose, look in nsMsgPrompts.cpp

To abort a send process, just return an error code. As you do not want a second 
alert message to popup (because you abort the send), return 
NS_ERROR_BUT_DONT_SHOW_ALERT

Comment 12

[nhottanscp]

put beta2 keyword, however I am not sure if we need this for text/html case.
I put my comment in bug 29272. So I mean beta2 for text/plain case only.

Comment 13

[nhottanscp]

Attachment: patch file, r=ducarroz

Comment 14

[nhottanscp]

Checked in the alert for plain text send. If you send á in japanese plain text 
you will see the alert, please check the string to correct English.
As I mentioned before, I am not sure about if we need the alert for html case.
Moving M17 to wait for the decision.

Comment 15

[leger]

Putting on [nsbeta2+] radar.  Need to implement for 5/16 or may pull.

Comment 16

[nhottanscp]

I checked in the code to check the subject header in addition to the plain text 
body.
For HTML body, we have not decided whether we need that feature (see bug 29272).

I will mark this as FIXED and HTML issue should be discussed under 29727.

Comment 17

[nhottanscp]

fixed in M16

Comment 18

[Katsuhiko Momoi]

** Checked with 6/1/2000 Win 32 build **

While there is no way to check all possibilities in which
the chosen charset does not match the actual charset of 
the body or header data, some samplings indicate that this
feature is working as intended. 
The only thing I found is that the warning does come up 
when we are composing HTML mail.
This issue will be discussed in Bug 29272 and it should be
OK to leave it as is. (It's either this way or some other
way.)

Comment 19

[Katsuhiko Momoi]

Marking it verified as fixed.

Comment 20

[Katsuhiko Momoi]

To clarify what I meant for HTML mail.
The warning comes up for both header and body mismatches. 
I tried ASCII header with a Latin 1 body under Japanese 
and the warning came up, for example.

Comment 21

[nhottanscp]

>This issue will be discussed in Bug 29272
Yes, I think 29272 need to be re evaluated if we need that feature. The SaveAs 
spec has changed since the bug was filed.