Last Comment Bug 8405 - [Beta] MIME-encoded JPN address-from characters are not displayed correctly in thread pane
: [Beta] MIME-encoded JPN address-from characters are not displayed correctly i...
Product: MailNews Core
Classification: Components
Component: Backend (show other bugs)
: Trunk
: x86 Windows NT
P3 critical (vote)
: M14
Assigned To: scottputterman
: Katsuhiko Momoi
: 25176 (view as bug list)
Depends on:
Blocks: 7228 11091 14356 25176
  Show dependency treegraph
Reported: 1999-06-17 11:52 PDT by Katsuhiko Momoi
Modified: 2008-07-31 01:21 PDT (History)
6 users (show)
See Also:
Crash Signature:
QA Whiteboard:
Iteration: ---
Points: ---

screen shot and its inbox data (66.51 KB, application/octet-stream)
1999-09-21 09:27 PDT, nhottanscp
no flags Details

Description User image Katsuhiko Momoi 1999-06-17 11:52:41 PDT
** Observed with 6/17/99 Win32 build **

I have a msg in which the address-to field contains MIME-encoded
Japanese characters along with the offical e-mail address. The
line looks like this un-decoded:

From: =?iso-2022-jp?B?GyRCRW0wZj4hSScbKEI=?= <>

This should be decoced and displayed correctly but it is not in the
thread pane. Instead it shows as 4 dots, which seem to corresond
to the number of Japanese characters. Naoki had a case where his
3 Japanese characters show as 3 dots.

This header does show correctly in the message viewing window's header.

Scott, I can send you a sample message in which the addree-to header,
subject header, and the identical strings. (Just le me know your test
address). With a Japanese font somewhere in your system, you should
be able to see the difference between thread and message viewing pane.
Comment 1 User image Katsuhiko Momoi 1999-06-17 11:53:59 PDT
Assigned myself as teh QA contact.
Comment 2 User image Scott MacGregor 1999-06-17 11:57:59 PDT
Adding scottip to the cc list as well.

When I fetch new mail from a pop server, I'm not decoding the header info before
inserting it into the database. I wonder if the decoding of the From header
needs to happen here (we could convert it to the character set of the folder we
are adding the msg to) OR if the header needs to be decoded when we go to
display it in the thread pane (which would be a scottip area).
Comment 3 User image Katsuhiko Momoi 1999-06-17 12:04:59 PDT
I'm sorry my example was for "address-from" field, not "address-to" filed.
Scott, do you want me to try the address-to field with some JPN
characters. (I corrected the summary above).
Comment 4 User image Katsuhiko Momoi 1999-06-17 12:09:59 PDT
I tried including JPN in the adress-to field.
These characters showed OK in the message viewing headers.
The address-to is not used in the thread pane and so this does
not seem to be an issue.
Comment 5 User image nhottanscp 1999-06-17 13:13:59 PDT
Additional info, I have a case of ISO-8859-1 and that displays correctly. If
MIME decode were missing, that should have failed too.

From: =?iso-8859-1?Q?Sant=E9?=  Nouveau!
Comment 6 User image Scott MacGregor 1999-07-01 14:07:59 PDT
I need to look at triaging this bug and giving it a target milestone fix. naoki,
your last set of comments confused me. It made me think that things are working?
I think I've just lost track of what needed fix. Or are we doing some decoding
as being evidenced by your discovery on 6/17 just not all the necessary

Is the encoding problem in the thread pane or in the message pane for the
address-from line?

Comment 7 User image nhottanscp 1999-07-01 14:20:59 PDT
What I meant was that this might not be the case where MIME decode is missing.
In order to show headers correctly, we need to MIME decode and convert to
unicode. As far as I've seen MIME deocode is done because I don't see raw MIME
encoded strings like below. But charset conversion may be missing because I
cannot see the fist example below but I can see the second one (Latin1
range usually works without charset conversion).
From: =?ISO-2022-JP?B?GyRCSlQ9ODZJGyhC?= <>
From: =?iso-8859-1?Q?Sant=E9?=  Nouveau!
Comment 8 User image Scott MacGregor 1999-07-07 00:24:59 PDT
changing TFV to M11.
Comment 9 User image David :Bienvenu 1999-08-16 15:00:59 PDT
I thought this was more a scottip issue, because we're storing the header
undercoded, as planned, and only doing the decoding before display. Scott, isn't
this right? It looks like we're doing the GetMime2DecodedAuthor, which seems
like what we're supposed to do. Or is it the charset conversion we're missing?
Comment 10 User image scottputterman 1999-08-16 15:05:59 PDT
I'm not doing anything with charsets.
Comment 11 User image Katsuhiko Momoi 1999-09-17 22:51:59 PDT
What's going on with this bug? It is marked for M11 and
we haven't seen any noise on it for a month now.
Scott, please update.
Comment 12 User image Scott MacGregor 1999-09-19 22:24:59 PDT
I haven't started to look at it. I'd say it's on the bubble for me getting to it
for m11.
Comment 13 User image Scott MacGregor 1999-09-19 22:39:59 PDT
(I should add that Phil is helping me priortize my M11 bugs and I believe he's
going to be making a pass through all our bugs Monday).

I was just scanning through this bug report and I'm still slightly confused by
Naoki's last comments. He makes it sound like MIME decode is happening (which
would be my end of the boat) but the problem may be in some character set
conversion stuff ?? (would that be scottip?)
Comment 14 User image lchiang 1999-09-20 16:01:59 PDT
(target milestone is M11 or M12 - add to mail beta tracking bug)
Comment 15 User image nhottanscp 1999-09-21 09:27:59 PDT
Created attachment 1793 [details]
screen shot and its inbox data
Comment 16 User image nhottanscp 1999-09-21 17:30:59 PDT
I debugged this and found that nsAutoCString(sender) is used in the code. That
only works for Latin1 characters.
In order to work correctly, it needs unicode conversions. Since it's converting
between UCS2 and UTF-8, there is a easier way.
My code below leaks, so do not paste this, this is just an idea how to convert.

#include "nsTextFormater.h"
//sender is the string we need to parse.  senderuserName is the parsed user name
we get back.
nsresult nsMsgMessageDataSource::GetSenderName(nsAutoString& sender,
nsAutoString *senderUserName)
	//XXXOnce we get the csid, use Intl version
	nsresult rv = NS_OK;
		char *name;
#if 0
		if(NS_SUCCEEDED(rv = mHeaderParser->ExtractHeaderAddressName
(nsnull, nsAutoCString(sender), &name)))
			*senderUserName = name;
		if(NS_SUCCEEDED(rv = mHeaderParser->ExtractHeaderAddressName
("UTF-8", sender.ToNewUTF8String(), &name)))
      PRUnichar *u;
      nsAutoString fmt("%s");
			u = nsTextFormater::smprintf(fmt.GetUnicode(),name);
Comment 17 User image scottputterman 1999-09-21 17:32:59 PDT
reassigning to me.
Comment 18 User image Katsuhiko Momoi 1999-10-25 10:44:59 PDT
Im raising the severity of this bug. Being able to see the
sender's name correctly in the thread pane seems to be a
basic functionality we expect in mail. Also evaluating this
for Dogfood.
Comment 19 User image Katsuhiko Momoi 1999-12-24 23:42:59 PST
We need to get this into Beta 1. I can't think of pushing Beta without
this working OK. Mail looks terrible when you cannot see the sender's
names in CJK languages.
Bob, please add this to the Beta 1 list.
Comment 20 User image Katsuhiko Momoi 2000-01-27 01:26:25 PST
*** Bug 25176 has been marked as a duplicate of this bug. ***
Comment 21 User image Katsuhiko Momoi 2000-01-27 01:29:40 PST
*** Bug 25176 has been marked as a duplicate of this bug. ***
Comment 22 User image Katsuhiko Momoi 2000-01-27 01:34:42 PST
*** Bug 25176 has been marked as a duplicate of this bug. ***
Comment 23 User image leger 2000-01-28 17:48:47 PST
Putting on PDT+ radar for beta1.
Comment 24 User image scottputterman 2000-02-01 17:48:34 PST
fix checked in.  The message the momoi sent me showed up with the correct sender 
in the thread pane on all 3 platforms.
Comment 25 User image Katsuhiko Momoi 2000-02-03 13:10:59 PST
** Checked with 2/3/2000 Win32 build **

Looks great. I have been able to confirm so far that properly MIME-encoded
Sender names in Latin 1, Russian (KOI8-R-), Japanese (ISO-2022-JP), and
Trad Chinese (Big5) dispay correctly in the thread pane.

ji, can you confirm that this is OK on Linux.
I'm going to mark this verified/fixed. Please re-open if there is a problem on Linux.

Note You need to log in before you can comment on or make changes to this bug.