20520 - Euro symbol in the subject doesn't display in the thread pane.

Status: VERIFIED FIXED

(SeaMonkey :: MailNews: Message Display, defect, P3)

Description

[ji]

The Euro symbol in the mail subject doesn't display in the thread pane.
It displays in the message pane.

Steps of repro:
1. The zip file shown in the above URL is the mail folder we used
   for smoke test.  Unzip and put it the mail directory of your testing
   account.
2. The last message in the smoke test folder is sent from 4.6. Its
   subject contains Euro symbol. Go to that message, you'll see
   the Euro symbol doesn't display in the thread pane, but it displays
   in the message pane.

Comment 1

[nhottanscp]

This seems to be specific to the mail thread pane (and happens across platform).
I tried bookmark and it shows the Euro symbol correctly.
Reassign to putterman for investigation.

Comment 2

[nhottanscp]

According to momoi, this is a regression from M11.

Comment 3

[Katsuhiko Momoi]

The last known Win32 build on which this was working for M12 is the build from 11/12/99.
We know that the problem exists with the 11/16/99 M12 build. Will try to pinpoint the date
when the problem appeared.

Comment 4

[Katsuhiko Momoi]

The problem started with the 11/13/99 build (on the Win32 build). On the 11/12/99 build, the EURO
displayed OK in the thread pane.

Comment 5

[scottputterman]

setting M15

Comment 6

[leger]

Due to Beta indication in Summary, putting beta1 into keyword field.

Comment 7

[rickg]

This isn't interesting enough to make PDT. 

Comment 8

[Tobias Burnus]

Beta1 should probably beta2.
I get something similar: A message which contains a u umlaut ü is truncated just
before the ü. This happens in the message header displayed when the message is
shown and in the window which opens when I click on the "From:" (full header
view), but in the thread view it is shown correctly.
Re-reading the description, I think this is just the otherway round here.
Internal it looks like this:

Subject: [Vortrag] Das Betriebskonzept der ZEDAT für PC-Lehr- und
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

See several news at news://bln.announce.fub.zedat

Comment 9

[scottputterman]

reassigning to rhp.  Is this still true.  The comments from 5/25 implies that 
this is no longer happening in the thread pane but in the message pane.

Comment 10

[rhp (gone)]

Yes, this is still a problem. I'll look into it.

- rhp

Comment 11

[rhp (gone)]

Actually, this isn't my problem. The message is displayed correctly, but the 
thread pane is messed up. See my attached GIF file for proof :-)

Back to you scott.

- rhp

Comment 12

[rhp (gone)]

Attachment: The problem as I see it...

Comment 13

[scottputterman]

ah sorry.  You are right the summary just refers to the euro symbol in which 
case I prepare to get flamed as I future this.

Comment 14

[scottputterman]

marking nsbeta1+ and moving to mozilla0.8.  Whoever ends up owning this bug, if
it looks like this is going to take more than 1/2 a day, please update the bug.

Comment 15

[scottputterman]

reassigning to sspitzer

Comment 16

[scottputterman]

reassigning to chuang.

Comment 17

[scottputterman]

moving to mozilla0.9

Comment 18

[nhottanscp]

I found this is a generic issue with sending characters which only exists in
Windows-1252 charsets as ISO-8859-1.
The message in the smoketest has a charset ISO-8859-1 and Euro is encoded as
0x80 which is not a valid code point of ISO-8859-1 (it is a valid code point of
Windows-1252).
Using Netscape 6.0, if the user inputs those Windows-1252 specific characters
(Euro, TradeMark, smart quotes, etc...) and sends as ISO-8859-1, the charset
mismatch dialog comes up asking the user to change the charset. If the user
sends as UTF-8 or Windows-1252 which contain Euro in their code point then the
thread pane shows Euro correctly. If the user send it as ISO-8859-1 those
characters are transliterated to ASCII (e.g. EUR, (TM), etc.).

Reassign this to i18n, we need to decide how to support these invalid messages
(generated by 4.x and other mailers).

Comment 19

[Katsuhiko Momoi]

nhotta is looking into the coding details to see
if there is a workable solution. But in the meantime,
we need to consider how important this fix is from
a market point of view. 

1. This problem is caused by Communicator 4.x not doing the 
   right thing when sending the Euro symbol using 8-bit.
   There might be other mailers which do the same thing.
2. Mozilla does the right thing. Should we help delfect
   the legacy clients' defect? 

Factors to consider:

A. There might be other mailers which do the same thing
   as Communicator 4.x. If so what are they and how important
   are they in the market?
B. How widely is Communicator 4.x used in Europe where
   the need to display the EURO is very important?

Depending on the answers to these questions, we might
evaluate this problem accordingly.

Comment 20

[nhottanscp]

I investigated this proeblem and found that ISO-8859-1 decoder actually maps 
0x80 to \u20AC.
The reason it is not shown in the thread pane is related to an optimization made 
for ISO-8859-1. It uses AssignWithConversion() for ISO-8859-1. That works if the 
string is really ISO-8859-1 but not for Windows-1252 characters like Euro sign.
I wrote a patch to do more generic optimization by caching an unicode decoder 
instead of using AssignWithConversin().

Comment 21

[nhottanscp]

Attachment: Patch, cache unicode decoder in nsMimeConverter.

Comment 22

[Jean-Francois Ducarroz]

Looks good. R=ducarroz

Comment 23

[David :Bienvenu]

sr=bienvenu

Comment 24

[(not reading, please use [e-mail] instead)]

is there a mismatched free lying in this code?

You have PR_Malloc() and ToNewUnicode()

do they both use the same allocator?

Comment 25

[nhottanscp]

I will change PR_Malloc to nsMemory::Alloc.

Comment 26

[nhottanscp]

checked in

Comment 27

[ji]

Verified with 2001-02-08 win32, linux and Mac trunk build. It is fixed.