120728 - forward as inline mishandles part of Japanese text body

Status: RESOLVED FIXED

(MailNews Core :: Composition, defect)

Description

[Katsuhiko Momoi]

** Observed with 2002-01-17 Win32 trunk build **

I hope there is a bug for this already. A look through known bugs
did not turn up the exact bug like this one.

1. I have a single part plain text msg in Japanese with Content-type
   charset of iso-2022-jp.
2. When I try to forward this message as inlie, some parts of 
    the body text get corrupted. I will attach an imgae of the problem
    and also will attach the test message below.

Comment 1

[Katsuhiko Momoi]

Attachment: A .zip fiel containing the problem message.

Un-archive the zip file and place the resulting mailbox file 
into a mail directory. Use it to make forward as inline function 
under plain text or HTML mail options.

Comment 2

[Katsuhiko Momoi]

Attachment: A .jpg file showing a corruption problem.

It seems that where the text begins to get corrupted differs
whether or not the Compose option is set to HTML or plain text.
It also seems that where certain characters trigger this problem.

Comment 3

[Katsuhiko Momoi]

Is there another bug that deals with this issue? 
If not, someone should look at this seriously. A lot
of users forward messages inline and not being able to do that 
would amount to disabling a basic functionality.
I would like a follow-up as soon as possible.

Comment 4

[nhottanscp]

Does this still happen?

Comment 5

[nhottanscp]

I tested with 4/18 commercial branch. Now it is broken for all the lines.
I will attach a screen shot.
Let me take a look at this.

Comment 6

[nhottanscp]

Attachment: Screen shot, result with 4/18 commercial branch.

Comment 7

[nhottanscp]

*** Bug 134762 has been marked as a duplicate of this bug. ***

Comment 8

[nhottanscp]

According to bug 134762, 03/20 commercial build has the raw JIS display problem.
So far, I have not figured out what kind of message show this problem.

Comment 9

[nhottanscp]

It looks like this is related to the JIS to Unicode converter behavior change
(bug 128580) which was checked in 3/20.
I think the actual problem is in the caller of the converter. Need more
investigation.

Comment 10

[nhottanscp]

I looked at this with Frank and found there is a problem in ISO-2022-JP
converter (bug 138578).
Frank thinks it is too risky to change the converter now, so I am going to try
workaround by the callers.

Comment 11

[Frank Tang]

this is regression from n6.2 and feb 2002 builds. 
This bug impact japanese only because we have a bug in the ISO-2022-JP converter. 
Impact users: Japanese users- 52.1M users 9.2% of total internet users.
chance to hit this bug: HIGH, almost every mail users will use forward as
inline. For all the japanese mail users use forward as inline, they will see
this problem 
This is [adt1] bug since it impact the major market in major function

Comment 12

[nhottanscp]

Attachment: Changed to use a different util function which does not call the converter repeatedly in a loop in order to workaround the converter problem.

Comment 13

[Jean-Francois Ducarroz]

Comment on attachment 80104 [details] [diff] [review]
Changed to use a different util function which does not call the converter repeatedly in a loop in order to workaround the converter problem.

R=ducarroz

Comment 14

[(not reading, please use [e-mail] instead)]

Comment on attachment 80104 [details] [diff] [review]
Changed to use a different util function which does not call the converter repeatedly in a loop in order to workaround the converter problem.

sr=sspitzer

Comment 15

[nhottanscp]

checked in to the trunk, asked branch check in approval 

Comment 16

[ji]

QA contact to myself.

Comment 17

[ji]

It's fixed on 04/23 trunk build. The problem described in bug 134762 is also fixed. 

Comment 18

[nhottanscp]

fixed in the trunk

Comment 19

[Frank Tang]

this patch fix a high visibility problem by using a different function to do the
same thing. This is a work around for a converter problem which are too risky to
change. please adt1.0.0+ it 

nhotta, please send mail to [e-mail] to ask driver to approve it also. 

QA already verify it on the trunk
>------- Additional Comment #17 From [e-mail] 2002-04-23 11:04 -------
>It's fixed on 04/23 trunk build. The problem described in bug 134762 >is also
fixed. 

Comment 20

[ji]

Verified as fixed with 04/23 trunk build.

Comment 21

[scottputterman]

adding adt1.0.0+.  Please check this into the branch as soon as possible and add
the fixed1.0.0 keyword.

Comment 22

[Randell Jesup [:jesup] (needinfo me)]

Comment on attachment 80104 [details] [diff] [review]
Changed to use a different util function which does not call the converter repeatedly in a loop in order to workaround the converter problem.

a=[e-mail] for branch checkin

Comment 23

[nhottanscp]

fixed1.0.0, checked in to the branch

Comment 24

[ji]

Verified as fixed on 4/29 branch build.

Comment 25

[Tomoo Nomura]

It seems to reappear in the latest build, 2002100714.
I can't forward any Japanese messages.

Comment 26

[nhottanscp]

I can reproduce this using 10/21 trunk.

Comment 27

[ji]

Just fyi, it doesn't happen on the latest 1.0 branch build.

Comment 28

[nhottanscp]

This seems to be HTML mail only.

Comment 29

[nhottanscp]

It started from 6/19 (build ID 2002061908).

Comment 30

[nhottanscp]

This is a regression by bug 146584.
There was a change in mimedrft.cpp to escape HTML body. Since ISO-2022-JP data
may contain '<', '>', we cannot apply escaping there. The escape may be applied
safely when the body data is in Unicode.

Comment 31

[nhottanscp]

Attachment: Changed to do UTF-8 conversion right after we get the body.

This is a historical code.
Need to cover testing for bug 66098, bug 117956 also.

Comment 32

[nhottanscp]

Attachment: fixed memory leak

Comment 33

[Jean-Francois Ducarroz]

Comment on attachment 106113 [details] [diff] [review]
fixed memory leak

Can you use PR_Free(mimeCharset) instead of PR_FREEIF(mimeCharset). Except
that, R=ducarroz

Comment 34

[David :Bienvenu]

Comment on attachment 106113 [details] [diff] [review]
fixed memory leak

sr=bienvenu

Comment 35

[nhottanscp]

checked in to the trunk

Comment 36

[Paul Wyskoczka]

--> qa contact [e-mail]