Closed Bug 263797 Opened 20 years ago Closed 20 years ago

Can't forward a long Japanese plain text message in ISO-2022-JP correctly: body text is treated just as ASCII

Categories

(MailNews Core :: Internationalization, defect)

x86
All
defect
Not set
major

Tracking

(Not tracked)

VERIFIED FIXED

People

(Reporter: nomura, Assigned: jshin1987)

References

Details

(Keywords: regression)

Attachments

(2 files)

User-Agent:       Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a4) Gecko/20040927
Build Identifier: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a4) Gecko/20040927

When replying to Japanese message in line, mozilla can't show the message
correctly. Is this the same as #66098 ?

Reproducible: Always
Steps to Reproduce:
1. unzip the attached file. 
2. copy it to mozilla Mail folder
3. launch mozilla
4. choose the message in test1 folder.
5. click "forward inline" then you will see non Japanese character, may be 8th
bit has been dropeped.
Actual Results:  
Akiko Matusima wrote:

>$B!c!c!c(BKissNet$B%K%e!<%9!!30It5!4X>pJs!!(B2004$B!!(BOct,07$B!d!d!d(B
>
>$B4X@>#I#T6&F1BN!J#K#I#S#S!K(B
>$B2q0w3F0L(B
>
>$B2<5-#27o$N>pJs$r$4;29M$^$G$K$*CN$i$;CW$7$^$9!#!c#K#I#S#S;vL36I!d(B
>$B(,"##I#N#D#E#X"#(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(,(B
>
>$B#1!%#C#R#M6(5D2q4X@>K\It!!Bh#22sL\%;%_%J!<(B
>$B#2!%Bh#22s(B $B%/%j%(%$%7%g%s!&%3%"ElBg:e!V$b$N$E$/$j3W?7=N!W(B

Expected Results:  
Akiko Matusima wrote:


>><<<KissNetニュース 外部機関情報 2004 Oct,07>>>
>>
>>関西IT共同体(KISS)
>>会員各位
>>
>>下記2件の情報をご参考までにお知らせ致します。<KISS事務局>
>>━■INDEX■━━━━━━━━━━━━━━━━━━━━━━━━
>>
>>

Is this related to #263254 ?
Attached file test1 forwared
This is tha same as #263254 Attachment No.1
> Is this related to #263254 ?

Maybe, but this one is easy to reproduce.
I get the same result whether my composition preference is set to ISO-2022-JP, 
or to ISO-8859-1 with the "Always use this encoding" preference checked or 
unchecked.  In each case, the actual encoding used is ISO-2022-JP: if pref is 
set to 8859, the titlebar shows "ISO-2022-JP" but the Options|Encoding flyout 
menu does not show any encoding checked; if the pref is set to 2022, the 
titlebar doesn't show any encoding (as expected), and the menu shows 2022 
checked.

However, I have another test message (spam) that matches the test message in 
these ways: text/plain, charset=ISO-2022-JP; Subject header is RFC2047-tagged as 
ISO-2022-JP; From: header is in ASCII -- and differs in that there is no To: 
header, and was sent by (reportedly) Eudora rather than Outlook Express.  That 
message is encoded correctly when forwarded, and also causes no problems in the 
Encoding menu not showing as checked regardless of the preference for 
composition charset.

Reproduced with
  Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.8a5) Gecko/20041005
Does not happen with Mozilla 1.7.2 or TB 0.8.
Status: UNCONFIRMED → NEW
Ever confirmed: true
Keywords: regression
OS: Linux → All
At one point, I was able to reproduce this bug and suspected that one of my
patches resulted in this regression. However, it turned out that my build (with
which I reproduced the bug) was based on an outdated source tree (that didn't
include two latest patches to nsMsgCompose.cpp. I have separate source trees on
two computers and I tested it on the compute with the outdated tree). With the
latest build of my own, I can't reproduce this bug any more. 

Can you try the latest nightly and see if you can still reproduce this bug. 
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8a5) Gecko/20041013 still
reproduces the problem.
Ok. Now I figured out why I wasn't able to reproduce this bug. I was forwarding
an email sent with Mozilla-mail which has

Content-Type: text/....; charset=ISO-2022-JP

while Nomura-san was forwarding (so was I at one point) an email sent with MS OE
which has 

Content-Type: text/....; 
    charset="iso-2022-jp"

This shouldn't make a difference, but somehow it does when we forward such a
message. I have a couple of suspicous spots I'm gonna take a look at.
Assignee: smontagu → jshin
It turned out that it's not the way Content-Type header is written but U+2014 in
your test message that triggered this problem. Our ISO-2022-JP converter regards
U+2014 (EM dash) as covered by ISO-2022-JP. However, iconv in glibc thinks that
it's not covered by ISO-2022-JP.  It seems that we use the native converter
(iconv on Linux) in a place where we're not supposed to. However, I couldn't
figure out where. My brief search turned up nothing. 

In addition, Mike reported that the same thing happened on Windows, too, which
makes my hypothesis less likely to be the case.  
Status: NEW → ASSIGNED
This is file of Drafts folder and contains 2 draft mails I created.
 (1) Subject : Bug 263797 - short message
 (2) Subject : Bug 263797 - long message

Both are plain text mail of "text/plain; iso-2022-jp".
Only single Japanese character is used in these mails, repeated though.
Difference between 2 mail is line number of body text only.

Save this text file in your mail deirectry and restart Mozilla or Thunderbird.
Say file name of "TEST".
You will find mail folder of "TEST". 
Try "Forward" and "Edit as new".
Next problem occurs only on second mail (long mail).
 - ISO-2022-JP is displayed as "ASCII text" when "Forward" and "Edit as new".
Copy 2 mails to Drafts folder then try "Edit Draft".
Next problem occurs only on second mail (long mail).
 - ISO-2022-JP is displayed as "ASCII text" when "Edit Draft"
(In reply to comment #7)
> but U+2014 in your test message that triggered this problem.

Jshin, is it true?
I think this bug can be said as follows ;
 - If iso-2022-jp plain text mail's body size exceeds some limit
   (such as buffer size for code translation)
   no code translation is done on displaying mail,
   or code translation failure is ignored on displaying mail,
   when "Forward", "Edit as new" and "Edit Draft".
Product: MailNews → Core
There is different but similar problem - Bug 269812, "Save As Text" problem when
long non-ascii mail.
Root cause of it and this bug seems to be same.

Adding descrption in summary for ease of search.
Summary: Can't forward Japanese message correctly → Can't forward Japanese message correctly(when long non-ascii text mail, body text is treated as 8bit ascii)
This doesn't happen with Japanese messages in Shift_JIS or EUC-JP, does it? 
Summary: Can't forward Japanese message correctly(when long non-ascii text mail, body text is treated as 8bit ascii) → Can't forward a long Japanese plain text message in ISO-2022-JP correctly: body text is treated just as ASCII
(In reply to comment #11)
> This doesn't happen with Japanese messages in Shift_JIS or EUC-JP, does it?

As you say, this bug's problem didn't occur on "Edit Draft"/"Forward" when
Shift_JIS mail on Japanese MS Windows of standard locale, Japan
(Japanese,Shift_JIS).
(Note :  Bug 269812 occured even when Shift_JIS mail on MS Win-JP) 
Isn't it the same reason as Bug 134762 ?
I reported both of two.
I used to see the same symptom at very early version.
Thanks. It helps ! This may depend on bug 138578. (see also bug 120278)
Depends on: 138578
"Forward"/"Edit as new"/"Edit Draft" worked very well for
ISO-2022-JP,plain/text, long(11KB and 72KB) mail with 2005022405-trunk on
Japanese Win-2K.
Jshin, thanks a lot.

Closing as FIXED (by fix for Bug 138578) and VERIFIED.
Status: ASSIGNED → RESOLVED
Closed: 20 years ago
Resolution: --- → FIXED
Status: RESOLVED → VERIFIED
Product: Core → MailNews Core
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: