70341 - Non-ASCII attachment filename is not MIME encoded if filename is short, or with no or other extensions.

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect, P2)

Description

[ji]

Build: 02/26 mtrunk Mac build

On Mac, whether the filename is MIME encoded or not depends on the 
filename length. If the Ja filename contains 4 or less Japanese 
chars,the filename won't get MIME encoded. If the filenames contains more chars,
it gets MIME encoded, but not correctly encoded if it is 
sent from an English Mac OS. So the only working condition is to send 
the mail from a Japanese MAC OS and the filename must contains 5 or more
Ja chars.

Steps to reproduce:
1. Launch Mail on JA Mac OS.
2. Send a mail with an JPN attachment filename which contains 4 or less Ja chars.
3. After the mail is received, view the attachment filename in the envelope or
view the message source of the mail, you'll see the attachment filename is not
MIME encoded.
4. Send a mail with an JPN attachment filename which conains 5 or more Ja chars.
This time the filename is MIME encoded.
5. Repeat step 4 on an En Mac OS + Ja language kit, you'll see the filename is
MIME encoded, but not correctly. The filename shows as a meaningless ASCII string.

Comment 1

[ji]

Attachment: A Ja html file with filename containing 4 Ja chars.

Comment 2

[ji]

Attachment: A Ja html file with filename containing 5 chars.

Comment 3

[Jaime Rodriguez, Jr.]

Adding jenm to cc: list.

i18n thinks this is a cosmetic issue for mac users only. Suggest low priority.

Comment 4

[nhottanscp]

How about 6.0 and 6.01, are they working correctly?

Comment 5

[ji]

6.0/6.01 doesn't have this problem.

Comment 6

[nhottanscp]

cc to putterman, ducarroz.
Anything has changed since 6.0 around Macintosh attachment hanlding?

Comment 7

[Jean-Francois Ducarroz]

I have implemented support for AppleDouble!

Comment 8

[nhottanscp]

IQA, could you try if this is reproducible with US system with European
characters then I can reassign to ducarroz?

Comment 9

[marina]

i don't see any difference in files with short or long non-ascii string on Mac
with 2001-04-30 build ( they are both MIME encoded, first message has 4
non-ascii chars for file name , second one has 13 chars for file name) 

Comment 10

[nhottanscp]

Marina, are you saying this is "worksforme"?

Comment 11

[ji]

What kind of filenames have you tried with? European or Japanese or both? Are you on an En Mac OS?

Comment 12

[marina]

as per Naoki's request i tried Europeen chars on US system ( Mac 9.0) with
language packs.

Comment 13

[nhottanscp]

okay, looks like it's Japnese only.

Comment 14

[Frank Tang]

looks like a serious one. leave it to moz0.9.2

Comment 15

[ji]

It's still happening on 06-27-04-0.9.2 Mac build.

Comment 16

[msanz]

frank, is this going to be fixed for RTM?

Comment 17

[nhottanscp]

Attachment: Screen shot of the problem.

Comment 18

[nhottanscp]

There are XP_MAC lines in nsMsgAttachmentHandler.cpp which seems to be doing 
file name creation.
http://lxr.mozilla.org/seamonkey/source/mailnews/compose/src/nsMsgAttachmentHand
ler.cpp#570

I do not see charset conversion there (from OS charset to mail charset), so 
probably we need it there. But I am not sure about why MIME encode is missing 
for Macintosh only.
Anyway, I feel changing the attachment code now is not low risk.
Jean-Francois, what do you think?

Comment 19

[nhottanscp]

I found the reason of this. In the following code "m_real_name" contains MIME 
encoded string and "tempName" contains broken (non MIME) string. In case of 
Macintosh, if the string is more than 31 character length then "tempName" is 
deleted and "m_real_name" won't be overwritten (see line 549). That's why this 
happens when the file name is short.
But the problem should always happen for Windows since "m_real_name" is always 
overwritten by "tempName". Xianglang, could you test this on Windows?

532 tempName = GenerateFileNameFromURI(mURL); // Make it a sane name
533 #ifdef XP_MAC
534 if (tempName && PL_strlen(tempName) >= 31)
535 PR_DELETE(tempName)
536 #endif
537 538 mCompFields = compFields;
539 540 // First, get as file spec and create the stream for the
541 // temp file where we will save this data
542 if ((!tempName) || (!*tempName))
543 mFileSpec = nsMsgCreateTempFileSpec("nsmail.tmp");
544 else
545 mFileSpec = nsMsgCreateTempFileSpec(tempName);
546 547 // Set a sane name for the attachment...
548 if (tempName)
549 m_real_name = PL_strdup(tempName);
550 

http://lxr.mozilla.org/seamonkey/source/mailnews/compose/src/nsMsgAttachmentHand
ler.cpp#521

Comment 20

[ji]

Did more testing on this, here is results:
1. On windows and linux, the Ja attachment file get MIME encoded or not depends 
on the extension of the file, not on the filename length. No matter how long 
the filename is, as long as the extension is not .txt or .html, the filename 
doesn't get MIME encoded. For example, file with .doc, or .blt or w/o any 
extension will be sent out with filename in escaped format.

2. On Mac, it depends on the filename length. Even the file doesn't any 
extension, the system can figure out the file type ( you can see this by looking 
at the icon).

Due to this problem, fix for bug 68993 (unable to send out attachment file with 
filename containing Shift-JIS trail byte 7C) won't work for the file with no 
or other file extensions.

Changed platform to ALL, priority to P1, Severity to critical.
Added nsbeta1 keyword.
Modified the summary accordingly.

Comment 21

[marina]

same is true for Latin-1 names

Comment 22

[ji]

Modified the summary based on Marina's comments. 

Comment 23

[Frank Tang]

too risky and no data lost. mark it as future. 

Comment 24

[ji]

On Windows, for an ASCII atachment file with the filename like test[aaa.doc, the 
filename will show as test%5Baaa.doc after received.

Comment 25

[marina]

i think it might be related to bug # 75449 ( though that bug doesn't deal with
.doc and other file formats)

Comment 26

[nhottanscp]

Attachment: Patch, do not use the file URL directly, unescape and MIME encode file name.

Comment 27

[Jean-Francois Ducarroz]

cc'ing varada

Comment 28

[msanz]

marking pdt+ as per Friday's pdt meeting

Comment 29

[Frank Tang]

nhotta said this one is cosmetic , we can still send the file out but the file
name is broken but no data will lost, the attachement will still be send out and
received correctly. 

On window, this can only be reproduce in the following condiction:

not HTML, not TXT, The extension type is something we don't have with MIME mapping. 

> [e-mail] 2001-06-28 12:37 -------
>Due to this problem, fix for bug 68993 (unable to send out attachment file with 
>filename containing Shift-JIS trail byte 7C) won't work for the file with no 
>or other file extensions.
This assumption is in correct. There are two seperate caused. Nhotta is fixing
68993 now.  I think we need to fix 68993, but not this oen. 

Comment 30

[Frank Tang]

msanz, I want to remove the PDT+ here. The patch have more risk and I think it
is not worthy because it only happen on edge cases. And even it happen, it does
not look that bad. 

Comment 31

[Frank Tang]

remove [PDT+] per ftang/msanz agreement.

Comment 32

[Jean-Francois Ducarroz]

Sorry for the delay. Looks good. R=ducarroz

Comment 33

[[e-mail] (away - not reading bugmail)]

reassigning to myself, nhotta is on sabbatical.

seth: could you sr this? I'd land it for nhotta then...

Comment 34

[Shoji Matsumoto]

see http://www.emaillab.org/essay/japanese-filename.html (in Japanese..)

It is necessary to implement RFC2231 and/or MIME B encoding for non-ascii
attachment filename.
# In Japan, Mime-B is defact (80%). But it is not so good (ref: RFC2047 5).

Comment 35

[Frank Tang]

mark it as nsbranch+ and move to m0.9.4

Comment 36

[(not reading, please use [e-mail] instead)]

sr=sspitzer, but please make sure to test nothing breaks on US-ASCII machines.

Comment 37

[[e-mail] (away - not reading bugmail)]

fix checked in - thanks everyone!

marina, xianglang - could you please watch out for any non-Japanese attachment 
file name regressions?

Comment 38

[ji]

Verified with 08/22 build. It's fixed, and there are no regressions to 
non-Japanese attachment filenames.

Comment 39

[ji]

Marked it as verified.