103313 - Attachment filename in multi-byte language is corrupted, unable to open after received.

Status: VERIFIED FIXED

(MailNews Core :: Simple MAPI, defect, P2)

Description

[ji]

Build: 10/03 MAPI comm. build
OS: Ja XP 

When the document's filename contains non-ASCII chars, I can't send it out using
mapi.

Steps to reproduce:
1. Install Mozilla as the default mail application.
2. Bring up MS Word 
3. Enter something and save the document in a non-ASCII filename, like tëst or a
Japanese filename.
4. On MS Word, select File | Send To | Mail Recipient, mail account logon window
comes up. 
5. Logon with correct username and passwd, the MS Word pops up an error: Word
couldn't send mail because of MAPI failure: "Attachment not found", no mail
compose window comes up.

If I save the doc in an ASCII filename and follow the same scenario, I don't see
this error and I can get a mail compose window with the doc attached.

Comment 1

[ji]

Added intl, nsbranch, nsenterprise keyword.

Comment 2

[Jaime Rodriguez, Jr.]

Can we discuss this one at the PDT meeting today?

Comment 3

[ji]

Same problem with 10/05 branch build.
I'm on Japanese XP with Office 2000 installed.

Comment 4

[marina]

with 10-05 branch build i am able to send doc with Latin-1 file names only,
cyrillic name isn't working. Using WordPad on WinXP.

Comment 5

[msanz]

can you confirm which ones are working?
Latin-1?
Latin-2?
Cyrillic?
CJK?

Comment 6

[ji]

It depends on the platform.
On a US or latin-1 platform, it only works for ASCII or latin-1, but not working
for any other languages.
On a non latin-1 platform, like CJK, Cyrillic, it only works for ASCII, it
doesn't work for any other languages.
So it's totally broken for Central and Eastern European, CJK platforms.

Comment 7

[ji]

CC'ed mscott. 

Comment 8

[Scott MacGregor]

hey Rajiv, can you think of any spots in the mapi code where we may be casting
down from unicode to raw ascii causing us to lose / corrupt the file name?

Comment 9

[msanz]

thanks Xianglan; Tiantian, can you triage the nsbranch nomination?

Comment 10

[Tiantian Kong]

reassign to Rajiv.

Comment 11

[Rajiv Dayal]

There are two things that could be the reason :

1. MS Word / Wordpad is passing the Japenese filename but not setting the flag
MAPI_UNICODE (as defined by MAPI specs) which tells me that the filename is
unicode or ascii.

2. The nsIMsgCompFields expects the attachments in the form of a char *, a
string of file URLs delimited by commas. I use the nsIFile::GetURL function to
get the URL for the file which returns a char *. So maybe this conversion is
creating the problem. I talked to JF regarding this earlier asking if we should
take attachments as 'wstring' rather than char * but he mentioned that since
URLs are always in ascii the GetURL will take care of converting correctly and
it should work fine.

I will try and debug to see what is the FLAG and the filename when the
MAPISendMail function gets it from the MAPI app on a Japenese machine. Shirley
can i use ur japenese machine to do the debugging ? Meanwhile, JF and Scott can
u please give ur thoughts on 2 above.

Comment 12

[Katsuhiko Momoi]

Per rdayal's request, let me add that the problem is also reproducible
on Windows 2000 Professional Edition with the locale default set 
to Japanese. 

In addition,

1. Even if your word file has an ASCII name, if the first line of your
document contains Japanese, then that line will be used as the Subject
of the MAPI mail and that does not show correctly in Japanese/
This will be an additional problem, once you get past the final
name problem. --> see an image which will be attached below.

2. If you select the Japanese-name file in Windows Explorer and send it to
   "Mail Recipient" with the right-mouse click on the file, Netscape 6.2
   mail window does come up with the file attached and the file name correctly
   showing in the attachment window. However, the subject line
   which uses the filename does not show correctly in Japanese.

From 1 & 2, it seems that we are not dealing with the final names properly
via Word application but OK via the OS. In addition we are not dealing with
data passed to the subject line correctly in situation 1 or 2.

Comment 13

[Katsuhiko Momoi]

Attachment: An image of successful MAPI operation but with corrupt subject header (and therefore the window title).

Comment 14

[Katsuhiko Momoi]

The subject line should be identical to the first line
of the Word document, which is shown in Japanese in the 
image.

Comment 15

[Rajiv Dayal]

Thanks Kat for checking this out. - rajiv.

Comment 16

[Katsuhiko Momoi]

Attachment: 3 test files in .zip format.

Comment 17

[Katsuhiko Momoi]

The 3 files in the .zip attachment are as follows:

1. nihongo.doc (filename in ASCII, title in Japanese)
2. nihongo2.doc (filename in ASCII, title in ASCII)
3. "Nihongo".doc (filename in Japanese, title in Japanese)

File 1 should succeed but will have the subject line prolem
File 2 should succeed without a problem.
File 3 currently fails to go though simple MAPI. And will have
 a subject line problem once the 1st problem is resolved.

Comment 18

[Jaime Rodriguez, Jr.]

Marking ETA as 10.10 per PDT Simple MAPI Update.

Comment 19

[Katsuhiko Momoi]

Attachment: Excel and PowerPoint test files in.zip format

Comment 20

[Katsuhiko Momoi]

Per discussio with rdayal,

1. Wordpad has no problem with word doc with JPN filename.
   Wordpad also does not pass the subject/title to MAPI.
  ** To test this, use the Word test files, open them with
     WordPad and try the MAPI function.
 
2. PPT file with JPN filename fails silently.
3. Excel file with JPN filename fails with an error dialog.

The above attachment contains, PPT and Excel test files -- one 
with a JPN name and the other with an ASCII name. (4 files in all.)

Comment 21

[Katsuhiko Momoi]

Attachment: A zip file containing 1 .doc file -- file name in JPN but title in ASCII.

Comment 22

[Hong Kwon]

marking nsenterprise-; will be reevaluated for nsenterprise in future release.

Comment 23

[msanz]

rajiv, is there any evaluation on what it will take to fix this bug? It is
pretty serious

Comment 24

[Tiantian Kong]

Hi, this is a must fix as listed in the meta bug for simple MAPI, bug 91702. 
Rajiv has been actively working on this. Target fix date is 10/10.

Comment 25

[Rajiv Dayal]

Please find below the patch to fix the problem of sending files with filenames
with non Ascii characters, eg : Japenese filenames.

Currently the code converts all the non unicode filenames passed to unicode if
the the MAPI_UNICODE flag is not set and then uses nsIFile Unicode operations
(for code optimization). However the MS Office apps send these non Ascii
filenames as char * and also DONOT set the MAPI_UNICODE flag as specified in the
MAPI specs.


I have modified the code such that it does not convert the filenames and uses
nsIFile non unicode operations if MAPI_UNICODE is not set and uses the nsIFile
unicode operations if MAPI_UNICODE is set.

Comment 26

[Rajiv Dayal]

Attachment: patch with the fix for the problem in sending non Ascii filenames from Word, Excel and Powerpoint

Comment 27

[Rajiv Dayal]

Hi JF and Scott,
Can u please review and super review the patch attached above. 
Thanks,
- Rajiv.

Comment 28

[ji]

Rajiv, does this patch address the subject problem that Kat mentioned?

Comment 29

[Jean-Francois Ducarroz]

Comment on attachment 52925 [details] [diff] [review]
patch with the fix for the problem in sending non Ascii filenames from Word, Excel and Powerpoint

R=ducarroz

Comment 30

[Scott MacGregor]

Comment on attachment 52925 [details] [diff] [review]
patch with the fix for the problem in sending non Ascii filenames from Word, Excel and Powerpoint

sr=mscott

Nice fix Rajiv! In the future can you send me email directly when you want a sr instead of just adding a comment in the bug? I'm more likely to see the email faster.

Comment 31

[Rajiv Dayal]

Attachment: fix for the subject line in non-Ascii charset

Comment 32

[Jaime Rodriguez, Jr.]

pls check in the first patch = PDT+

PDT+ for the 2nd patch pending positive r/sr reviews.

Comment 33

[nhottanscp]

Comment on attachment 52985 [details] [diff] [review]
fix for the subject line in non-Ascii charset

r=nhotta, please fix the indent

Comment 34

[Rajiv Dayal]

Please find above the fix for the subject line not appearing correctly in the
Compose window if the subject line is in non-Ascii charset when sending from MS
Office apps. The code now finds the systems's currently set character set to do
the conversion for the subject (and also for the body) before sending the data
to nsIMsgCompose. 
Thanks very much Naoki for your help and the review. 
Scott, can u please sr the patch. thanks. - rajiv.

Comment 35

[Scott MacGregor]

Comment on attachment 52985 [details] [diff] [review]
fix for the subject line in non-Ascii charset

sr=mscott

Comment 36

[Rajiv Dayal]

checked into the 094 branch.

Comment 37

[ji]

Verified with 10/11 0.9.4 build. It's fixed. 
Added vtrunk keyword.

Comment 38

[ji]

On 01/22 trunk build, the Ja document filename is not shown correctly on mail
compose window. Screenshot to follow.
Reopened the bug.

Comment 39

[ji]

Attachment: A screenshot on a US W2K with default locale set to Japanese. The Ja document filename is corrupted.

Comment 40

[Jaime Rodriguez, Jr.]

did this ever get checked into the trunk? 

Comment 41

[Rajiv Dayal]

the patch (attachment id=52985) has landed on the trunk. The patch (attachment,
 id=52925) is not required as per the bug # 107697.

Comment 42

[Rajiv Dayal]

This could be related to bug # 128154.

Comment 43

[saari (gone)]

ADT needs info., Jeff Hooker, how many people are affected, and what is the
severity? Are you going after international enterprise customers?

Comment 44

[ji]

Please note that with this problem all the non-English spoken users won't be
able to send out MS documents with the filename in their own languages, and we
don't have this problem with NS6.2.1.

Comment 45

[nhottanscp]

Latin1 case mentioned in the original report "tëst" works on my machine.
Used 04/09 commercial trunk on Windows 2000 US with MS Word 2002.


For Japanese, I had the problem, the Japanese file name got corrupted, like this
screen shot,
http://bugzilla.mozilla.org/attachment.cgi?id=66162&action=view
Subject was fine. 

I used test case,
http://bugzilla.mozilla.org/attachment.cgi?id=52315&action=view

Used 04/09 commercial trunk on Windows NT 4 JA with MS Word 2000 JA.


I had a problem after I sent a document in MS Word, I filed as bug 136448, that
is not specific to intl data.

Comment 46

[nhottanscp]

The Japanese test case I mentioned in my last comment, although the file name
was corrupted, the message and the attached file were sent. I was able to open
the recieved attachment.

Comment 47

[ji]

Yes, I'm also able to send it out. I should have modified the summary when
reopening the bug, sorry.
On my Simplified Chinese Windows XP with Simplified Chinese Office XP installed,
I'm unable to open the attachment with a corrupted Chinese filename after
received. After I click on Ok button on the download dialog with "Open using
Word" selected, MS Word doesn't launch. I don't have this problem with another
mail which has the exactly same attachment file but the attachment filename is
an ascii string.

Comment 48

[Rajiv Dayal]

There have been changes in MAPI code to deal with changes in the Msg Compose
code for trunk landing, at one place comparision of non Unicode platform charset
strings is not done correctly. Please find a fix below.

Comment 49

[Rajiv Dayal]

Attachment: fix for the updated trunk code

Comment 50

[nhottanscp]

Comment on attachment 79743 [details] [diff] [review]
fix for the updated trunk code 

r=nhotta

Comment 51

[Scott MacGregor]

Comment on attachment 79743 [details] [diff] [review]
fix for the updated trunk code 

sr=mscott

Comment 52

[Frank Tang]

this looks like an [adt1] to me. The Office will default any document they
created to localized filename, so it is very common for non english user to use
non ascii file name. It looks like this bug will cause mapi useless outside us
market. 

Comment 53

[Rajiv Dayal]

Thank you very much Naoki and Scott for your r and sr.

Latest patch (id=79743), fix for the regression, checked in to the trunk.
Marking as fixed.

I agree with Frank, this definitly should go in to the branch in order for non
English users to use MAPI from MAPI client apps like MS Office, etc.
Nominating for 1.0 branch checkin.

Comment 54

[Jaime Rodriguez, Jr.]

Pls check this into the trunk. Once ji has verified the fix, and that there are
no related regressions, we will consider it for the 1.0 branch.

Comment 55

[ji]

Verfied as fixed with 04/19 trunk build. The multi-byte attachment filename 
shows correctly and I can open it after received.

Comment 56

[Jaime Rodriguez, Jr.]

Ji - Pls mark as verified, when you have verified the fix on the trunk.

adt1.0.0+ (on ADT's behalf) approval for checking into the 1.0 branch. Pls check
this in today, then add the fixed1.0.0 keyword.

Comment 57

[Randell Jesup [:jesup] (needinfo me)]

a=[e-mail] for branch checkin

Comment 58

[Rajiv Dayal]

has been checked into the branch yesterday, should be in today's build. Marked
fixed for the branch.

Comment 59

[ji]

Verified as fixed on 04/29 branch build.