128154 - Crash: opening non-ascii attachments

Status: VERIFIED FIXED

(MailNews Core :: Internationalization, defect, P1)

Description

[marina]

*** observed with 2002-02-27 build ***
i tried word and excel files that have cyrillic name, opening those attachments
from the envelope is causing a crash.
Steps to reproduce:
- select a mail where the attachment has a non-ascii name ( i'll attach such
mail later);
- rightclick on the attachment, select open;
//note: the file start opening but the process never ends and if you cancel it
finally it'll crash

Comment 1

[marina]

Attachment: here is a mail with word attchment where the file has a cyrillic name

Comment 2

[marina]

i have to add that it doesn't happen with the ascii attachments and might be dur
to a recent fix for attachment, cc'ing jean-francois and scott

Comment 3

[scottputterman]

Do you have a talkback report?

Comment 4

[marina]

somehow that talkback didn't come up with this build

Comment 5

[marina]

could that be related to fixes that went into bugs ##111985,127066 and similar
that had attachment problems?

Comment 6

[nhottanscp]

Marina, could you rename the file to ASCII file name try againa? 
This way, we can see if the crash is related to the Cyrillic file name or the
file contents.

Comment 7

[marina]

Naoki, it is not related to file contents because i have a doc file with
cyrillic contents that bears an ascii name and i can open it without a crash ( i
forgot to mention it in my first comments)

Comment 8

[nhottanscp]

Did you use English Windows?
I think non ASCII file name only supported using with localized systems (e.g. JA
file name on JA Windows), althouth crash is not good.

Comment 9

[marina]

yes, i am on EN system. the behavior of 6.2.1 is different: i am not able to
open such files (with names different from the sytem encoding)  but i don't crash.

Comment 10

[nhottanscp]

Could you also try other non ASCII like JA on US System?

Comment 11

[marina]

yes, same happens with the mail that has ja attachment ( attachment with ja
name):it crashes on EN win2k

Comment 12

[nhottanscp]

I was able to open the attachment, it was opened by MS Word.
I used my debug build on Windows 2000 US.

Comment 13

[marina]

hmm, i still can repro this : when the dlgbox comes with a prompt to open or
save the attachment, clicking on Open will try to bring up MS Word ( with all
other file names it does come up and open the attachment but not with chars
different from the system encoding)

Comment 14

[nhottanscp]

I used today's commercial trunk and does not crash
Version of MS Word, mime is XP, Marina's is 97, that might be related.

Comment 15

[Frank Tang]

nsbeta1+ , give to simon to take a look at this. simon: nhotta's load is too heavy
pleas help
save that attachement into a file and put into your mail directory and open it
to try to crash it.
You need to install office 97 first. work with marina. thanks

Comment 16

[Frank Tang]

this could be a MAPI issue. putterman- who should we cc about this bug about
MAPI issue?

Comment 17

[Simon Montagu :smontagu]

Do we know which version of Word the file was saved from? Word 97 crashes
routinely when opening files saved in Word 2000.

Comment 18

[scottputterman]

rdayal handles MAPI.

Comment 19

[Rajiv Dayal]

When you open an attachment from Compose, MAPI is not used. MAPI is used when
you try to send from MS Office apps or other MAPI clients using Mozilla Mail as
default

However there is a related bug in MAPI too, bug # 103313. I think fixing this
may fix that one too.

Comment 20

[ji]

I'm confused, in bug 103313, Ja filename is not parsed correctly to mozilla, and
it's using MAPI as you said. But this one is to open a MS attachment from Mail,
it's not using MAPI, how are they related?

Comment 21

[Rajiv Dayal]

MAPI code sends the attachment file to Msg Compose, if Msg Compose handling of
non Ascii file has some problem the MAPI problem 'may' be related to this one. 

Comment 22

[Simon Montagu :smontagu]

-> ftang, because I have no Office 97 to test this on

Comment 23

[Shanjian Li]

take the bug for now. I will look at it tomorrow. 

Comment 24

[Shanjian Li]

I could not reproduce the bug, but I guess this might be the dup of 128825. It
is very likely to be caused by patch for 102595. Patch for 102595 was checked in
on 1/14/2002. Marina, could you verify this? Get a build before 1/14/2002 and a
build after 1/14/2002, you should see the difference if that is the case. 

BTW, could you also forward the problematic email to my mailbox directly, ie.
[e-mail]

Comment 25

[Shanjian Li]

I tried this on win2000 with the mail marina sent to me, but failed to reproduce 
the problem. However, I still believe it is a dup of 128825. Hopefully marina 
can prove that this is a regression of 102595.

Comment 26

[marina]

i am still crashing with 2002-03-12 build on win2k, Shianjan, what version of MS
Word are you using?Thanks. 

Comment 27

[Shanjian Li]

I am using word97. But I don't think this has anything to do with word. It seems
like the problem is in somewhere else. marina, could you check tomorrow's build
to make for sure? I will take a look at it tomorrow. 

Comment 28

[Shanjian Li]

I reproduced the problem just now. I am not exactly sure if this is the same 
crash. It happens inside java script.  

_NMSG_WRITE(int 10) line 221
abort() line 44 + 7 bytes
JS_Assert(const char * 0x00f64ac0, const char * 0x00f64aa4, int 1095) line 175
js_AddScopeProperty(JSContext * 0x02efc8f8, JSScope * 0x0539aa00, long 87082176, 
int (JSContext *, JSObject *, long, long *)* 0x05196c80, int (JSContext *, 
JSObject *, long, long *)* 0x05196cd0, unsigned long 19, unsigned int 49, 
unsigned int 0, int 0) line 1095 + 28 bytes
js_ChangeScopePropertyAttrs(JSContext * 0x02efc8f8, JSScope * 0x0539aa00, 
JSScopeProperty * 0x052efee0, unsigned int 49, unsigned int 17, int (JSContext 
*, JSObject *, long, long *)* 0x05196c80, int (JSContext *, JSObject *, long, 
long *)* 0x05196cd0) line 1241 + 53 bytes
js_DefineNativeProperty(JSContext * 0x02efc8f8, JSObject * 0x051970f8, long 
87082176, long -2147483647, int (JSContext *, JSObject *, long, long *)* 
0x00000000, int (JSContext *, JSObject *, long, long *)* 0x05196cd0, unsigned 
int 33, unsigned int 0, int 0, JSProperty * * 0x00000000) line 2022 + 94 bytes
js_DefineProperty(JSContext * 0x02efc8f8, JSObject * 0x051970f8, long 87082176, 
long -2147483647, int (JSContext *, JSObject *, long, long *)* 0x00000000, int 
(JSContext *, JSObject *, long, long *)* 0x05196cd0, unsigned int 33, JSProperty 
* * 0x00000000) line 1975 + 41 bytes
js_Interpret(JSContext * 0x02efc8f8, long * 0x0012d530) line 3584 + 57 bytes
js_Execute(JSContext * 0x02efc8f8, JSObject * 0x05196878, JSScript * 0x0539cd00, 
JSStackFrame * 0x00000000, unsigned int 0, long * 0x0012d530) line 968 + 13 
bytes
JS_ExecuteScript(JSContext * 0x02efc8f8, JSObject * 0x05196878, JSScript * 
0x0539cd00, long * 0x0012d530) line 3258 + 25 bytes
mozJSComponentLoader::GlobalForLocation(const char * 0x00d1d718, nsIFile * 
0x052e1a58) line 1202 + 27 bytes
mozJSComponentLoader::ModuleForLocation(const char * 0x00d1d718, nsIFile * 
0x00000000) line 1000 + 19 bytes
mozJSComponentLoader::GetFactory(mozJSComponentLoader * const 0x00dc8080, const 
nsID & {...}, const char * 0x00d1d718, const char * 0x00d1aad8, nsIFactory * * 
0x0012d6b8) line 462 + 14 bytes
nsFactoryEntry::GetFactory(nsIFactory * * 0x0012d6b8, nsComponentManagerImpl * 
0x008ad300) line 326 + 58 bytes
nsComponentManagerImpl::FindFactory(const char * 0x019044c4, nsIFactory * * 
0x0012d6b8) line 1628
nsComponentManagerImpl::CreateInstanceByContractID(nsComponentManagerImpl * 
const 0x008ad300, const char * 0x019044c4, nsISupports * 0x00000000, const nsID 
& {...}, void * * 0x0012d714) line 1857 + 16 bytes
nsComponentManager::CreateInstance(const char * 0x019044c4, nsISupports * 
0x00000000, const nsID & {...}, void * * 0x0012d714) line 115
nsCreateInstanceByContractID::operator()(const nsID & {...}, void * * 
0x0012d714) line 151 + 27 bytes
nsCOMPtr<nsIProgressDialog>::assign_from_helper(const nsCOMPtr_helper & {...}, 
const nsID & {...}) line 972 + 18 bytes
nsCOMPtr<nsIProgressDialog>::nsCOMPtr<nsIProgressDialog>(const nsCOMPtr_helper & 
{...}) line 554
nsExternalAppHandler::ShowProgressDialog() line 1390
nsExternalAppHandler::LaunchWithApplication(nsExternalAppHandler * const 
0x04efa31c, nsIFile * 0x00000000, int 0) line 1642
XPTC_InvokeByIndex(nsISupports * 0x04efa31c, unsigned int 7, unsigned int 2, 
nsXPTCVariant * 0x0012d974) line 106
XPCWrappedNative::CallMethod(XPCCallContext & {...}, XPCWrappedNative::CallMode 
CALL_METHOD) line 2025 + 42 bytes
XPC_WN_CallMethod(JSContext * 0x04f09ae8, JSObject * 0x04e57788, unsigned int 2, 
long * 0x053741b8, long * 0x0012dc24) line 1266 + 14 bytes
js_Invoke(JSContext * 0x04f09ae8, unsigned int 2, unsigned int 0) line 788 + 23 
bytes
js_Interpret(JSContext * 0x04f09ae8, long * 0x0012e544) line 2745 + 15 bytes
js_Invoke(JSContext * 0x04f09ae8, unsigned int 1, unsigned int 2) line 805 + 13 
bytes
js_InternalInvoke(JSContext * 0x04f09ae8, JSObject * 0x04e583b0, long 85550400, 
unsigned int 0, unsigned int 1, long * 0x0012e7a4, long * 0x0012e674) line 880 + 
20 bytes
JS_CallFunctionValue(JSContext * 0x04f09ae8, JSObject * 0x04e583b0, long 
85550400, unsigned int 1, long * 0x0012e7a4, long * 0x0012e674) line 3412 + 31 
bytes
nsJSContext::CallEventHandler(nsJSContext * const 0x051950e0, void * 0x04e583b0, 
void * 0x05196540, unsigned int 1, void * 0x0012e7a4, int * 0x0012e7a8, int 0) 
line 1016 + 33 bytes
nsJSEventListener::HandleEvent(nsJSEventListener * const 0x05329840, nsIDOMEvent 
* 0x03e86898) line 180 + 77 bytes
nsEventListenerManager::HandleEventSubType(nsListenerStruct * 0x053298a8, 
nsIDOMEvent * 0x03e86898, nsIDOMEventTarget * 0x053296e0, unsigned int 8, 
unsigned int 7) line 1217 + 20 bytes
nsEventListenerManager::HandleEvent(nsEventListenerManager * const 0x053297c8, 
nsIPresContext * 0x05313898, nsEvent * 0x0012f2c4, nsIDOMEvent * * 0x0012f178, 
nsIDOMEventTarget * 0x053296e0, unsigned int 7, nsEventStatus * 0x0012f310) line 
2207 + 36 bytes
nsXULElement::HandleDOMEvent(nsXULElement * const 0x053296d8, nsIPresContext * 
0x05313898, nsEvent * 0x0012f2c4, nsIDOMEvent * * 0x0012f178, unsigned int 1, 
nsEventStatus * 0x0012f310) line 3457
PresShell::HandleDOMEventWithTarget(PresShell * const 0x05314ce0, nsIContent * 
0x053296d8, nsEvent * 0x0012f2c4, nsEventStatus * 0x0012f310) line 6101 + 36 
bytes
nsButtonBoxFrame::MouseClicked(nsIPresContext * 0x05313898, nsGUIEvent * 
0x0012f4c0) line 195
nsButtonBoxFrame::HandleEvent(nsButtonBoxFrame * const 0x05361fdc, 
nsIPresContext * 0x05313898, nsGUIEvent * 0x0012f4c0, nsEventStatus * 
0x0012f79c) line 142
PresShell::HandleEventInternal(nsEvent * 0x0012f4c0, nsIView * 0x00000000, 
unsigned int 1, nsEventStatus * 0x0012f79c) line 6069 + 38 bytes
PresShell::HandleEventWithTarget(PresShell * const 0x05314ce0, nsEvent * 
0x0012f4c0, nsIFrame * 0x05361fdc, nsIContent * 0x053296d8, unsigned int 1, 
nsEventStatus * 0x0012f79c) line 6023 + 22 bytes
nsEventStateManager::CheckForAndDispatchClick(nsEventStateManager * const 
0x0532cd10, nsIPresContext * 0x05313898, nsMouseEvent * 0x0012f98c, 
nsEventStatus * 0x0012f79c) line 2602 + 63 bytes
nsEventStateManager::PostHandleEvent(nsEventStateManager * const 0x0532cd18, 
nsIPresContext * 0x05313898, nsEvent * 0x0012f98c, nsIFrame * 0x05361fdc, 
nsEventStatus * 0x0012f79c, nsIView * 0x053147c8) line 1684 + 28 bytes
PresShell::HandleEventInternal(nsEvent * 0x0012f98c, nsIView * 0x053147c8, 
unsigned int 1, nsEventStatus * 0x0012f79c) line 6074 + 43 bytes
PresShell::HandleEvent(PresShell * const 0x05314ce4, nsIView * 0x053147c8, 
nsGUIEvent * 0x0012f98c, nsEventStatus * 0x0012f79c, int 1, int & 1) line 5977 + 
25 bytes
nsViewManager::HandleEvent(nsView * 0x053147c8, nsGUIEvent * 0x0012f98c, int 1) 
line 2043
nsView::HandleEvent(nsViewManager * 0x05314548, nsGUIEvent * 0x0012f98c, int 1) 
line 306
nsViewManager::DispatchEvent(nsViewManager * const 0x05314548, nsGUIEvent * 
0x0012f98c, nsEventStatus * 0x0012f88c) line 1857 + 23 bytes
HandleEvent(nsGUIEvent * 0x0012f98c) line 83
nsWindow::DispatchEvent(nsWindow * const 0x05314874, nsGUIEvent * 0x0012f98c, 
nsEventStatus & nsEventStatus_eIgnore) line 865 + 10 bytes
nsWindow::DispatchWindowEvent(nsGUIEvent * 0x0012f98c) line 886
nsWindow::DispatchMouseEvent(unsigned int 301, unsigned int 0, nsPoint * 
0x00000000) line 4711 + 21 bytes
ChildWindow::DispatchMouseEvent(unsigned int 301, unsigned int 0, nsPoint * 
0x00000000) line 4963
nsWindow::ProcessMessage(unsigned int 514, unsigned int 0, long 11927854, long * 
0x0012fda8) line 3596 + 28 bytes
nsWindow::WindowProc(HWND__ * 0x0028092c, unsigned int 514, unsigned int 0, long 
11927854) line 1130 + 27 bytes
USER32! 77e71820()

Comment 29

[Frank Tang]

marina, can you make a screen shot of the dialog box before the crash. That will
help us locate the js code related to it. thanks

Comment 30

[marina]

Attachment: this is the only screen shot that i was able to get while system freezes, the one that would point to the path of the doc that the program tries to load didn't come up

Comment 31

[Frank Tang]

marina, can you make the screen shot BEFORE it will freeze. I want to see what
kind of UI Gecko show so I can find the related code easier. 

Comment 32

[Frank Tang]

shanjian wrote:
I could reproduce the problem once, but then it was OK again. It should not 
have anything to do with word97. The problem is most likely to
be in nsProgressDialog.js. 

Comment 33

[Frank Tang]

shanjian is on vacation, roy- can you take a look at this 

Comment 34

[marina]

Attachment: here is a screen shot of the dlgbox before freeze/crash

Comment 35

[Roy Yokoyama]

Taking and setting the Milestone.
(It looks like this bug went around the entire i18n team members)

Comment 36

[Roy Yokoyama]

Though, I failed to reproduce this bug,
it asserts at nsRDFXMLSerializer::SerializeDescription() in 
nsRDFXMLSerializer.cpp. 
>    const char* s;
>    rv = aResource->GetValueConst(&s);
>    if (NS_FAILED(rv)) return rv;
>
>    nsAutoString uri = NS_ConvertUTF8toUCS2(s);  <== HERE !!

s contains DBCS string; not UTF8 encoded string.

Comment 37

[Roy Yokoyama]

marina: can you email me ([e-mail]) 
with the  non-ascii name document attached?

Comment 38

[Roy Yokoyama]

Marina: thanks for the email.  I was hoping I can reproduce this bug; but
no cigar. :(  
Is this one of bugs that only reproducable in NS commercial?
I don't think so because shanjian was able to reproducet it somehow.
I am going to pull NS tree to see if it makes easiler to reproduce.

Comment 39

[Roy Yokoyama]

marina and I tried this in her machine. 

We found that we have to manually set
the encoding to Cyrillic from the CharEncoding menu
before Opening the DOC in order for this to _crash_.
If you don't, then it won't crash.

Unfortunately, I still can't reproduce this in
my machine.  However, from above test, the Cyrillic
Decoder (may be Encoder too) potentially the cause
of this problem. 

Comment 40

[Frank Tang]

adt1 since this is a crasher. P1

Comment 41

[marina]

On czeck NT40 i am not crashing but instead of opening the file
(rightclick;chose Open menu item) it is saving it to the disk. I can not open it
at all either i change the encoding to cyrillic or leave it as it is (nomime header)

Comment 42

[Roy Yokoyama]

shanjian: you were able to reproduce this once.  What was your OS system 
language and other settings?  I just can't reproducet this in my machine.
(I changed OS system default locales, email locales, etc)

Comment 43

[Roy Yokoyama]

I can't reproduce it.
Send back to the original owner who may have better luck than me.

Comment 44

[Shanjian Li]

I could not reproduce it any more. The progress dialog still does not work properly, but 
crash does not happen. Reset TM to 1.01

Comment 45

[Jaime Rodriguez, Jr.]

Agree this is a severe bug, but it sounds like it only affects Marina at this
time. Changing to ADT3, because this sounds like it is only happening on one
machine, which requires manually setting the encoding.

Comment 46

[Frank Tang]

Impact Summery
Impact Platform: Window only
Impact language users: 560M 100%
Probability of hitting the problem: LOW, currently only marina can reproduce this.
Severity if hit the problem in the worst case: crash
Way of recover after hit the problem: None
Risk of the fix: Unknown
Potential benefit of fix this problem: unknown
Suggest: adt2

Comment 47

[Kasumi]

I couldn't reproduce it on 04-01-trunk.
OS = Win XP Pro. JA, Mac OS X(10.1), 9.1 JA.
using file types are: doc, txt, jpg, html having Japanese file name

Comment 48

[Frank Tang]

this is a window only bug, please don't waste your time on non window platform.
thanks

Comment 49

[marina]

Kasumi, you need a version of MSword to be different from your system locale ( i
have an attachment with cyrillic name of the  msword file that could be only
produced on the russian system and i am trying to open it on the english system) 

Comment 50

[Shanjian Li]

reassign to roy. I could not reproduce the problem with any of my system. On my winnt4.0(en),
if the charset used to interprete attachment filename is acceptable to system, (like iso-8859-1, 
you can use forced charset to change it). it will be openned correctly. If not, nothing displayed 
but there is not crash or freeze either. In any case, I am allowed to save it with a name I choose. 

Roy, could you try out with win2k(en)? You might consider reassign to simon if you don't have time. 
It should have nothing to do with MS Office. Without ms office installed, wordpad will be used to 
open the file, but the result should be the same. 

Comment 51

[Frank Tang]

marina, spam that problem mail to i18ngrp, thanks

Comment 52

[Roy Yokoyama]

resetting the milestone to 1.0.0 for keeping it under my radar.

Comment 53

[marina]

actually, i just tried another attachment with .zip file that has a cyrillic
name and it crashes as well.i'll attach the mail, maybe you'll be luckier to
reproduce this one.

Comment 54

[marina]

Attachment: here is a mail with zip file as an attachment that has a cyrillic name

Comment 55

[Roy Yokoyama]

I am trying this in my dev machine (W2K-ja). I don't have en machine.
(how sad....)
I used Korean files and marina's email.  They both don't freeze/crash.
One different behavior between Korean and Cyrillic file:
- Cyrillic brings up MS Word; but Korean doesn't
  (both have doc. extension)

Note: Double clicking <Korean>.doc file from File Explorer
      will launch MS Word.

Comment 56

[marina]

Roy, did you try the mail with zip file?

Comment 57

[Roy Yokoyama]

>Roy, did you try the mail with zip file?
No, but I had <korean>.zip.
I am not sure how to make your zip attachment to simulate this case.
It may be better if you could email your new zip file directly.

Comment 58

[Roy Yokoyama]

Any other QA can reproduce this bug?

Comment 59

[marina]

Roy, i was able to reproduce the crash on Teruko's machine with 04-05-0.9.9
build, win2K english

Comment 60

[marina]

I am experiencing the same crash with double-byte file names ( euc-kr, for ex.)
on  win2k english system with 2002-04-09-03 build

Comment 61

[marina]

Attachment: korean doc file

Comment 62

[Frank Tang]

give to shanjian for the last try

Comment 63

[Shanjian Li]

I could not reproduce the crash, but I believe I find out the cause of the freeze. 
The default character in unicode to file system charset conversion is '?', and
'?' character is illegal in as file name. When we try to create locale file, we
try to append a number from 1 to 1000 in order to find a unique name. But
because of the '?', this will always fail. The loop may freeze the application
for a little while in certain situation. Another problem is that the attachment
won't be able to be launched. 

Comment 64

[Shanjian Li]

Attachment: patch

Comment 65

[Shanjian Li]

ftang/alecf, could you give r/sr?

Comment 66

[Alec Flett]

Comment on attachment 79303 [details] [diff] [review]
patch

can that "defaultChar" be a const?
if it can, make it so. if not, never mind.
sr=alecf with whichever of the above must be done.

Comment 67

[Frank Tang]

Comment on attachment 79303 [details] [diff] [review]
patch

r=ftang

Comment 68

[Frank Tang]

although the patch touch the part which all the file open/create on windows. The
effect won't happen unless the file name contains non ASCII characters.
therefore, the risk of the patch will only impact Window file operation with non
ASCII file name and have smaller impact to English users if the patch have proble.

Shanjian- what will happe if the mailer send out mail and put "?" as the
attached file name, will we still crash ?

please land to the trunk. and ask [e-mail] for m1.0 check in and adt
for adt1.0.0 check in. 

Comment 69

[Shanjian Li]

>> Shanjian- what will happe if the mailer send out mail and put "?" as the
>> attached file name, will we still crash ?
That's theoritically possible, but I don't know how user will generate such a
file in real world situation. I have been thinking of validating all existing
characters in a filename, but I didn't find a good reason to justify this
effort. BTW, 
it is not a crash, the worst case I can expect is several minute of freeze.

Comment 70

[Shanjian Li]

Attachment: update patch

Comment 71

[Asa Dotzler [:asa]]

Comment on attachment 79354 [details] [diff] [review]
update patch

a=asa (on behalf of drivers) for checkin to the 1.0 branch

Comment 72

[Jaime Rodriguez, Jr.]

Please check this onto the trunk and update the bug when it's been tested.

Comment 73

[Shanjian Li]

fix checked into trunk.

Comment 74

[marina]

tested with 2002-04-16-03 trunk build, it is fixed!!! Thanks, Shianjan in
detecting and fixing this. I used four attachments and succeessfully viewed all
of them.

Comment 75

[Frank Tang]

adt1.0.0+ per adt meeting. shanjian, please land into m1.0 branch asap. thanks

Comment 76

[Shanjian Li]

fix checked into branch. 

Comment 77

[Asa Dotzler [:asa]]

adding fixed1.0.0 keyword (branch resolution). This bug has comments saying it
was fixed on the 1.0 branch and a bonsai checkin comment that agrees. To verify
the bug has been fixed on the 1.0 branch please replace the fixed1.0.0 keyword
with verified1.0.0.

Comment 78

[marina]

verified with 2002-05-28-1.0.0 build