107074 - M095 N620 Trunk Crash on Browse Binary Page [@ nsShiftJISToUnicode::Convert]

Status: VERIFIED FIXED

(Core :: Internationalization, defect)

Description

[Yuki]

Browser smetimes crash when I browese *.dat or *.txt files which are broken.

Talk Back 37246490Z
From 9.5
To 2001102603/Win2k

Comment 1

[Alex Vincent [:WeirdAl]]

Reporter:  Can you give us some additional info? 
http://www.mozilla.org/quality/bug-writing-guidelines.html

Comment 2

[Yuki]

Mozilla force to display binarry page that example url input with manual.

Crash sometimes happens with unknown conditions,which are windows size or 
Page contet or size(?)

Comment 3

[Koike Kazuhiko]

2001102808/Win2k crashed.
http://homepage2.nifty.com/aqua_magius/m5.dat

Comment 4

[Asa Dotzler [:asa]]

Stack Trace
nsShiftJISToUnicode::Convert
[d:\builds\seamonkey\mozilla\intl\uconv\ucvja\nsJapaneseToUnicode.cpp, line 132]
nsScanner::Append [d:\builds\seamonkey\mozilla\htmlparser\src\nsScanner.cpp,
line 345]
ParserWriteFunc [d:\builds\seamonkey\mozilla\htmlparser\src\nsParser.cpp, line 2425]
nsPipe::nsPipeInputStream::ReadSegments
[d:\builds\seamonkey\mozilla\xpcom\io\nsPipe2.cpp, line 427]
nsParser::OnDataAvailable
[d:\builds\seamonkey\mozilla\htmlparser\src\nsParser.cpp, line 2482]
nsDocumentOpenInfo::OnDataAvailable
[d:\builds\seamonkey\mozilla\uriloader\base\nsURILoader.cpp, line 260]
nsHttpChannel::OnDataAvailable
[d:\builds\seamonkey\mozilla\netwerk\protocol\http\src\nsHttpChannel.cpp, line 2361]
nsOnDataAvailableEvent::HandleEvent
[d:\builds\seamonkey\mozilla\netwerk\base\src\nsStreamListenerProxy.cpp, line 203]
PL_HandleEvent [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c, line 591]
PL_ProcessPendingEvents [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c,
line 524]
_md_EventReceiverProc [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c, line
1072]
nsAppShellService::Run
[d:\builds\seamonkey\mozilla\xpfe\appshell\src\nsAppShellService.cpp, line 303]
main1 [d:\builds\seamonkey\mozilla\xpfe\bootstrap\nsAppRunner.cpp, line 1297]
main [d:\builds\seamonkey\mozilla\xpfe\bootstrap\nsAppRunner.cpp, line 1610]
WinMain [d:\builds\seamonkey\mozilla\xpfe\bootstrap\nsAppRunner.cpp, line 1628]
WinMainCRTStartup()
KERNEL32.dll + 0x17d08 (0x77e67d08) 

Comment 5

[Roy Yokoyama]

I can't reproduce this. I have 20011025 trunk build. I've asked
iQA to see if she can reproduce this; but not.
I also forced to change the charset to Shift-JIS.

Asa/Yuki: I need more info.
1) OS-language?
2) Character encoding
3) Auto-Detect On/Off?  If On, then which one?

Comment 6

[Yuki]

2001110103/Win95[english] Crashed.

Navigator Language Japanese[EUC-JP]
OS Language is not correlated.
Auto Detect off

Comment 7

[Roy Yokoyama]

I still can't reproduce this on my XP-En.
I pulled new build today.  

teruko/ylong: can you please try it to see if you can reproduce the crash?
yuki: thanks for the info; but what is do you mean by 
     "Navigator Language Japanese[EUC-JP]"?

Comment 8

[Yuki]

In preferences/Navigator/Languages
I set default character Encoding "Japanese [EUC-JP]"

Comment 9

[Yuying Long]

I tried really hard on WinME-Ja, win2k, WinXP with N6.2 and recently trunk
build, I almost couldn't reproduce it, and finally I got one crash, but when I
re-start browser and try again, then there was no crash at same place any more. 

The one crash on WinXP-Ja which looks like the same place as before:
------------------------------------
Incident ID 37568902 
Stack Signature nsShiftJISToUnicode::Convert 66c4457d  
Bug ID  
Trigger Time  2001-11-02 17:36:58  
Email Address  [e-mail]  
URL visited   
User Comments  bug107074  
Build ID 2001102218  
Product ID Netscape6.20  
Platform ID Win32  
Trigger Reason Access violation  
Stack Trace  
nsShiftJISToUnicode::Convert
[d:\builds\seamonkey\mozilla\intl\uconv\ucvja\nsJapaneseToUnicode.cpp, line 116] 
nsScanner::Append [d:\builds\seamonkey\mozilla\htmlparser\src\nsScanner.cpp,
line 329] 
ParserWriteFunc [d:\builds\seamonkey\mozilla\htmlparser\src\nsParser.cpp, line
2628] 
nsInputStreamTee::WriteSegmentFun
[d:\builds\seamonkey\mozilla\xpcom\io\nsInputStreamTee.cpp, line 82] 
nsPipe::nsPipeInputStream::ReadSegments
[d:\builds\seamonkey\mozilla\xpcom\io\nsPipe2.cpp, line 412] 
nsInputStreamTee::ReadSegments
[d:\builds\seamonkey\mozilla\xpcom\io\nsInputStreamTee.cpp, line 138] 
nsParser::OnDataAvailable
[d:\builds\seamonkey\mozilla\htmlparser\src\nsParser.cpp, line 2683] 
nsDocumentOpenInfo::OnDataAvailable
[d:\builds\seamonkey\mozilla\uriloader\base\nsURILoader.cpp, line 244] 
nsStreamListenerTee::OnDataAvailable
[d:\builds\seamonkey\mozilla\netwerk\base\src\nsStreamListenerTee.cpp, line 57] 
nsHttpChannel::OnDataAvailable
[d:\builds\seamonkey\mozilla\netwerk\protocol\http\src\nsHttpChannel.cpp, line
2226] 
nsOnDataAvailableEvent::HandleEvent
[d:\builds\seamonkey\mozilla\netwerk\base\src\nsStreamListenerProxy.cpp, line 188] 
PL_HandleEvent [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c, line 591] 
PL_ProcessPendingEvents [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c,
line 524] 
_md_EventReceiverProc [d:\builds\seamonkey\mozilla\xpcom\threads\plevent.c, line
1072] 
USER32.dll + 0x3c076 (0x77d2c076) 
USER32.dll + 0x3c076 (0x77d2c076) 
netscp6.exe + 0x6770 (0x00406770) 
kernel32.dll + 0x3bb86 (0x77e5bb86) 

Comment 10

[Roy Yokoyama]

Charset menu is completely none-functional
(ie. selecting a page to Shift_JIS doesn't do anything)
Waiting until 106749 gets fixed.

Comment 11

[greer]

Adding info for talkback tracking. This one is on the topcrash lists for M095 
and N620.

Comment 12

[Roy Yokoyama]

I just got a fresh trunk build and I still can't reprodcue this bug.
nsJapaneseToUnicode.cpp, line 116 is
const unsigned char* srcEnd = (unsigned char*)aSrc + *aSrcLen;
which both aSrc and aSrcLen are passed into nsShiftJISToUnicode::Convert()

Does anybody with dev machine can reproduce this?
cc'ing bunch of people.....

Comment 13

[Roy Yokoyama]

ftang/nhotta: can you reproduce this?  It always works for me. :(

Comment 14

[Yuki]

Crashed (2001111903[win95])
TB38261500K

Comment 15

[nhottanscp]

Crash at line 132, cc to [e-mail].
http://lxr.mozilla.org/seamonkey/source/intl/uconv/ucvja/nsJapaneseToUnicode.cpp
#132

128           case 4: // IBM extention to NEC extention
129           {
130             ibmnec += *src;
131             ibmnec = gSjisIBMNECmap[ibmnec - 0xFA40];
132             if ( ibmnec == 0 ) {
133               *dest++ = 0xFFFD;
134               ibmnec = 0;
135               fbibmnec = 0;
136               mState=0;
137               if(dest >= destEnd)
138                 goto error1;
139             } else {
140               *dest++ = gJis0208map[fbIdx[(ibmnec >> 8) & 0x7F ] 
141                                    + sbIdx[ibmnec & 0x00FF]];
142               if(dest >= destEnd)
143                 goto error1;
144               mState = 0;
145               break;
146             }
147           } 

Comment 16

[nhottanscp]

On my debug build, ibmnec was 0xFA39, subtracting 0xFA40 makes negative, the 
result is used for indexing the table which may cause illegal memory access.

ibmnec = gSjisIBMNECmap[ibmnec - 0xFA40];

Comment 17

[nhottanscp]

PRUint16 ibmnec;

This is not initialized, is that okay?

Comment 18

[Shoji Matsumoto]

please modify

128           case 4: // IBM extention to NEC extention
129           {
130             ibmnec += *src;
131             ibmnec = gSjisIBMNECmap[ibmnec - 0xFA40];

to

case 4:
{
  ibmnec += *src;
  if (ibmnec < 0xFA40) {
    ibmnec = 0;
  }
  ibmnec = gSjisIBMNECmap[ibmnec - 0xFA40];

Comment 19

[nhottanscp]

How about possiblity of uninitialized ibmnec?
Is that sure the value is initialized before coming to line 128 or it doesn't
matter as long as the check of (ibmnec < 0xFA40) is done?

Comment 20

[Shoji Matsumoto]

Sorry, it's my mistake.

IBMNEC.map includes charmap from 0xfa40 to 0xfcff.
When 1st byte is 0xfa, 0xfb or 0xfc, 'ibmnec' becomes 0x[1st][2nd].
Then 0xfa00 to 0xfa39 cause negative indexes.

#18 will correct this problem.
When ibmnec=0, converted char becomes 0xFFFD(Undefined).



 

Comment 21

[Shoji Matsumoto]

mState should be '4' -- case:4 -- only when the leading byte is 0xfa, 0xfb or 0xfc.

164 if((0xfa == *src) || (0xfb == *src) || (0xfc == *src)) {
165 ibmnec=((*src) << 8) & 0xFF00;
166 fbibmnec = *src;
167 mState = 4; // IBM Extra

Comment 22

[Roy Yokoyama]

Shoji-san: can you create a patch for us?

Comment 23

[Shoji Matsumoto]

Attachment: fix patch

Comment 24

[nhottanscp]

Comment on attachment 58741 [details] [diff] [review]
fix patch

r=nhotta

Comment 25

[Roy Yokoyama]

shojis-san: please request for /sr.
Let me know if you need help.

--assigning to [e-mail]

Comment 26

[Shoji Matsumoto]

Attachment: fix patch2 (58741 is bad)

Comment 27

[nhottanscp]

Comment on attachment 58785 [details] [diff] [review]
fix patch2 (58741 is bad)

r=nhotta

Comment 28

[nhottanscp]

I think this is needed for 0.9.7.
Matsumoto san, please get 'sr='.
Please see the detail about super review in here.
http://www.mozilla.org/hacking/reviewers.html

Comment 29

[Christopher Blizzard (:blizzard)]

Add a comment to this code, please:

+            if (ibmnec < 0xFA40 || ibmnec > 0xFCFF) {

and

+              ibmnec = gSjisIBMNECmap[ibmnec - 0xFA40];

and you have an sr=blizzard

Comment 30

[Shoji Matsumoto]

Attachment: new patch

sr=blizzard

Comment 31

[Roy Yokoyama]

I'll ask drivers to check into the trunk.

Comment 32

[Asa Dotzler [:asa]]

a=asa (on behalf of drivers) for checkin to 0.9.7

Comment 33

[Roy Yokoyama]

checked in

Comment 34

[Yuying Long]

I can not reproduce this crash on 12-14 trunk build.

Mark as verified.