638112 - Assertion failure: chars[length] == jschar(0), at js\src\jsstr.h:252

Status: RESOLVED FIXED

(Core :: XPCOM, defect)

Description

[Bob Clary [:bc] (inactive)]

1. http://www.explorimmo.com/immobilier-vente-appartement-marseille%252B5eme%252B13005-31.html?location=marseille%2525206eme%252520%252813006%2529%252Cmarseille%2525207eme%252520%252813007%2529%252Cmarseille%2525208eme%252520%252813008%2529%252Cmarseille%2525209eme%252520%252813009%2529%252Cmarseille%25252010eme%252520%252813010%2529%2526priceMax=170000.0

2. Assertion failure: chars[length] == jschar(0), at js\src\jsstr.h:252

Operating system: Windows NT
                  5.1.2600 Service Pack 3
CPU: x86
     GenuineIntel family 6 model 44 stepping 2
     1 CPU

Crash reason:  EXCEPTION_ACCESS_VIOLATION_WRITE
Crash address: 0x0

Thread 0 (crashed)
 0  mozjs.dll!JS_Assert [jsutil.cpp : 73 + 0x0]
    eip = 0x0084d8da   esp = 0x0012ae9c   ebp = 0x0012ae9c   ebx = 0x00000000
    esi = 0x0404f6f4   edi = 0xffff0007   eax = 0xffffffff   ecx = 0x126c768a
    edx = 0x003b3d38   efl = 0x00010206
    Found by: given as instruction pointer in context
 1  mozjs.dll!JSString::initFlat(unsigned short *,unsigned int) [jsstr.h : 252 + 0x23]
    eip = 0x006a873b   esp = 0x0012aea4   ebp = 0x0012aeb4
    Found by: call frame info
 2  mozjs.dll!JS_NewExternalString [jsapi.cpp : 2801 + 0xf]
    eip = 0x006a8680   esp = 0x0012aebc   ebp = 0x0012aed4
    Found by: call frame info
 3  xul.dll!XPCConvert::NativeData2JS(XPCLazyCallContext &,jsval_layout *,void const *,nsXPTType const &,nsID const *,unsigned int *) [xpcconvert.cpp : 415 + 0x17]
    eip = 0x111eae43   esp = 0x0012aedc   ebp = 0x0012b040
    Found by: call frame info
 4  xul.dll!XPCConvert::NativeData2JS(XPCCallContext &,jsval_layout *,void const *,nsXPTType const &,nsID const *,unsigned int *) [xpcprivate.h : 3262 + 0x1f]
    eip = 0x111eef72   esp = 0x0012b048   ebp = 0x0012b11c
    Found by: call frame info
 5  xul.dll!CallMethodHelper::GatherAndConvertResults() [xpcwrappednative.cpp : 2646 + 0x21]
    eip = 0x111f7ff8   esp = 0x0012b124   ebp = 0x0012b28c
    Found by: call frame info
 6  xul.dll!CallMethodHelper::Call() [xpcwrappednative.cpp : 2405 + 0x7]
    eip = 0x111f771c   esp = 0x0012b294   ebp = 0x0012b2a0
    Found by: call frame info
 7  xul.dll!XPCWrappedNative::CallMethod(XPCCallContext &,XPCWrappedNative::CallMode) [xpcwrappednative.cpp : 2354 + 0x15]
    eip = 0x111f744d   esp = 0x0012b2a8   ebp = 0x0012b424
    Found by: call frame info
 8  xul.dll!XPCWrappedNative::GetAttribute(XPCCallContext &) [xpcprivate.h : 2675 + 0xd]
    eip = 0x111dddee   esp = 0x0012b42c   ebp = 0x0012b434
    Found by: call frame info
 9  xul.dll!XPC_WN_GetterSetter(JSContext *,unsigned int,jsval_layout *) [xpcwrappednativejsops.cpp : 1663 + 0xb]
    eip = 0x111dddac   esp = 0x0012b43c   ebp = 0x0012b508
  
Windows XP, 2.0.0. Not Mac. Haven't tested linux.

Note also (Windows + Mac):

###!!! ASSERTION: Not a UTF-8 string. This code should only be used for converting from known UTF-8 strings.: 'Error', file c:\work\mozilla\builds\2.0.0\mozilla\firefox-debug\dist\include\nsUTF8Utils.h, line 452

###!!! ASSERTION: length mismatch: 'calculator.Length() == converter.Length()', file c:/work/mozilla/builds/2.0.0/mozilla/xpcom/string/src/nsReadableUtils.cpp, line 402

Comment 1

[Andreas Gal :gal]

Unterminated C-string. How refreshing. Luke?

Comment 2

[Bob Clary [:bc] (inactive)]

I reproduced this on the linux crash workers but not locally on my mac with a build from this morning. the automation is lagging at the moment for mac, but that is enough for OS->ALL

Comment 3

[Andreas Gal :gal]

bc, is this a recent regression?

Comment 4

[Mike Shaver (:shaver emeritus)]

If you can reproduce it, can you get a core file?

Comment 5

[Bob Clary [:bc] (inactive)]

(In reply to comment #3)
> bc, is this a recent regression?

I haven't tried to see if this is a recent regression on 2.0.0. It doesn't crash opt builds so I'd have to build to check. I don't see it on 1.9.2 or 1.9.1 though. If it is important, I can do some builds and check it out.

(In reply to comment #4)
> If you can reproduce it, can you get a core file?

wget craps out trying to save the page due to invalid multibyte characters in the file names and using Firefox and save complete page does not reproduce. I'm open to suggestions.

Comment 6

[Bob Clary [:bc] (inactive)]

fyi, was able to reproduce in the automation and locally on Mac. Not sure why I failed the first time.

Comment 7

[Luke Wagner [:luke]]

(gdb) f
#5 in XPCConvert::NativeData2JS at xpconnect/src/xpcconvert.cpp:415
(http://mxr.mozilla.org/mozilla-central/source/js/src/xpconnect/src/xpcconvert.cpp#404)
(gdb) p *cString
$14 = {mData = 0xa3801168 "Information Sans-Autorit", <incomplete sequence \351>, mLength = 25, mFlags = 5}
(gdb) p cString->mData[25]
$15 = 0 '\000'
(gdb) p p[25]
$16 = 42405
(gdb) p p[24]
$17 = 0

So it seems like UTF8ToNewUnicode is doing the wrong thing for the <incomplete sequence \351>.

Comment 8

[Luke Wagner [:luke]]

hg annotate shows 2010 changes in CalculateUTF8Length and ConvertUTF8toUTF16:

http://hg.mozilla.org/tracemonkey/annotate/2b2b968a4cf4/xpcom/string/public/nsUTF8Utils.h#l405

which seem relevant, namely:

changeset:   38628:c5520407a4ad
user:        Jonas Sicking <[e-mail]>
date:        Tue Feb 23 09:38:10 2010 -0800
summary:     Bug 422868 part 1: Fix UTF8 <-> UTF16 conversion code to deal with all encoding errors consistently. r=smontagu

given that the string in comment 7 seems to have an error.  Maybe Jonas has a better idea?

Comment 9

[Simon Montagu :smontagu]

(In reply to comment #7)
> (gdb) p *cString
> $14 = {mData = 0xa3801168 "Information Sans-Autorit", <incomplete sequence
> \351>, mLength = 25, mFlags = 5}

\351 = 0xE9, which is "é" in ISO-8859-1, so it looks as though the input is actually ISO-8859-1

Comment 10

[Mats Palmgren (inactive)]

Attachment: Raw response from http://www.explorimmo.com/perso/light/index.html

Loading it directly from the URL bar doesn't trigger the assertion.
It is loaded using a XMLHttpRequest from
http://www.explorimmo.com/pack/h-180574302.js
which is loaded from the document as:
<script type="text/javascript" src="/pack/h-180574302.js"
  charset="utf-8"></script>

Comment 11

[Mats Palmgren (inactive)]

Attachment: Fix CalculateUTF8Length

CalculateUTF8Length.write() has slightly different error handling than
ConvertUTF16toUTF8.write() and UTF8ToNewUnicode() is using the length from
the first for the out parameter 'aUTF16Count'.
http://mxr.mozilla.org/mozilla-central/source/xpcom/string/src/nsReadableUtils.cpp#384

In the CalculateUTF8Length loop when we see a byte that indicates a
multi-byte char we increment 'p' by how many bytes we expect AND we
increment 'mLength'.  If 'p' is outside the buffer we assert and leave
'mLength' although the last character was incomplete.
http://mxr.mozilla.org/mozilla-central/source/xpcom/string/public/nsUTF8Utils.h#380

ConvertUTF16toUTF8 on the other only writes valid characters so its
length will be one less than 'mLength'.
http://mxr.mozilla.org/mozilla-central/source/xpcom/string/public/nsUTF8Utils.h#482

This is what causes the second assertion:
###!!! ASSERTION: length mismatch: 'calculator.Length() == converter.Length()',

We should fix this regardless of what root cause of the bogus string is.
It'll fix the JS assertion and the "length mismatch" assertion.

Comment 12

[Mats Palmgren (inactive)]

I tried to write a mochitest using .sjs^headers^ but it seems that goes
through some JS code first so it didn't work.  Is there a way to send
the raw contents of a file as the response?

Comment 13

[Jeff Walden [:Waldo]]

(In reply to comment #12)
> Is there a way to send the raw contents of a file as the response?

http://hg.mozilla.org/mozilla-central/file/e56ecd8b3a68/netwerk/test/httpserver/nsIHttpServer.idl#l596

Comment 14

[Mats Palmgren (inactive)]

Attachment: fix + test

See comment 11.

Comment 15

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

Comment on attachment 523715 [details] [diff] [review]
fix + test

r=dbaron

Comment 16

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

In the future, however, please include commit messages within the patch when posting for review.

Comment 17

[Mats Palmgren (inactive)]

OK, I'll try to remember that.

http://hg.mozilla.org/mozilla-central/rev/639df63f952e

Comment 18

[Asa Dotzler [:asa]]

not going to track this for 6. thanks for the fix!