593106 - firefox 4.0b crashes [@ strlen | nsDependentCString::nsDependentCString(char const*) ] [@ strlen | AppendUTF8toUTF16(char const*, nsAString_internal&) ]

Status: VERIFIED FIXED

(Toolkit :: Startup and Profile System, defect)

Description

[chris hofmann]

This signature has been around for awhile, and fixes have been applied for bugs 522214 547894 that don't seem to have caught all problems.  A low volume crash on 3.x stll seems to be around but this bug is about trying to filter through that to see if we have a volume regression in 4.0betas

We have seen an interesting pattern around these crashes with spikes just after the release of beta4, but then another around aug 8 maybe related to users upgrading or using 4.0b2.  The high volume spikes seem to be affect b2 and b4, and might be related to people giving webgl a try.

20100801 35
20100802 65
20100803 72
20100804 80
20100805 81
20100806 91
20100807 76
20100808 196
20100809 278
20100810 125
20100811 106
20100815 75
20100816 97
20100817 86
20100818 41
20100819 36
20100820 30
20100821 42
20100822 11
20100823 26
20100824 24
20100825 79
20100826 91
20100827 117
20100828 141
20100829 146
20100830 77
20100831 210
20100901 173

here is a breakdown of the highest spike days.

checking --- strlen...nsDependentCString::nsDependentCString 20100809-crashdata.csv
found in: 4.0b4pre 4.0b2 3.6.8 3.6.6
release total-crashes
              strlen...nsDependentCString::nsDependentCString crashes
                         pct.
all     286214  278     0.000971301
4.0b4pre2245    150     0.0668151
4.0b2   15087   126     0.00835156
3.6.8   178205  1       5.61151e-06
3.6.6   13699   1       7.2998e-05

and on this day some of the top urls look to be webgl demos

   5 http://sio29.sakura.ne.jp/tmp/webgl/index_eruru.html
   3 https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/sdk/demos/webkit/SpiritBox.html
   3 http://x3dom.org/x3dom/example/x3dom_inline.xhtml
   3 http://www.rozengain.com/files/webgl/blenderexport/monkey.html
   3 http://tubagames.net/saloon/index.html
   3 http://people.mozilla.com/~vladimir/webgl/spore/sporeview.html
   3 file:///C:/Users/XXXXXXX/Desktop/Scarcello/WEBGL/Tests/index.html


ound in: 4.0b4 4.0b2 3.6.9 3.0b2 3.6.8 3.6
release total-crashes
              strlen...nsDependentCString::nsDependentCString crashes
                         pct.
all     309243  210     0.000679078
4.0b4   22625   159     0.00702762
4.0b2   1395    40      0.0286738
3.6.9   9699    4       0.000412414
3.0b2   407     4       0.00982801
3.6.8   192153  2       1.04084e-05
3.6     5989    1       0.000166973

mostly blank or \N urls were reported on that day


reports at

some of the 4.0b4 reports look like below and might benefit from additional skip listing to get a better breakdown of the stack.

http://crash-stats.mozilla.com/report/index/312fcf99-bcea-4b4a-9476-7515c2100902

Frame  	Module  	Signature [Expand]  	Source
0 	mozcrt19.dll 	strlen 	strlen.asm:81
1 	xul.dll 	nsDependentCString::nsDependentCString 	obj-firefox/dist/include/nsTDependentString.h:90
2 	xul.dll 	mozilla::WebGLContext::CompileShader 	content/canvas/src/WebGLContextGL.cpp:2867
3 	xul.dll 	nsICanvasRenderingContextWebGL_CompileShader 	obj-firefox/js/src/xpconnect/src/dom_quickstubs.cpp:25042
4 	mozjs.dll 	js::Interpret 	js/src/jsinterp.cpp:4713
5 	mozjs.dll 	js::InvokeCommon<int > 	js/src/jsinterp.cpp:588
6 	mozjs.dll 	js::Invoke 	js/src/jsinterp.cpp:691
7 	mozjs.dll 	js::InternalInvoke 	js/src/jsinterp.cpp:754
8 	mozjs.dll 	JS_CallFunctionValue 	js/src/jsapi.cpp:4838
9 	xul.dll 	nsJSContext::CallEventHandler 	dom/base/nsJSEnvironment.cpp:2248
10 	xul.dll 	nsJSEventListener::HandleEvent 	dom/src/events/nsJSEventListener.cpp:228
11 	xul.dll 	nsEventListenerManager::HandleEventSubType 	content/events/src/nsEventListenerManager.cpp:1106
12 	xul.dll 	nsEventListenerManager::HandleEventInternal 	content/events/src/nsEventListenerManager.cpp:1202
13 		@0x5a959af 

http://crash-stats.mozilla.com/report/list?product=Firefox&query_search=signature&query_type=startswith&query=strlen%20|%20nsDependentCString%3A%3AnsDependentCStri&date=09%2F02%2F2010%2011%3A46%3A34&range_value=1&range_unit=weeks&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=&signature=strlen%20|%20nsDependentCString%3A%3AnsDependentCString%28char%20const*%29

Comment 1

[Boris Zbarsky [:bzbarsky]]

That webgl stack looks like the thing that bug 585502 fixed.

Comment 2

[chris hofmann]

ok,  I'll make this a tracking bug that now depends on 585502, and dig though the rest of the stacks to see of there is anything else that turns up.  if not, we can close this a dup of 585502.

Comment 3

[chris hofmann]

durning august it looks like the  the webgl stack was most popular, but an "AppendUTF8toUTF16" stack looks like it might have grown in volume.  

this looks like this might be the most popular form of the stack on 8/31.  

http://crash-stats.mozilla.com/report/index/312fcf99-bcea-4b4a-9476-7515c2100902

Frame      Module      Signature [Expand]      Source
0     mozcrt19.dll     strlen     strlen.asm:69
1     xul.dll     nsDependentCString::nsDependentCString     obj-firefox/dist/include/nsTDependentString.h:90
2     xul.dll     AppendUTF8toUTF16     xpcom/string/src/nsReadableUtils.cpp:276
3     xul.dll     NS_ConvertUTF8toUTF16::NS_ConvertUTF8toUTF16     obj-firefox/dist/include/nsString.h:176
4     xul.dll     cvt_s     obj-firefox/xpcom/build/nsTextFormatter.cpp:565
5     xul.dll     dosprintf     
6     xul.dll     nsTextFormatter::vsmprintf     obj-firefox/xpcom/build/nsTextFormatter.cpp:1287
7     xul.dll     LogConsoleMessage     toolkit/components/commandlines/src/nsCommandLine.cpp:581
8     xul.dll     nsCommandLine::EnumerateHandlers     
9     xul.dll     nsCommandLine::Run     toolkit/components/commandlines/src/nsCommandLine.cpp:702
10     xul.dll     XRE_main     toolkit/xre/nsAppRunner.cpp:3630
11     firefox.exe     wmain     toolkit/xre/nsWindowsWMain.cpp:120
12     firefox.exe     __tmainCRTStartup     obj-firefox/memory/jemalloc/crtsrc/crtexe.c:591
13     kernel32.dll     BaseProcessStart     

Don't know if its related but LogConsoleMessage code in frame 7 change 2010-07-20

 Bug 579497 - Add error console logging for missing/incorrect command-line handlers

Comment 4

[chris hofmann]

lets just make this bug about  
strlen | nsDependentCString::nsDependentCString(char const*) | AppendUTF8toUTF16

since that seem to be the only major problem outstanding.

Comment 5

[Benjamin Smedberg]

Not XPCOM! This code was added recently, but the crash makes no sense:

http://mxr.mozilla.org/mozilla-central/source/toolkit/components/commandlines/src/nsCommandLine.cpp#622

None of the arguments can be null, really.  Dave, do you have any thoughts/clues?

Comment 6

[Dave Townsend [:mossop]]

(In reply to comment #5)
> Not XPCOM! This code was added recently, but the crash makes no sense:
> 
> http://mxr.mozilla.org/mozilla-central/source/toolkit/components/commandlines/src/nsCommandLine.cpp#622
> 
> None of the arguments can be null, really.  Dave, do you have any
> thoughts/clues?

I have no ideas why we'd be seeing a crash like this.

Comment 7

[Dave Townsend [:mossop]]

(In reply to comment #5)
> Not XPCOM! This code was added recently, but the crash makes no sense:
> 
> http://mxr.mozilla.org/mozilla-central/source/toolkit/components/commandlines/src/nsCommandLine.cpp#622
> 
> None of the arguments can be null, really.  Dave, do you have any
> thoughts/clues?

Should this be |if (!contractID.get())| ?

http://hg.mozilla.org/mozilla-central/annotate/84ee6bc0484d/toolkit/components/commandlines/src/nsCommandLine.cpp#l617

Either way this crash is still showing up so I guess we need to block

Comment 8

[Benjamin Smedberg]

Attachment: No nsXPIDLCString, rv-check, hope for the best, rev. 1

Comment 9

[Dave Townsend [:mossop]]

Comment on attachment 475877 [details] [diff] [review]
No nsXPIDLCString, rv-check, hope for the best, rev. 1

Confidence inspiring words!

Comment 10

[Benjamin Smedberg]

Isn't it now!

http://hg.mozilla.org/mozilla-central/rev/348fdd67d988

Marking FIXED for now, please reopen if this form of crash (nsCommandLine.cpp in the stack don't go away).

Comment 12

[Scoobidiver (away)]

Still crash in b7pre/20100917 and b7pre/2010918 builds @ strlen | AppendUTF8toUTF16(char const*, nsAString_internal&).

See reports :
http://crash-stats.mozilla.com/report/list?product=Firefox&version=Firefox%3A4.0b7pre&query_search=signature&query_type=exact&query=strlen%20|%20AppendUTF8toUTF16%28char%20const*%2C%20nsAString_internal%26%29&date=09%2F19%2F2010%2003%3A37%3A09&range_value=1&range_unit=weeks&hang_type=any&process_type=any&plugin_field=&plugin_query_type=&plugin_query=&do_query=1&admin=&signature=strlen%20|%20AppendUTF8toUTF16%28char%20const*%2C%20nsAString_internal%26%29

Comment 13

[Benjamin Smedberg]

I got some slightly better data out of a minidump:

entry.get is "b-jsconsole"
contractID.get() is not null:

0x01d137f0 <Bad Ptr>	char *
I don't know whether it's a <bad ptr> because it's a minidump and the heap memory isn't available, or whether it's truly a bogus pointer.
This pointer should be coming from CategoryNode::GetLeaf, which uses NS_strdup and hardly *looks* like it can fail. Unless there's some sort of memory-scribbling going on.

I have even fewer clues than before.

Comment 14

[Julian Seward [:jseward]]

For  http://sio29.sakura.ne.jp/tmp/webgl/index_eruru.html

and using libosmesa as provided on Ubuntu 10.04 (x86_64) with
M-C of 29 Oct, I got the report below.

Probably a red herring since slang_emit.c isn't one of ours, and it's
where the alleged undefinedness allegedly came from.  OTOH the trace
does contain

nsICanvasRenderingContextWebGL_CompileShader
-> mozilla::WebGLContext::CompileShader(nsIWebGLShader*)

as does the stack in comment #0.

Conditional jump or move depends on uninitialised value(s)
   at 0x259EF6B5: _mesa_lookup_parameter_constant (prog_parameter.c:594)
   by 0x259EFF7D: _mesa_add_unnamed_constant (prog_parameter.c:252)
   by 0x25AD4AC8: emit (slang_emit.c:2380)
   by 0x25AD4F4F: emit (slang_emit.c:847)
   by 0x25AD4B33: emit (slang_emit.c:1930)
   by 0x25AD4F4F: emit (slang_emit.c:847)
   by 0x25AD5034: emit (slang_emit.c:1393)
   by 0x25AD5407: emit (slang_emit.c:2286)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD5314: emit (slang_emit.c:2296)
   by 0x25AD6754: emit_if (slang_emit.c:1689)
   by 0x25AD5330: emit (slang_emit.c:2413)
   by 0x25AD5407: emit (slang_emit.c:2286)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD5314: emit (slang_emit.c:2296)
   by 0x25AD6700: emit_if (slang_emit.c:1668)
   by 0x25AD5330: emit (slang_emit.c:2413)
   by 0x25AD6754: emit_if (slang_emit.c:1689)
   by 0x25AD5330: emit (slang_emit.c:2413)
   by 0x25AD5407: emit (slang_emit.c:2286)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD5314: emit (slang_emit.c:2296)
   by 0x25AD53ED: emit (slang_emit.c:2283)
   by 0x25AD6E14: _slang_emit_code (slang_emit.c:2570)
   by 0x25AD1305: _slang_codegen_function (slang_codegen.c:5319)
   by 0x25A351A2: parse_code_unit (slang_compile.c:2444)
   by 0x25A3545D: compile_binary (slang_compile.c:2487)
   by 0x25A35993: _slang_compile (slang_compile.c:2557)
   by 0x59DD4E3: mozilla::WebGLContext::CompileShader(nsIWebGLShader*)
                 (GLContext.h:1610)
   by 0x5E8423F: nsICanvasRenderingContextWebGL_CompileShader(JSContext*,
                 unsigned int, unsigned long*) (dom_quickstubs.cpp:28605)
   by 0x6537CA0: CallCompiler::generateNativeStub() (jscntxtinlines.h:652)
   by 0x6533796: js::mjit::ic::NativeCall(js::VMFrame&,
                 js::mjit::ic::CallICInfo*) (MonoIC.cpp:851)
   by 0x1D4A3474: ???
   by 0x64FEF8B: js::mjit::EnterMethodJIT(JSContext*, JSStackFrame*, void*,
                 js::Value*) (MethodJIT.cpp:742)
   by 0x64FF0CB: js::mjit::JaegerShot(JSContext*) (MethodJIT.cpp:767)
   by 0x641B6C4: js::RunScript(JSContext*, JSScript*, JSStackFrame*)
                 (jsinterp.cpp:635)
   by 0x641C621: js::Invoke(JSContext*, js::CallArgs const&, unsigned int)
                 (jsinterp.cpp:741)

 Uninitialised value was created by a stack allocation
   at 0x25AD3DF0: constant_to_storage (slang_emit.c:393)

Comment 15

[Benjamin Smedberg]

Attachment: Even more extreme nullcheck, rev. 1

Comment 16

[Dave Townsend [:mossop]]

Comment on attachment 491915 [details] [diff] [review]
Even more extreme nullcheck, rev. 1

How does this tie with comment 13 where you said that neither of those things were null in the crash anyway?

Comment 17

[Benjamin Smedberg]

Hrm, I forgot about that comment. I have no clue what's going on, obviously, and the minidump doesn't have enough information to help fix this. I could disable the warning entirely, but that seems weird.

Comment 18

[Benjamin Smedberg]

Attachment: Use a pointer instead of a reference, rev. 1

Neil points out the probable cause of this crash in bug 579497 comment 3

Comment 19

[Dave Townsend [:mossop]]

Comment on attachment 492317 [details] [diff] [review]
Use a pointer instead of a reference, rev. 1

I think I prefer Neil's approach in bug 593106 so let's go with that.

Comment 20

[Dave Townsend [:mossop]]

By which I meant bug 613725

Comment 21

[Benjamin Smedberg]

Should be fixed by bug 613725. Please reopen if necessary.

Comment 22

[Benjamin Smedberg]

Crash-stats says yes.