Closed Bug 1190777 Opened 9 years ago Closed 9 years ago

WebGLShaderValidator

Categories

(Core :: Graphics, defect)

39 Branch
defect
Not set
critical

Tracking

()

RESOLVED FIXED
mozilla43
Tracking Status
firefox43 --- fixed

People

(Reporter: ugobejishvili, Assigned: kyle_fung)

Details

Attachments

(2 files, 1 obsolete file)

Attached file testcase.html
Program received signal SIGSEGV, Segmentation fault.
--------------------------------------------------------------------------------[regs]
$rax     0x0000000000000000 $rcx     0xf200f2f2f2f20000 $rdx     0x0000610000364ee8 $rbx     0x00006030008018b0 
$rsp     0x00007fff6b37b900 $rbp     0x00007fff6b37bc40 $rsi     0x0000000000000000 $rdi     0x00006030008018b0 
$rip     0x00007f752a1bfbd2 $r8      0x00006210001375d0 $r9      0x00006210001375d0 $r10     0x00007f74ece7519d 
$r11     0x0000000000000001 $r12     0x00006030008018b0 $r13     0x0000610000364eb0 $r14     0x00007fff6b37b980 
$r15     0x00000fffed66f730 $cs      0x0000000000000033 $ss      0x000000000000002b $ds      0x0000000000000000 
$es      0x0000000000000000 $fs      0x0000000000000000 $gs      0x0000000000000000 $eflags  [ PF ZF IF RF ] 

--------------------------------------------------------------------------------[stack]
0x00007fff6b37b900: 0x00007f74ed359430 -> 0x0000000200008b51 -> 0x0
0x00007fff6b37b908: "p6'"
0x00007fff6b37b910: 0x0000610000364ee8 -> 0x00007f752e40aa60 -> 0x0
0x00007fff6b37b918: 0x0000615000968a30 -> 0x0000633000054830 -> 0x00006130002ff2c0 -> 0x0
0x00007fff6b37b920: 0x000060e00026f978 -> 0x000060e00026f858 -> 0x000060c0004df838 -> "h2\""
0x00007fff6b37b928: 0x9
0x00007fff6b37b930: 0x000061e00039b480 -> 0x00007f7531d44bf0 -> 0x00007f752867d490 -> <mozilla::gl::GLContextGLX::~GLContextGLX()>: push rbp
0x00007fff6b37b938: 0x00007f74ecf8ecae
0x00007fff6b37b940: 0x0000000f00000007 -> 0x0
0x00007fff6b37b948: 0x0
--------------------------------------------------------------------------------[code]
=> 0x7f752a1bfbd2 <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+290>:   mov    rdi,QWORD PTR [rsi]
   0x7f752a1bfbd5 <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+293>:   mov    QWORD PTR [rsp+0x40],rsi
   0x7f752a1bfbda <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+298>:   call   0x7f752cd96a60 <ShGetUniforms(void*)>
   0x7f752a1bfbdf <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+303>:   mov    r13,rax
   0x7f752a1bfbe2 <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+306>:   mov    rax,r12
   0x7f752a1bfbe5 <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+309>:   shr    rax,0x3
--------------------------------------------------------------------------------[trace]
#0  0x00007f752a1bfbd2 in mozilla::webgl::ShaderValidator::CanLinkTo (this=<optimized out>, prev=<optimized out>, out_log=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShaderValidator.cpp:200
#1  0x00007f752a1b9442 in mozilla::WebGLShader::CanLinkTo (this=<optimized out>, prev=<optimized out>, out_log=0x610000364ee8) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShader.cpp:289
#2  0x00007f752a1b8e9a in mozilla::WebGLProgram::LinkProgram (this=0x610000364e40) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLProgram.cpp:564
#3  0x00007f752a173b0b in mozilla::WebGLContext::LinkProgram (this=0x61900058b180, prog=0x610000364e40) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLContextGL.cpp:1761
#4  0x00007f7529a1a86f in mozilla::dom::WebGLRenderingContextBinding::linkProgram (cx=<optimized out>, self=0x61900058b180, args=..., obj=...) at ./WebGLRenderingContextBinding.cpp:12326
0x00007f752a1bfbd2 in mozilla::webgl::ShaderValidator::CanLinkTo (this=<optimized out>, prev=<optimized out>, out_log=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShaderValidator.cpp:200
200     /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShaderValidator.cpp: No such file or directory.
gef> bt
#0  0x00007f752a1bfbd2 in mozilla::webgl::ShaderValidator::CanLinkTo (this=<optimized out>, prev=<optimized out>, out_log=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShaderValidator.cpp:200
#1  0x00007f752a1b9442 in mozilla::WebGLShader::CanLinkTo (this=<optimized out>, prev=<optimized out>, out_log=0x610000364ee8) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShader.cpp:289
#2  0x00007f752a1b8e9a in mozilla::WebGLProgram::LinkProgram (this=0x610000364e40) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLProgram.cpp:564
#3  0x00007f752a173b0b in mozilla::WebGLContext::LinkProgram (this=0x61900058b180, prog=0x610000364e40) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLContextGL.cpp:1761
#4  0x00007f7529a1a86f in mozilla::dom::WebGLRenderingContextBinding::linkProgram (cx=<optimized out>, self=0x61900058b180, args=..., obj=...) at ./WebGLRenderingContextBinding.cpp:12326
#5  0x00007f752a08b177 in mozilla::dom::GenericBindingMethod (cx=<optimized out>, argc=<optimized out>, vp=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/bindings/BindingUtils.cpp:2501
#6  0x00007f752d6d57b3 in js::CallJSNative (cx=<optimized out>, native=<optimized out>, args=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/jscntxtinlines.h:235
#7  0x00007f752d69ed17 in js::Invoke (cx=<optimized out>, args=..., construct=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/vm/Interpreter.cpp:463
#8  0x00007f752d6c9e38 in Interpret (cx=0x6140001dfa40, state=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/vm/Interpreter.cpp:2592
#9  0x00007f752d6ba25e in js::RunScript (cx=<optimized out>, state=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/vm/Interpreter.cpp:420
#10 0x00007f752d69ee77 in js::Invoke (cx=<optimized out>, args=..., construct=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/vm/Interpreter.cpp:489
#11 0x00007f752d670505 in js::Invoke (cx=0x6140001dfa40, thisv=..., fval=..., argc=<optimized out>, argv=<optimized out>, rval=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/vm/Interpreter.cpp:526
#12 0x00007f752df18512 in JS::Call (cx=0x6140001dfa40, args=..., thisv=..., fval=..., rval=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/js/src/jsapi.cpp:4331
#13 0x00007f7529ca7855 in mozilla::dom::EventHandlerNonNull::Call (this=<optimized out>, cx=0x6140001dfa40, event=..., aRv=..., aThisVal=..., aRetVal=...) at ./EventHandlerBinding.cpp:259
#14 0x00007f752a31a514 in mozilla::dom::EventHandlerNonNull::Call<nsISupports*> (this=<optimized out>, thisVal=<optimized out>, event=..., aRv=..., aExceptionHandling=<optimized out>, aCompartment=<optimized out>, aRetVal=...) at ../../dist/include/mozilla/dom/EventHandlerBinding.h:347
#15 0x00007f752a318e00 in mozilla::JSEventHandler::HandleEvent (this=0x60400056f7d0, aEvent=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/JSEventHandler.cpp:214
#16 0x00007f752a2f1995 in mozilla::EventListenerManager::HandleEventSubType (this=<optimized out>, aListener=<optimized out>, aDOMEvent=<optimized out>, aCurrentTarget=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/EventListenerManager.cpp:965
#17 0x00007f752a2f2a7a in mozilla::EventListenerManager::HandleEventInternal (this=0x60d00043c3f0, aPresContext=<optimized out>, aEvent=<optimized out>, aDOMEvent=<optimized out>, aCurrentTarget=<optimized out>, aEventStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/EventListenerManager.cpp:1113
#18 0x00007f752a2e9225 in mozilla::EventTargetChainItem::HandleEvent (this=0x62200000a908, aVisitor=..., aCd=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/EventDispatcher.cpp:206
#19 0x00007f752a2e865a in mozilla::EventTargetChainItem::HandleEventTargetChain (aChain=..., aVisitor=..., aCallback=<optimized out>, aCd=...) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/EventDispatcher.cpp:299
#20 0x00007f752a2ea7a7 in mozilla::EventDispatcher::Dispatch (aTarget=<optimized out>, aPresContext=<optimized out>, aEvent=<optimized out>, aDOMEvent=<optimized out>, aEventStatus=<optimized out>, aCallback=<optimized out>, aTargets=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/events/EventDispatcher.cpp:633
#21 0x00007f752b5bbb3c in nsDocumentViewer::LoadComplete (this=0x6120002e8640, aStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/layout/base/nsDocumentViewer.cpp:998
#22 0x00007f752bf6918f in nsDocShell::EndPageLoad (this=<optimized out>, aChannel=<optimized out>, aStatus=<optimized out>, aProgress=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/docshell/base/nsDocShell.cpp:7535
#23 0x00007f752bf66a5e in nsDocShell::OnStateChange (this=<optimized out>, aProgress=<optimized out>, aRequest=<optimized out>, aStateFlags=<optimized out>, aStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/docshell/base/nsDocShell.cpp:7352
#24 0x00007f752bf6a7f0 in non-virtual thunk to nsDocShell::OnStateChange(nsIWebProgress*, nsIRequest*, unsigned int, nsresult) () at Unified_cpp_docshell_base0.cpp:7359
#25 0x00007f752842d5d2 in nsDocLoader::DoFireOnStateChange (this=<optimized out>, aProgress=<optimized out>, aRequest=0x61a000216cd0, aStateFlags=<optimized out>, aStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/uriloader/base/nsDocLoader.cpp:1263
#26 0x00007f752842cc2b in nsDocLoader::doStopDocumentLoad (this=<optimized out>, request=0x61a000216cd0, aStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/uriloader/base/nsDocLoader.cpp:844
#27 0x00007f752842a9ee in nsDocLoader::DocLoaderIsEmpty (this=0x619000aa5f80, aFlushLayout=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/uriloader/base/nsDocLoader.cpp:734
#28 0x00007f752842bfe7 in nsDocLoader::OnStopRequest (this=<optimized out>, aRequest=<optimized out>, aStatus=<optimized out>, aCtxt=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/uriloader/base/nsDocLoader.cpp:618
#29 0x00007f752842c6fd in non-virtual thunk to nsDocLoader::OnStopRequest(nsIRequest*, nsISupports*, nsresult) () at Unified_cpp_uriloader_base0.cpp:622
#30 0x00007f75273740c8 in nsLoadGroup::RemoveRequest (this=<optimized out>, request=<optimized out>, ctxt=<optimized out>, aStatus=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/netwerk/base/nsLoadGroup.cpp:663
#31 0x00007f7528d49297 in nsDocument::DoUnblockOnload (this=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/base/nsDocument.cpp:9140
#32 0x00007f7528d48fe5 in nsDocument::UnblockOnload (this=<optimized out>, aFireSync=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/base/nsDocument.cpp:9068
#33 0x00007f7528d2cc10 in nsDocument::DispatchContentLoadedEvents (this=0x61d000b34c80) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/base/nsDocument.cpp:5205
#34 0x00007f7528d767e2 in _ZN20nsRunnableMethodImplIM10nsDocumentFvvELb1EJEE3RunEv (this=<optimized out>) at ../../dist/include/nsThreadUtils.h:666
#35 0x00007f75271df62b in nsThread::ProcessNextEvent (this=<optimized out>, aMayWait=<optimized out>, aResult=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/xpcom/threads/nsThread.cpp:855
#36 0x00007f75272443ef in NS_ProcessNextEvent (aThread=<optimized out>, aMayWait=false) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/xpcom/glue/nsThreadUtils.cpp:265
#37 0x00007f75279ae36e in mozilla::ipc::MessagePump::Run (this=<optimized out>, aDelegate=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/ipc/glue/MessagePump.cpp:99
#38 0x00007f75279275c2 in MessageLoop::RunInternal (this=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/ipc/chromium/src/base/message_loop.cc:233
#39 0x00007f7527927469 in MessageLoop::Run (this=0x61400002c640) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/ipc/chromium/src/base/message_loop.cc:200
#40 0x00007f752b0687f7 in nsBaseAppShell::Run (this=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/widget/nsBaseAppShell.cpp:164
#41 0x00007f752c410ef6 in nsAppStartup::Run (this=0x60700000f7c0) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/toolkit/components/startup/nsAppStartup.cpp:281
#42 0x00007f752c4ce9b6 in XREMain::XRE_mainRun (this=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/toolkit/xre/nsAppRunner.cpp:4170
#43 0x00007f752c4cfbda in XREMain::XRE_main (this=0x7fff6b385520, argc=<optimized out>, argv=<optimized out>, aAppData=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/toolkit/xre/nsAppRunner.cpp:4250
#44 0x00007f752c4d0723 in XRE_main (argc=1, argv=0x7fff6b386df8, aAppData=<optimized out>, aFlags=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/toolkit/xre/nsAppRunner.cpp:4469
#45 0x000000000048c251 in do_main (argc=<optimized out>, argv=<optimized out>, xreDirectory=<optimized out>) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/browser/app/nsBrowserApp.cpp:294
#46 0x000000000048b7d2 in main (argc=1, argv=0x0) at /builds/slave/m-rel-l64-asan-d-0000000000000/build/browser/app/nsBrowserApp.cpp:688
gef> i f
Stack level 0, frame at 0x7fff6b37bc50:
 rip = 0x7f752a1bfbd2 in mozilla::webgl::ShaderValidator::CanLinkTo (/builds/slave/m-rel-l64-asan-d-0000000000000/build/dom/canvas/WebGLShaderValidator.cpp:200); saved rip 0x7f752a1b9442
 called by frame at 0x7fff6b37bc80
 source language c++.
 Arglist at 0x7fff6b37bc40, args: this=<optimized out>, prev=<optimized out>, out_log=<optimized out>
 Locals at 0x7fff6b37bc40, Previous frame's sp is 0x7fff6b37bc50
 Saved registers:
  rbx at 0x7fff6b37bc18, rbp at 0x7fff6b37bc40, r12 at 0x7fff6b37bc20, r13 at 0x7fff6b37bc28, r14 at 0x7fff6b37bc30, r15 at 0x7fff6b37bc38, rip at 0x7fff6b37bc48
gef> i r
rax            0x0      0
rbx            0x6030008018b0   105759283091632
rcx            0xf200f2f2f2f20000       -1008539191259037696
rdx            0x610000364ee8   106652631453416
rsi            0x0      0
rdi            0x6030008018b0   105759283091632
rbp            0x7fff6b37bc40   0x7fff6b37bc40
rsp            0x7fff6b37b900   0x7fff6b37b900
r8             0x6210001375d0   107820860274128
r9             0x6210001375d0   107820860274128
r10            0x7f74ece7519d   140140167516573
r11            0x1      1
r12            0x6030008018b0   105759283091632
r13            0x610000364eb0   106652631453360
r14            0x7fff6b37b980   140734992202112
r15            0xfffed66f730    17591874025264
rip            0x7f752a1bfbd2   0x7f752a1bfbd2 <mozilla::webgl::ShaderValidator::CanLinkTo(mozilla::webgl::ShaderValidator const*, nsCString*) const+290>
eflags         0x10246  [ PF ZF IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
Component: Developer Tools: WebGL Shader Editor → Graphics
Product: Firefox → Core
It looks like maybe this is a null deref crash?
Assignee: nobody → kfung
Ucha, could you give me a stack trace from the current latest nightly?
Flags: needinfo?(ugobejishvili)
(In reply to kfung from comment #2)
> Ucha, could you give me a stack trace from the current latest nightly?


On 42.0a1 (2015-08-10) ( latest nightly build ) testcase is not reproducible.
Flags: needinfo?(ugobejishvili)
Attached patch canlinkto-null-check.patch (obsolete) — Splinter Review
Attachment #8645890 - Flags: review?(dglastonbury)
Comment on attachment 8645890 [details] [diff] [review]
canlinkto-null-check.patch

Review of attachment 8645890 [details] [diff] [review]:
-----------------------------------------------------------------

::: dom/canvas/WebGLShaderValidator.cpp
@@ +209,5 @@
>  bool
>  ShaderValidator::CanLinkTo(const ShaderValidator* prev, nsCString* const out_log) const
>  {
> +    if (!prev) {
> +        nsPrintfCString error("Passed in NULL prev ShaderValidator.");

This just creates a formatted string on the stack. I think you need to assign it to out_log to return the error message.

@@ +217,5 @@
>      {
> +        const std::vector<sh::Uniform>* vertPtr = ShGetUniforms(prev->mHandle);
> +        const std::vector<sh::Uniform>* fragPtr = ShGetUniforms(mHandle);
> +        if (!vertPtr || !fragPtr) {
> +            nsPrintfCString error("Could not create uniform list.");

Same here.

@@ +221,5 @@
> +            nsPrintfCString error("Could not create uniform list.");
> +            return false;
> +        }
> +        const std::vector<sh::Uniform>& vertList = *vertPtr;
> +        const std::vector<sh::Uniform>& fragList = *fragPtr;

I'd remove vertList and fragList and just use the validated vertPtr & fragPtr

@@ +244,5 @@
>      {
> +        const std::vector<sh::Varying>* vertPtr = ShGetVaryings(prev->mHandle);
> +        const std::vector<sh::Varying>* fragPtr = ShGetVaryings(mHandle);
> +        if (!vertPtr || !fragPtr) {
> +            nsPrintfCString error("Could not create varying list.");

Same here.
Attachment #8645890 - Flags: review?(dglastonbury) → review-
Whoops, should have caught those.
Attachment #8645890 - Attachment is obsolete: true
Attachment #8646367 - Flags: review?(dglastonbury)
Is this just a null deref crash then?
Flags: needinfo?(kfung)
I can't be entirely sure which version of WebGLShaderValidator.cpp was in the build that exhibited the crash. Ucha, do you have the build number for it?
Flags: needinfo?(kfung) → needinfo?(ugobejishvili)
(In reply to kfung from comment #8)
> I can't be entirely sure which version of WebGLShaderValidator.cpp was in
> the build that exhibited the crash. Ucha, do you have the build number for
> it?

https://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-release-linux64-asan/1437767267/firefox-39.0.2.en-US.linux-x86_64-asan.tar.bz2
Flags: needinfo?(ugobejishvili)
Yeah the crash occurs at a deference (https://hg.mozilla.org/releases/mozilla-release/file/ec21f96665f7/dom/canvas/WebGLShaderValidator.cpp#l200) so I'm pretty sure it's a NULL deref.
Attachment #8646367 - Flags: review?(dglastonbury) → review+
No indications here that this is security sensitive.
Group: core-security
No try push, since this is just a patch for null checks.
Keywords: checkin-needed
https://hg.mozilla.org/mozilla-central/rev/7630d1aab497
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Target Milestone: --- → mozilla43
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: