679646 - (MakeContextCurrent) webgl crash @mozilla::WebGLContext::Viewport

Status: RESOLVED WORKSFORME

(Core :: Graphics: CanvasWebGL, defect)

Description

[David Maciejak]

Attachment: debug_webgl.txt

User Agent: Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0
Build ID: 20110812234425

Steps to reproduce:

Hi,

I was playing again with my fuzzer, and got the crash below (see details enclosed)
due to the nature of the fuzzer based on cross_fuzz it's very hard to get a small test case ...
Anyway hope it helps.

Program received signal SIGSEGV, Segmentation fault.
mozilla::gl::GLContext::MakeCurrent (this=0x0, aForce=0) at ../../../dist/include/GLContext.h:492
	in ../../../dist/include/GLContext.h
(gdb) bt full
#0  mozilla::gl::GLContext::MakeCurrent (this=0x0, aForce=0) at ../../../dist/include/GLContext.h:492
No locals.
#1  0xb6ffa841 in mozilla::WebGLContext::MakeContextCurrent (this=<value optimized out>)
    at /build/buildd/firefox-6.0+build1+nobinonly/build-tree/mozilla/content/canvas/src/WebGLContext.h:475
No locals.
#2  0xb6ffffac in mozilla::WebGLContext::Viewport (this=0xab274520, x=0, y=0, width=999917, height=106)
    at /build/buildd/firefox-6.0+build1+nobinonly/build-tree/mozilla/content/canvas/src/WebGLContextGL.cpp:3858
No locals.
#3  0xb73b6288 in nsIDOMWebGLRenderingContext_Viewport (cx=0x983ef680, argc=4, vp=0xb07fe2e8)
    at /build/buildd/firefox-6.0+build1+nobinonly/build-tree/mozilla/objdir-i686-linux-gnu/js/src/xpconnect/src/dom_quickstubs.cpp:33805
        self = 0xab274520
        selfref = {ptr = 0xab274520}
        argv = 0xb07fe2f8


Regards,
David Maciejak
Fortinet's FortiGuard Global Security Research Team





Actual results:

Segmentation fault


Expected results:

no crash

Comment 1

[Daniel Veditz [:dveditz]]

this=0x0 is generally going to lead to an unexploitable null dereference. Did you see any exploitable-looking evidence?

Comment 2

[David Maciejak]

no, i believe it s just a denial of service

Comment 3

[Benoit Jacob [:bjacob] (mostly away)]

Can this be reproduced in Firefox >= 7 (that is, in Beta)? If yes, can it be reproduced in Nightly? I ask because several crashes have been fixed not too long ago, that gave this kind of null GLContext pointer derefs.

Comment 4

[David Maciejak]

got this crash on firefox 7.0 beta bp-76d0cd20-ac90-4a1b-96aa-2bc812110825 and i was not able to reproduce it for now on nightly build (24-Aug-2011)

Comment 5

[David Maciejak]

got a crash (bp-ad970a51-f2c5-4ae2-bb63-0e5d62110825) on nighly build (24-Aug-2011) but seems it's related to other component @nsPNGEncoder::ConvertHostARGBRow and linked to bug id 614144

Comment 6

[David Maciejak]

able to reproduce on nighlty build too (bp-f604e58c-040a-46a1-9672-de5152110825) the same ff7 issue @mozilla::WebGLContext::Viewport

Comment 7

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to David Maciejak from comment #6)
> able to reproduce on nighlty build too
> (bp-f604e58c-040a-46a1-9672-de5152110825) the same ff7 issue
> @mozilla::WebGLContext::Viewport

Interesting, thanks. Does your testcase resize the canvas, or in any other way forces the canvas' backbuffer to be recreated (e.g. a format change) just before the crash?

Comment 8

[David Maciejak]

yes I would like to know too :) but it's not easy to get the hands on the poc with the way the fuzzer is working... it's based on cross_fuzz the dom fuzzer so i believe the canvas resize is possible, but i am not sure for the format change.
Sorry to not be able to give you more info, i will try to see if i can save the page before it's loading in the browser. Feel free to ask me if you need some info or do some tests

Comment 9

[Benoit Jacob [:bjacob] (mostly away)]

One thing that you can do is get a debug build and run it with this environment variable:

MOZ_GL_DEBUG_VERBOSE=1

the ~1000 last lines of terminal output before the crash should be interesting.

Also, when it crashes, in a debugger, you can do:

call DumpJSStack()

this will print the JS stack to the terminal you're running Firefox from. It would then be interesting to look at the surrounding JS code.

Comment 10

[David Maciejak]

ok, will try it

Comment 11

[David Maciejak]

I did some more testing on 2011-08-25-mozilla-beta-debug => firefox-7.0.en-US.debug-linux-i686.tar.bz2 file.
The crash still occurs, see logs extract below:

WARNING: Overflowed nscoord_MAX in conversion to nscoord: 'aValue <= nscoord_MAX', file ../../dist/include/nsCoord.h, line 106
WARNING: Overflowed nscoord_MAX in conversion to nscoord: 'aValue <= nscoord_MAX', file ../../dist/include/nsCoord.h, line 106
WARNING: NS_ENSURE_TRUE(browserChrome) failed: file ../../../dom/base/nsBarProps.cpp, line 85
WARNING: NS_ENSURE_TRUE(browserChrome) failed: file ../../../dom/base/nsBarProps.cpp, line 85
###!!! ASSERTION: You can't dereference a NULL nsRefPtr with operator->().: 'mRawPtr != 0', file ../../../dist/include/nsAutoPtr.h, line 1117


As you said seems it's happen on onresize events.
Please find the full log output enclosed in the report (file ff_debug_679646.log).

As it's a null pointer dereference, i suggest to raise the importance to critical as this kind of issue may cause remote code execution.

Comment 12

[David Maciejak]

Attachment: debug build output

Comment 13

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to David Maciejak from comment #11)
> I did some more testing on 2011-08-25-mozilla-beta-debug =>
> firefox-7.0.en-US.debug-linux-i686.tar.bz2 file.
> The crash still occurs, see logs extract below:
> 
> WARNING: Overflowed nscoord_MAX in conversion to nscoord: 'aValue <=
> nscoord_MAX', file ../../dist/include/nsCoord.h, line 106
> WARNING: Overflowed nscoord_MAX in conversion to nscoord: 'aValue <=
> nscoord_MAX', file ../../dist/include/nsCoord.h, line 106
> WARNING: NS_ENSURE_TRUE(browserChrome) failed: file
> ../../../dom/base/nsBarProps.cpp, line 85
> WARNING: NS_ENSURE_TRUE(browserChrome) failed: file
> ../../../dom/base/nsBarProps.cpp, line 85
> ###!!! ASSERTION: You can't dereference a NULL nsRefPtr with operator->().:
> 'mRawPtr != 0', file ../../../dist/include/nsAutoPtr.h, line 1117

Thanks; can you please run with XPCOM_DEBUG_BREAK=abort ? This will allow us to get a backtrace to this first null dereference.


> 
> 
> As you said seems it's happen on onresize events.
> Please find the full log output enclosed in the report (file
> ff_debug_679646.log).

Thanks a lot. Based on this, the most likely hypothesis is that nsHTMLCanvasElement::UpdateContext is failing to resize the context, here:

http://hg.mozilla.org/mozilla-central/file/cc1e08803869/content/html/content/src/nsHTMLCanvasElement.cpp#l596

As you can see, in this case, it's then nulling out the pointer to the canvas context, which is very likely the cause of the null deref.

We then need to figure why it's still continuing to execute WebGL calls on this now-null context. We had a similar crash recently that got fixed by the addition of the mCurrentContextId.AssignLiteral(""); but apparently that's still not enough.

> 
> As it's a null pointer dereference, i suggest to raise the importance to
> critical as this kind of issue may cause remote code execution.

I don't understand: how can a _null_ pointer dereference allow code execution?

Comment 14

[David Maciejak]

i am trying to reproduce it, not easy, maybe i was lucky the first time ...
Sometime, a null pointer dereference is an hint to find memory corruption, and memory corruption may allow code execution.

Comment 15

[David Maciejak]

Attachment: debug build output 2

PCOM_DEBUG_BREAK=abort

Comment 16

[David Maciejak]

please see ff_debug_679646_2.log enclosed

Comment 17

[Benoit Jacob [:bjacob] (mostly away)]

Thanks, but I really wanted a native backtrace to this point, not just a JS backtrace :)

Also a JS backtrace is really mostly useful to see where it was in the JS code. So it really becomes useful if you can paste here the surrounding JS code.

Comment 18

[David Maciejak]

what do you mean, you want a poc ?

Comment 19

[Benoit Jacob [:bjacob] (mostly away)]

I meant 2 separate things in comment 17:

* I would like you to get a native backtrace in GDB, i.e. "bt", when it aborts on the null deref with XPCOM_DEBUG_BREAK=abort.

* You already mentioned it would be hard to get a poc; so I thought that seeing at least the JS code surrounding the crash point would already help. Even if it's not runnable by itself. The JS stack you got tells you where the crash point is so you could just attach here the ~100 lines of JS code leading to this point... if that's OK with you :-)

Comment 20

[David Maciejak]

ok will rerun gdb and display the stacktrace.
For the code it will be hard as it's created on the fly in JS, but i will try to attach a minimal version of the fuzzer here, next week.

Comment 21

[David Maciejak]

the native backtrace is giving nothing as gdb is not finding any debugging symbols in the debug build, how can i do ?

Comment 22

[David Maciejak]

./fetch-symbols.py is not returning me valid result, just got a:

Fetching symbol index http://symbols.mozilla.org/firefox/firefox-7.0-Linux-20110824131855-symbols.txt
HTTP error 404 retrieving http://symbols.mozilla.org/firefox/firefox-7.0-Linux-20110824131855-symbols.txt

Comment 23

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to David Maciejak from comment #21)
> the native backtrace is giving nothing as gdb is not finding any debugging
> symbols in the debug build, how can i do ?

Hah, yes, the debug builds provided by mozilla don't have symbols and I never figured how to make the symbols server work.

What I had in mind was that you'd make a debug build yourself, from sources... but I do understand if you don't have time for that ;-)

If you do have time, instructions are given there:
https://developer.mozilla.org/En/Simple_Firefox_build

Comment 24

[David Maciejak]

no much time for that, i am trying to do a *simple* poc

Comment 25

[David Maciejak]

Attachment: fuzz case

Comment 26

[David Maciejak]

i uploaded the fuzz case, i tried to make it simple. Just uncompress the tgz.
I used to test it using CLI with cmd line below (of course adapt the path if needed)

./firefox -private-toggle  file:///fuzzcase/cross_fuzz_seed_ftnt.html -jsconsole

Comment 27

[Benoit Jacob [:bjacob] (mostly away)]

Thanks a lot! I can reproduce the crash now. Will investigate asap.

Comment 28

[David Maciejak]

yes don't hesitate to give some feedback, thx

Comment 29

[David Maciejak]

any news ? i believe this issue is also happening on windows, as i saw strange crashes too.

Comment 30

[Benoit Jacob [:bjacob] (mostly away)]

Sorry for the late reply. Here's what happened. When I first tried your fuzzer in Nightly I immediately got a crash, so I posted comment 27 above and started investigating, but in a debug build I couldn't reproduce a crash. I've since tried several more times but haven't been able to reproduce the crash reported here. I did reproduce bug 682637 though. Still trying.

Comment 31

[David Maciejak]

strange, maybe i can try to get latest nightly build and have a try too.

Comment 32

[David Maciejak]

Benoit, I was able to reproduce it using last debug build i found on 
http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2011-09-03-mozilla-beta-debug/
firefox-7.0.en-US.debug-linux-i686.tar.bz2 (md5=a21d6a749f9301510c8f108fef6c5071)


see below the how-to steps i used:

*i download the fuzzcase and uncrompress it
*download and uncompress the firefox-7.0.en-US.debug-linux-i686.tar.bz2, jump into the dir
*i removed the profile (.mozilla dir)
*run ./firefox to set some pref:
**in pref, tabs tab verify if 'open new windows in a new tab instead' is checked
**in pref, content tab verify 'block pop-up windows' is unchecked
**set dom.max_chrome_script_run_time and dom.max_script_run_time to 60

*run: ./firefox -private-toggle  file:///fuzzcase/cross_fuzz_seed_ftnt.html -jconsole
*waiting 3 min and got the crash, see below the extract

WARNING: Surface size too large (would overflow)!: file ../../../gfx/thebes/gfxASurface.cpp, line 397
PNG Encoder: (null)
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file ../../../../../content/html/content/src/nsHTMLCanvasElement.cpp, line 273
WARNING: NS_ENSURE_SUCCESS(rv, rv) failed with result 0x80004005: file ../../../../../content/html/content/src/nsHTMLCanvasElement.cpp, line 318
###!!! ASSERTION: You can't dereference a NULL nsRefPtr with operator->().: 'mRawPtr != 0', file ../../../dist/include/nsAutoPtr.h, line 1117

Comment 33

[David Maciejak]

Guys, any news ?

Comment 34

[Benoit Jacob [:bjacob] (mostly away)]

It's been very difficult for me to reproduce as I'm on a 64bit debian system with NVIDIA, and this can only be reproduced in 32bit builds. I first had to get the 32bit NVIDIA libraries going. Finally did it today. I can reproduce the assertion failure using the exact same build as you, but I can't using the standard Firefox 7 build (don't get any crash there). It seems that there is something that makes this only easy to reproduce in debug builds (even if asserts are disabled, null pointer derefs still crash release builds...). Investigating.

Comment 35

[Benoit Jacob [:bjacob] (mostly away)]

I needed to make a 32bit local build to be able to debug (since we don't have debug info for the archived debug builds), but I can't manage to on my 64bit debian box.

This is on hold until someone can reproduce in a local build with debug info. The steps to debug will be: break on this null refptr deref (not on earlier assertions), get a c++ stack and a js stack, figure what the JS code there is doing, minimize the testcase.

Comment 36

[David Maciejak]

ok thanks for the info, seems it's not crashing anymore in ff v7.0 on my xp 32bits too.

Comment 37

[David Maciejak]

Hi Benoit, I did some tests today. I believe the issue is still here :S
Below see what i get with Mozilla Firefox 7.0.1 on Linux 32 bits, i am trying to send you a poc too.

Program received signal SIGSEGV, Segmentation fault.
mozilla::gl::GLContext::MakeCurrent (this=0x0, aForce=0) at ../../../dist/include/GLContext.h:492
492	../../../dist/include/GLContext.h: No such file or directory.
	in ../../../dist/include/GLContext.h
(gdb) bt
#0  mozilla::gl::GLContext::MakeCurrent (this=0x0, aForce=0) at ../../../dist/include/GLContext.h:492
#1  0xb69a3099 in mozilla::WebGLContext::MakeContextCurrent (this=<value optimized out>)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/content/canvas/src/WebGLContext.h:477
#2  0xb69a93d6 in mozilla::WebGLContext::Viewport (this=0xa52697c0, x=0, y=0, width=1224, height=723)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/content/canvas/src/WebGLContextGL.cpp:3858
#3  0xb6d40f66 in nsIDOMWebGLRenderingContext_Viewport (cx=0xadcde080, argc=4, vp=0xb08c2460)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/obj-i686-linux-gnu/js/src/xpconnect/src/dom_quickstubs.cpp:33951
#4  0xb733a476 in CallJSNative (cx=0xadcde080, argsRef=..., construct=js::NO_CONSTRUCT)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jscntxtinlines.h:284
#5  js::Invoke (cx=0xadcde080, argsRef=..., construct=js::NO_CONSTRUCT)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jsinterp.cpp:656
#6  0xb75499d2 in js::Interpret (cx=0xadcde080, entryFrame=0xb08c23c0, interpMode=js::JSINTERP_NORMAL)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jsinterp.cpp:4091
#7  0xb733ad54 in js::Execute (cx=0xadcde080, script=0x9cf48690, scopeChain=..., thisv=..., type=js::EXECUTE_DIRECT_EVAL, evalInFrame=0x0, 
    result=0xb08c2398) at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jsinterp.cpp:908
#8  0xb735c4b3 in EvalKernel (cx=0xadcde080, call=..., evalType=DIRECT_EVAL, caller=0xb08c2318, scopeobj=...)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jsobj.cpp:1246
#9  0xb735ca1a in js::DirectEval (cx=0xadcde080, call=...) at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/jsobj.cpp:1309
#10 0xb74f6363 in js::mjit::stubs::Eval (f=..., argc=<value optimized out>)
    at /build/buildd/firefox-7.0.1+build1+nobinonly/build-tree/mozilla/js/src/methodjit/InvokeHelpers.cpp:399

Comment 38

[David Maciejak]

Attachment: second test case working on ff 7.0.1

Comment 39

[David Maciejak]

i just attached the case, please download fuzz_ftnt2.tgz and follow comment 32 to test. I can reproduce the issue on Linux but NOT on Windows xp.

Comment 40

[Benoit Jacob [:bjacob] (mostly away)]

Is this still reproducible?

Comment 41

[David Maciejak]

Hi Benoit,

here the one i got on ff 13.0.2 i downloaded from the site i ran on Linux i686 - Ubuntu 12.04:

https://crash-stats.mozilla.com/report/index/bp-cc841265-39ed-4c8c-bf98-77ce02120625

Comment 42

[David Maciejak]

moreover, is there a way to load the symbols from firefox-13.0.2.crashreporter-symbols.zip using gdb ?? i can't achieve to load them, got a file format not recognized.

Comment 43

[Scoobidiver (away)]

(In reply to David Maciejak from comment #41)
> https://crash-stats.mozilla.com/report/index/bp-cc841265-39ed-4c8c-bf98-
> 77ce02120625
It's a different issue. File a new bug.

Comment 44

[David Maciejak]

yes, i will. For now, i can't reproduce the [@ mozilla::WebGLContext::Viewport ] issue on 13.0.2

Comment 45

[David Maciejak]

https://bugzilla.mozilla.org/show_bug.cgi?id=767947 added for the GetMozBattery issue (seems related to the closed bug 740707)

Comment 46

[Benoit Jacob [:bjacob] (mostly away)]

Thanks for checking, and sorry that we were unable to actively fix this bug before it apparently went away by itself. Maybe you can close this bug as WORKSFORME if you can't reproduce the WebGL viewport crash anymore. Regarding loading symbols for nightly builds, I've never done it, instead what works well (and what every developer does) is to just make your own Firefox build which is easy enough ( https://developer.mozilla.org/En/Simple_Firefox_build ), preferably a non-optimized debug build (add --enable-debug and --disable-optimize to your mozconfig file).

Comment 47

[David Maciejak]

ok thanks