673075 - Recursion in WebGL shader causes memory corruption

Status: RESOLVED FIXED

(Core :: Graphics: CanvasWebGL, defect)

Description

[Marc Schoenefeld]

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.122 Safari/534.30

Steps to reproduce:

gdb --args /Applications/Firefox.app/Contents/MacOS/firefox-bin

i) open http://www.iquilezles.org/apps/shadertoy/

ii) Paste the following into the script textarea 
     It basically recalls main() within the shader code    

==start paste==
#ifdef GL_ES
precision highp float;
#endif

uniform vec2 resolution;
uniform float time;
uniform sampler2D tex0;
uniform sampler2D tex1;

float reccount; 

void main(void)
{
    reccount = reccount + 1.0 ; 
    if (reccount > 60.0) {
    	return;
    } 
  
    main();
}
==end paste==
  

iv) Compile (alt+enter)



Actual results:

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_PROTECTION_FAILURE at address: 0x00007fff5f3fffc4
0x00007fff80b8d40f in glvmStoreDestTempWithMask ()


(gdb) i r 
rax            0x72041000	1912868864
rbx            0x3902	14594
rcx            0x7fff5f400250	140734791418448
rdx            0x7fff5f400480	140734791419008
rsi            0x7fff5fbfad60	140734799785312
rdi            0x7fff5fbfb5e0	140734799787488
rbp            0x7fff5f400240	0x7fff5f400240
rsp            0x7fff5f400038	0x7fff5f400038
r8             0x22000072041000	9570151121031168
r9             0x4000000000000022	4611686018427387938
r10            0xe408	58376
r11            0x7204	29188
r12            0x22	34
r13            0x7fff5fbfad50	140734799785296
r14            0x1c81	7297
r15            0x22	34
rip            0x7fff80b8d40f	0x7fff80b8d40f <glvmStoreDestTempWithMask+63>
eflags         0x10246	66118
cs             0x27	39
ss             0x0	0
ds             0x0	0
es             0x0	0
fs             0x0	0
gs             0x0	0
(gdb) disass $pc $pc+1
Dump of assembler code from 0x7fff80b8d40f to 0x7fff80b8d410:
0x00007fff80b8d40f <glvmStoreDestTempWithMask+63>:	mov    %r12d,-0x27c(%rbp)


(gdb) bt
#0  0x00007fff80b8d40f in glvmStoreDestTempWithMask ()
#1  0x00007fff80b9f89f in glvmInterpretFPTransformFourInner ()
#2  0x00007fff80b9d8fb in glvmInterpretFPTransformFourInner ()
#3  0x00007fff80b9d8fb in glvmInterpretFPTransformFourInner ()
...
#10671 0x00007fff80b9d8fb in glvmInterpretFPTransformFourInner ()
#10672 0x00007fff80b9d8fb in glvmInterpretFPTransformFourInner ()
#10673 0x00007fff80ba14d0 in glvmInterpretFPTransformFour ()
#10674 0x0000000114d51b3c in gldLLVMFPTransformFallback ()
#10675 0x0000000114d601a3 in gldLLVMVecPolyRender ()
#10676 0x0000000114d5c371 in gldSetPolyRenderFunc ()
#10677 0x0000000114d6d23f in gldTessellateTrianglesRGBA ()
#10678 0x0000000114d6d315 in gldRenderTriangles ()
#10679 0x0000000115828b51 in gleVPRenderTrianglesSmooth ()
#10680 0x0000000115795e13 in gleDrawArraysOrElements_ExecCore ()
#10681 0x000000011579c1e5 in glDrawArrays_ACC_Exec ()



Expected results:

looks like invalid write, probably those recursion constructs could be detected before they hit a place they could cause harm.

Comment 2

[Daniel Veditz [:dveditz]]

Benoit or Jeff: please investigate.

Comment 3

[Kelsey Gilbert [:jgilbert]]

At first glance, I would guess this is a problem with 'reccount' never being initialized, and likely starting at some large negative value, making the stack likely to explode. I'm not sure what the spec says re: uninitialized values.

Testing on 5 and 8 show that they both work for me, so maybe it depends on implementation/platform? (Tested on Win7 w/nVidia card)

Comment 4

[Benoit Jacob [:bjacob] (mostly away)]

http://www.opengl.org/wiki/GLSL_Core_Language#Recursion

"The GLSL memory model does not allow for recursive function calls. This allows GLSL to execute on hardware that simply doesn't allow for recursion."

So it seems like a bug in both ANGLE and the driver, that they allow recursion. We should get this fixed in ANGLE.

Comment 5

[Benoit Jacob [:bjacob] (mostly away)]

Here, I confirm that ANGLE allows recursion. I don't get the crash however because my NVIDIA driver correctly rejects the shader.

Comment 6

[Benoit Jacob [:bjacob] (mostly away)]

Mac: what's your GPU?

Comment 7

[Benoit Jacob [:bjacob] (mostly away)]

Filed ANGLE security bug:
http://code.google.com/p/angleproject/issues/detail?id=191

Comment 8

[Daniel Veditz [:dveditz]]

Marc: on your machine that exhibits this problem please open about:support and paste the "Graphics" block into this bug.

I think we've confirmed at least a DoS recursion bug, and possibly something exploitable with Marc's specific configuration that we have not been able to reproduce.

Comment 9

[Marc Schoenefeld]

MacBook Air late 2010 model that crashes: 
        
Adapter Description 0x22600,0x20400
Vendor ID 0000
Device ID 0000
Adapter RAM
Adapter Drivers
Driver Version
Driver Date
Direct2D Enabled false
DirectWrite Enabled false
WebGL Renderer NVIDIA Corporation -- NVIDIA GeForce 320M OpenGL Engine -- 2.1 NVIDIA-1.6.36
GPU Accelerated Windows 1/1 OpenGL

Comment 10

[Marc Schoenefeld]

Not crashing on iMac early 2010, ATI:  

Karten-Beschreibung 0x21a00,0x20400
Vendor-ID0000
Geräte-ID0000
Karten-Ram
Karten-Treiber
Treiber-Version
Treiber-Datum
Direct2D aktiviert false
DirectWrite aktiviert false
WebGL-Renderer ATI Technologies Inc. -- ATI Radeon HD 4850 OpenGL Engine -- 2.1 ATI-1.6.36
GPU-beschleunigte Fenster1/1 OpenGL

Comment 11

[Zhenyao Mo]

Recursion detection has been implemented in ANGLE r711.  Such shaders won't be allowed in WebGL when that revision is rolled in.

Comment 12

[Benoit Jacob [:bjacob] (mostly away)]

Imported ANGLE r711:

http://hg.mozilla.org/mozilla-central/rev/7a92e6b3093f

Comment 13

[Marc Schoenefeld]

Attachment: Crash scenario eating all available hard disk space

Illustrates crash scenario eating all available hard disk space , 51G in this case and DoSing the machine. The sligthly modified reproducer is calling main() two times per recursion level.

Comment 14

[Benoit Jacob [:bjacob] (mostly away)]

I don't understand how a shader can eat all disk space, but anyways, Firefox 8+ should now be rejecting recursive shaders.

Do we want to take the fix in Beta / Firefox 7? It adds a nontrivial amount of new code, but fixes a serious issue.

Comment 15

[Daniel Veditz [:dveditz]]

I can't see ANGLE bug 191 -- did they think this was worse than a recursion DoS? I also don't understand how a recursive shader uses any disk let alone 51G; swap maybe?

Comment 16

[daniel-bzmz]

I presume it was talking lots of memory and eventually swap.  Recursion isn't allowed in an version of GLSL so it's now properly detected.

Comment 17

[Benoit Jacob [:bjacob] (mostly away)]

(In reply to Daniel Veditz from comment #15)
> I can't see ANGLE bug 191 -- did they think this was worse than a recursion
> DoS?

I just initially filed it as a security bug, don't read anything specific into that. As Daniel says it's been fixed for a while anyways.

Comment 18

[u279076]

qa- as this requires a debug build to verify the fix. If someone has a debug build, please verify the fix using comment 0.