Closed Bug 687256 (oomfuzz) Opened 13 years ago Closed 4 years ago

[meta] Need fuzz testing of OOM conditions to find exploitable crashes


(Core :: Fuzzing, defect)

Not set





(Reporter: briansmith, Assigned: decoder)


(Depends on 1 open bug, Blocks 1 open bug)


(Keywords: meta, sec-other, Whiteboard: [sg:nse meta])


(1 file)

+++ This bug was initially created as a clone of Bug #557545 +++

1. We need to have tests that cause malloc() and friends to return null in as many circumstances as possible, to detect exploitable bugs in the handling (or lack of handling) of allocation failures.

2. We need to have tests for the behavior of the browser in genuine out-of-memory situations, so that we can find exploitable bugs that only occur when third-party or operating-system code fails to allocate memory, using an allocator (like VirtualAlloc, or one internal to the library) that we may not be able to modify as we would for #1.

When I was working on bug 557545, I remember stumbling once or twice across (non-reproducible) DEP violations, which indicates to me that there is potentially a serious problem in this area.
Brandon: seems like something for your team to fit in somewhere.
Assignee: nobody → bsterne
Keywords: meta
Whiteboard: [sg:nse meta]
Benjamin Otte suggested a brilliant trick: on every malloc, fork and return null in the child.

I'd rather see bug 611123 fixed than try to test the current mess, though.
Attached file dtrace example
Hm, the crash-tester.c script is not available anymore on his website but to keep the brainstorming process up, I will attach a dtrace script. Although it will only find "double free" errors but dtrace might also be used for this kind of purpose. Execute it like this: dtrace -qs free.d -p <pid>
I had another idea which involves LLCov instrumentation on all locations where malloc is used. Once the fuzzer can see those locations, it can run our tests first, record all mallocs touched, then use gcparam (is that the right function in the shell?), to cause artificial OOMs at every single place in a linear way (just reducing till all malloc points have failed once).
Actually my last comment was referring to the JS engine, but I realized now that this bug has a wider scope. First of all, are we talking about an OS/Arch specific problem here or just the more general problem of failing to check if allocation succeeded? And secondly, which functions are we talking about, system malloc? Also, I'm trying to imagine which OOM specific failure could lead to e.g. DEP violations as comment 0 suggested. Not checking malloc's returned pointer will only null deref. Of course, an error handling path for OOM condition could contain a regular bug that is only triggered for this path, is that what you meant?
A eax[0+offset] for example would make a null ptr bug exploitable.

Exploiting zero sized heap allocations is probably also relevant to this topic:
(In reply to Christoph Diehl [:cdiehl] from comment #7)
> A eax[0+offset] for example would make a null ptr bug exploitable.

Only if offset is really large, which is usually not the case.

> Exploiting zero sized heap allocations is probably also relevant to this
> topic:

I'll take a look at this :)
I believe it depends on the subject. There were some issues in WebGL exactly like that. Also, most researchers stop examining a crash when they see ECX(0x0)+45 and often don't even know that the offset is controllable.
I wrote a patch that modifies moz_malloc, such that it will obtain a call trace when called, and search the call trace for a specific caller location. If that location is found, the function simulates OOM by returning NULL. It can also output the call traces (with configurable length).

What we could do with this is: 

1) Run a test with call trace output (with length e.g. 2-3)
2) Force OOM at every recorded point and observe crash

The problem is how to triage crashes, as I don't know what our expectation here is. I assume NULL crashes should be ok. What about near-NULL? And how far would we allow that?
Assignee: brandon → choller
Depends on: 731083
Depends on: 731087
Depends on: 731096
Depends on: 731167
Depends on: 731169
Depends on: 731176
Depends on: 731179
Depends on: 731184
Depends on: 731187
Depends on: 731191
I talked to khuey about the strategy here, and in general we should file all bugs that crash due to fallible allocation and do not use mozalloc_abort. That will also make this kind of testing a lot easier.
Depends on: 732941
Depends on: 732951
Depends on: 732955
Depends on: 733004
Depends on: 733009
Depends on: 733014
Depends on: 733031
Depends on: 733047
Depends on: 733112
Blocks: fuzz
Depends on: 737875
Depends on: 738383
Alias: oomfuzz
Group: core-security → dom-core-security

No activity for 8 years, closing

Closed: 4 years ago
Component: Security → Platform Fuzzing Team
Resolution: --- → FIXED
Summary: Need fuzz testing of OOM conditions to find exploitable crashes → [meta] Need fuzz testing of OOM conditions to find exploitable crashes
Group: dom-core-security → core-security-release
Group: core-security-release
You need to log in before you can comment on or make changes to this bug.