1129512 - Make OOM in gecko malloc clear GC buffers and retry

Status: RESOLVED WONTFIX

(Core :: Memory Allocator, defect)

Description

[Terrence Cole [:terrence]]

This will probably be difficult to hook up, but it should prevent a large number of OOMs; or at least delay the inevitable a few more milliseconds.

Comment 1

[Emanuel Hoogeveen [:ehoogeveen]]

I think the place to do this might potentially be mozalloc_handle_oom() [1]. I don't know if that can just pull in SM bits, but the comment starting on line 33 seems to imply that this is where any last ditch memory reclamation should happen.

Having said that, not all mozalloc functions [2] actually check for OOM. I assume this is intentional to avoid additional overhead for small allocations - really you'd want to have a memory pressure event before this point, ensuring that an OOM really is an OOM, but I'm not sure how good we are about sending those (especially on desktop).

[1] https://dxr.mozilla.org/mozilla-central/source/memory/mozalloc/mozalloc_oom.cpp#28
[2] https://dxr.mozilla.org/mozilla-central/source/memory/mozalloc/mozalloc.cpp

Comment 2

[Mike Hommey [:glandium]]

Going back into SM code from allocator code is a hard problem. Not impossible, but hard.

OTOH, I don't think malloc failures are really an interesting trigger for this:
- malloc will fail when allocating incredibly large buggers, most of the time because of bad maths, integer overflows, or whatnot. Freeing a few things won't fix this.
- on 32 bits systems, they can happen because of address space exhaustion: if the allocator needs to allocate new chunks of memory because all its current ones are full, it can fail to mmap a new one when there is no address space left. In this case, freeing a few things will help, but for how long?

Other than the above, malloc failure is hard to come by. The reality, though is that:
- On most desktop systems, the system will start being unresponsive because of swapping before malloc fails. And on Linux systems, the kernel will kill the process (oom killer) before malloc fails.
- On Android/B2G, there is no swap, and the system will run out of physical memory before running out of address space.

However, we do have system notifications for Low memory conditions, at least on Android/B2G (ISTR there is something like that on Windows, too). /That/ is IMHO the best timing to free some buffers.

Comment 3

[Steve Fink [:sfink] [:s:]]

Yeah, I was hoping this could be hooked up internally to malloc, before it decides whether to report an oom. The original thought was that if we've got a background freeing thread, it would be a good idea to join on that thread before giving up and declaring oom. (Of course, that would require finding the right thing to join on using only TLS or something.) Then terrence thought of other caches and stuff that could be dropped in an emergency.

Comment 4

[Steve Fink [:sfink] [:s:]]

glandium's probably right. I was just worried that on a platform like OSX where freeing can be very slow, we might start getting ahead of the background freeing thread and gradually accumulate a mound of dead garbage.

Comment 5

[Terrence Cole [:terrence]]

I agree 100% with glandium. That said, OOM small has been steady at about 1% of crashes for the last forever. This is never to be a major win; however, it's a pretty huge crash volume, so maybe the dregs is enough to make it worth doing?

Comment 6

[Mike Hommey [:glandium]]

I know we have a large OOM count for address space exhaustion on 32 bits windows. Is that the 1% crash volume you're talking about? If it is, I doubt freeing a few buffers will help. There are two main causes that I'm aware of: address space fragmentation, and address space leak from 3D drivers or related. Freeing a few buffers will only delay the inevitable.

Comment 7

[Terrence Cole [:terrence]]

Seems its probably not worth the investment. Please re-open or file a new bug if we have evidence otherwise.