1134336 - Be less aggressive about allocating when out of memory

Status: NEW

(Core :: JavaScript: GC, defect)

Description

[Terrence Cole [:terrence]]

Looking at bug 1133247, I think how we handle chunk allocation failures is extremely sub-optimal. Specifically, we're trying so hard to pack every last allocation in that the rest of the system is crippled in its ability to cope under tight memory situations because the GC will just greedily use /everything/. Even worse, as we get close to the heap limit, we start GCing so frequently that the system becomes unusable. The user experience here is that the browser will appear to get sluggish, freeze, then crash.

We do have to be careful though: when we are actually out of memory (mmap failed), there are a couple possible scenarios:

  1) size(retained) > size(memory) for a single, large script.

     If we let the memory hog continue to run, it will just crash the system anyway. The system should run a LAST_DITCH to free up any memory it can for continued operation, then raise an OOM error so that at least we won't crash the entire browser. The website not working "in Firefox" (really on the user's hardware) is probably no worse or maybe better than "the website crashes Firefox." Ideally, we'd warn the user about the tab with something like the slow-script dialog to mitigate user frustration when the hog stops working.

     A second, unfortunate, follow-on problem here is that any memory reachable from a global will not go away until we actually fully close the tab. If the hog keeps allocating via independent events, we'll still end up in the same "firefox freezes on this site" problem. Given that this too is ultimately a resource policy issue, I don't think we'll even be able to address it effectively without a slow-script-like dialog.

  2) size(retained) > size(memory) for the multiple tabs the user has opened.

     Some straw has to break the camel's back, but the straw that does so may not be the best straw to take off the camel's back to fix the problem; i.e. the OOM killer problem. Ideally, we'd raise an OOM exception on the ad running in a background tab, rather than OOMing the gmail tab that the user is actively using and actually cares about. I'm not sure how we can address this, but at the moment, simply raising an OOM error somewhere rather than crashing the browser after an extended period of it being unusably slow would probably be preferable.

  3) size(retained) < size(memory), even though size(allocated) > size(memory)

     I am confident at this point that our GC triggers are robust enough that this is a case we do not really have to worry about in practice. We may want to consider checking how much memory was freed by the LAST_DITCH GC and letting the script continue if there appears to be tons of memory left now. I think it's probably not worth the extra complexity at the moment.

Upshot
------
Given the above, my plan is to make our LAST_DITCH trigger do the last-ditch GC, wait for the background sweeping, then immediately return nullptr from the failing allocation. Does this sound reasonable?

Comment 1

[Terrence Cole [:terrence]]

Jon pointed out on IRC that we scale the ALLOC_TRIGGER with heap size, so JS heavy workloads will probably have an issue with (3). Also, doing tab unloading for background tabs would be another way to address OOM here.