1495355 - Add explicit GC tuning modes (low memory, tab in background, etc)

Status: NEW

(Core :: JavaScript: GC, enhancement, P2)

Description

[Paul Bone [:pbone]]

Add a low memory mode to the spidermonkey GC.  This mode will (currently) do two things.  It'll reduce the maximum nursery size, and reduce the heap growth factor.  The mode will be entered when the GC receives either a MEM_PRESSURE or LAST_DITCH_GC event.

This will make collections more frequent and hopefully avoid some OOM crashes.  More tweeks could be added to this in the future, for example to sacrifice some responsiveness for more GC progress.

Comment 1

[Paul Bone [:pbone]]

I'd like a way to detect that an ongoing low memory condition has ended.  Smaug do you know of any events (like those in:
https://searchfox.org/mozilla-central/source/dom/base/nsJSEnvironment.cpp#339) to do this?  Alternatively the GC could revert to its normal mode if no low memory events were received for 30 seconds.

Comment 2

[Paul Bone [:pbone]]

20:21 <@emilio> pbone: I think there is a `MemPressure_Stopping` enum you can 
                use, though not sure how it's hooked into the global observer 
                topic stuff
20:21 <@emilio> pbone: looks like you can listen to the 
                `"memory-pressure-stop"` event
20:23 <@emilio> pbone: so you'd add another `strcmp` to that function

Thanks emilio!

Comment 3

[Olli Pettay [:smaug][[e-mail]]]

Last time I looked at, memory-pressure was working only on some platforms.
... and seems like so. Android only, and on other platforms it happens only when large allocation fails.

Comment 4

[Steve Fink [:sfink] [:s:]]

In generally, I'm always a little worried about going through extra effort to handle near-OOM situations, because in some fraction of those it won't do any good; something's gobbling up memory and if you shrink things down to let it run longer, it'll just consume more. My guess is that this is the majority of low-memory situations. The result is that things run slower before they crash, and you potentially lengthen the period it's swapping like crazy.

Especially in a post-Fission world, the better thing to do is terminate the thing that is consuming so much memory (assuming it's Firefox in the first place; it might be some other process.)

On the other hand, I'm sure there are situations where you could legitimately keep running smoothly if only you dropped memory usage down a bit.

Is there some way to estimate the relative frequency of these situations? Obviously, we could do it by implementing a low-memory mode and then doing A/B testing with telemetry, but is there some way that doesn't involve a lot of implementation effort?

Comment 5

[Paul Bone [:pbone]]

(In reply to Steve Fink [:sfink] [:s:] from comment #4)
> In generally, I'm always a little worried about going through extra effort
> to handle near-OOM situations, because in some fraction of those it won't do
> any good; something's gobbling up memory and if you shrink things down to
> let it run longer, it'll just consume more. My guess is that this is the
> majority of low-memory situations. The result is that things run slower
> before they crash, and you potentially lengthen the period it's swapping
> like crazy.
> 
> Especially in a post-Fission world, the better thing to do is terminate the
> thing that is consuming so much memory (assuming it's Firefox in the first
> place; it might be some other process.)
> 
> On the other hand, I'm sure there are situations where you could
> legitimately keep running smoothly if only you dropped memory usage down a
> bit.

I agree with both these points.  I hope to reduce the number of OOMs that are reported.  BUt I kinda feel differently about the degraded performance.  I assume that a system that's swapping like crazy is going to have degraded performance no matter what we do here, but those that don't, who would just get some low memory GC tuning mode, would prefer that _slightly_ lower performance rather than a crash.

> Is there some way to estimate the relative frequency of these situations?
> Obviously, we could do it by implementing a low-memory mode and then doing
> A/B testing with telemetry, but is there some way that doesn't involve a lot
> of implementation effort?

There may be a way to process the OOM crash reports and see how much free memory they have, place it into a histogram etc.  But these slight changes arn't going to take long to implement except for the problem smaug mentions with these events only existing on some platforms (I guess we should fix that also).  At least in spidermonkey itself we4 can activate this wn the GC is unable to get a new chunk.

Comment 6

[Paul Bone [:pbone]]

Definitly P2, it could have a good impact but it's not going to make it in 64.

Comment 7

[Jon Coppeard (:jonco)]

Given comment 4 and other team priorities I think this is P3.

Comment 8

[Paul Bone [:pbone]]

(In reply to Olli Pettay [:smaug] from comment #3)
> Last time I looked at, memory-pressure was working only on some platforms.
> ... and seems like so. Android only, and on other platforms it happens only
> when large allocation fails.

Looks like it's possible on Windows: http://dblohm7.ca/blog/2015/07/28/interesting-win32-apis/

Comment 9

[Gabriele Svelto [:gsvelto]]

(In reply to Paul Bone [:pbone] from comment #1)
> I'd like a way to detect that an ongoing low memory condition has ended. 
> Smaug do you know of any events (like those in:
> https://searchfox.org/mozilla-central/source/dom/base/nsJSEnvironment.
> cpp#339) to do this?  Alternatively the GC could revert to its normal mode
> if no low memory events were received for 30 seconds.

You can listen to the "memory-pressure-stop" event:

https://searchfox.org/mozilla-central/rev/c9272ef398954288525e37196eada1e5a93d93bf/xpcom/threads/nsThread.cpp#1467

If you listen to "memory-pressure" events to detect low-memory conditions be sure to check the data payload. The first time we enter a memory-pressure condition it's set to "low-memory". If we're still in a low-memory condition after 30s all following events will have the payload set to "low-memory-ongoing". IIRC we don't run the GC/CC anymore in this case as it won't do much good. You can find more info on memory pressure events here:

https://searchfox.org/mozilla-central/source/xpcom/base/nsIMemory.idl

Note that we currently support detecting low-memory conditions only on 32-bit Windows (where we detect both low free commit-space and low free virtual address space), 64-bit Windows (low free commit-space only) and Android (low free memory). We haven't used CreateMemoryResourceNotification()/QueryMemoryResourceNotification() because it's unclear what thresholds it uses. Old Microsoft documentation suggests that it's 32 MiB per 4 GiB of memory which is too low for our purposes, it also doesn't mention what that value is being measured against so it's unclear if that can be used to monitor free address space on 32-bit Windows.

Comment 10

[Paul Bone [:pbone]]

Maybe add a test to fail failliable allocations when in this mode, even if the memory is available now. Then later allocations are less likely to fail.

Comment 11

[Jan Henning [:JanH]]

You can listen to the "memory-pressure-stop" event

Although at the moment that is hooked up only on Android, and even that only in Fennec, as we still haven't decided who should be responsible for memory pressure notifications in GeckoView.

Comment 12

[Jon Coppeard (:jonco)]

Low-memory notifications are fired by the AvailableMemoryTracker class:

https://searchfox.org/mozilla-central/source/xpcom/base/AvailableMemoryTracker.cpp#42

It seems like this could be updated to fire memory-pressure-stop when/if system recovers.

Comment 13

[Paul Bone [:pbone]]

I spoke with mconley, Gabriel & jesup about this kind of thing at Whistler. Based on that I want to re-title it and add some general "modes" to the GC/engine that affect tuning in more ways. We can think of:

• Normal foreground behaviour
• Tab is in the background
• System is low on memory

We also talked about having a 2nd "system is low on memory" threshold for more extreme options.

I intend to add an API to the engine that allows the embedding to choose the mode. I'll try to implement the modes from gecko's side as I've been given some info for how to do this (at least for background tabs), and there's some other info also on this bug.

Comment 14

[Paul Bone [:pbone]]

here's some info from mconley about background tabs.

23:59 < mconley> pbone: hey - we talked in Whistler about an observer
notification for when all tabs are in the background. At least
on Windows, you might be able to take advantage of the process
priority manager stuff, where we lower the process priority of
content processes whose tabs are all in the background
23:59 < mconley> when we change process priority, we send a message to the
content process, which is received here:
https://searchfox.org/mozilla-central/source/dom/ipc/ContentChild.cpp#2828-2840
Day changed to 26 Jun 2019
00:00 < mconley> pbone: so you could listen for the
ipc:process-priority-changed observer notification, and then
check the "priority" property being passed as the subject
00:00 < mconley> which will be a hal::ProcessPriority

Comment 15

[Caspy7]

I don't know where the proper place to bring this up is, but I filed bug 1524104, which is about lowering priority for offscreen iFrame processes.

It was pointed out that there may be multiple iFrames associated with processes, some of which are visible, so we'd need to ensure that all of them are hidden.

Perhaps it would be helpful to think of the "Tab is in the background" item from comment 13 as "Associated content is hidden".

I filed the bug in the first place when I learned that some pages may contain several separate origins.