1770768 - Use a square-root based heap limit rule for the main GC heap

Status: RESOLVED FIXED

(Core :: JavaScript: GC, task, P3)

Description

[Jon Coppeard (:jonco)]

First of all, use a square-root based rule for the main GC heap limit. Later we can apply this rule to other heap limits (malloc and JIT memory).

Comment 1

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 1: Add a GC parameter to enable or disable balanced heap limits r?sfink

Comment 2

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 2: Add a GC parameter for the heap growth factor used when balanced heap limits are enabled r?sfink

Depends on D152862

Comment 3

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 3: Add browser prefs for the new GC parameters r=mccr8!,sfink!

Depends on D152863

Comment 4

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 4: Calculate the garbage collection rate r?sfink

This estimates the collection rate per zone based on GC time (including
off-thread sweeping) and zone size pre-GC. The result is smoothed using an
exponential moving average.

We don't have per-zone time information for main thread GC time, so we estimate
proportionally based on the zone's initial size. The patch records per-zone
sweeping time and includes that in the calculation. Compacting time is not
included since that is relatively rare and will skew the result.

Depends on D152864

Comment 5

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 5: Calculate the allocation rate r?sfink

Calculate the allocation rate per zone. This requires remembering the previous
heap size and also tracking the amouth of memory freed in the mean time.

This uses an exponential moving average to smooth the result but with a much
lower factor, so that the allocation rate decays slowly when a zone stops
allocating.

Depends on D152865

Comment 6

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 6: Calculate balanced heap limits r?sfink

This patch adds the calculation of the heap limit based on the formula from the paper.

Depends on D152866

Comment 7

[Jon Coppeard (:jonco)]

Attachment: Bug 1770768 - Part 7: Update heap limits periodically r=sfink!,mccr8!

Once a second, update the heap limits based on the current allocation rate.
When zones become inactive the allocation rate falls and so does the heap
limit. This results in use collecting idle zones and relaiming memory.

To update the allocation rate periodically we need to use a timer.
Unfortunately we don't have that facility in SpiderMonkey itself so we have to
add some embedding callbacks to let us do this.

I found it was simpler to add callbacks to start and cancel a timer rather than
having a public API and telling the embedding to call it periodically.

The timer is used to update the heap limits once per second, except when the
allocation rate on all zones falls low enough, in which case it is restarted on
the next minor GC.

GCs triggered by reducing heap limits get their own GC reason.

This is only implemented in the browser. I'm not sure it's worth implementing
for the shell. Heap limits are still updated on GC start.

Depends on D152867

Comment 8

[Jon Coppeard (:jonco)]

I plan to land this preffed off and enable it in a separate bug.

Comment 9

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/autoland/rev/d09ccb4a3275
Part 1: Add a GC parameter to enable or disable balanced heap limits r=sfink
https://hg.mozilla.org/integration/autoland/rev/fec995e1ef8e
Part 2: Add a GC parameter for the heap growth factor used when balanced heap limits are enabled r=sfink
https://hg.mozilla.org/integration/autoland/rev/3beb50f4cdc4
Part 3: Add browser prefs for the new GC parameters r=mccr8,sfink
https://hg.mozilla.org/integration/autoland/rev/576dd240c167
Part 4: Calculate the garbage collection rate r=sfink
https://hg.mozilla.org/integration/autoland/rev/8e1a01843377
Part 5: Calculate the allocation rate r=sfink
https://hg.mozilla.org/integration/autoland/rev/39e7dcdae16e
Part 6: Calculate balanced heap limits r=sfink

Comment 10

[Sandor Molnar[:smolnar]]

https://hg.mozilla.org/mozilla-central/rev/d09ccb4a3275
https://hg.mozilla.org/mozilla-central/rev/fec995e1ef8e
https://hg.mozilla.org/mozilla-central/rev/3beb50f4cdc4
https://hg.mozilla.org/mozilla-central/rev/576dd240c167
https://hg.mozilla.org/mozilla-central/rev/8e1a01843377
https://hg.mozilla.org/mozilla-central/rev/39e7dcdae16e