1413218 - Malloc allocation GC threshold grows too aggressively

Status: RESOLVED FIXED

(Core :: JavaScript: GC, enhancement, P3)

Description

[Jon Coppeard (:jonco)]

There are a couple of bugs reporting OOMs when loading large PDFs, and this seems to be because the malloc threshold can grow without limit resulting in very large heap sizes.  

For example bug 1412794 reports FF getting killed by the OS on a 4GB system due to excessive memory use.

We do still want this threshold to be dynamic so that we don't constantly GC when we are allocating large amounts of memory.  However, we could let it grow more slowly and we could put a maximum limit on it.

Comment 1

[Jon Coppeard (:jonco)]

Attachment: bug1413218-limit-malloc-threshold-growth

Here's a patch to reduce the growth factor to 1.5 (from 2) and impose a 1GB limit on the malloc bytes threshold.

Comment 2

[Steve Fink [:sfink] [:s:]]

Comment on attachment 8923917 [details] [diff] [review]
bug1413218-limit-malloc-threshold-growth

Review of attachment 8923917 [details] [diff] [review]:
-----------------------------------------------------------------

The thresholds are a little arbitrary, and it got me thinking about the ramp-up case where you really are allocating a bunch of memory and we're going to do log_1.5(n) steps to get there. But sure, why not give this a try?

Comment 3

[Pulsebot]

Pushed by [e-mail]:
https://hg.mozilla.org/integration/mozilla-inbound/rev/8fb7879b388f
Make the malloc threashold grow a little slower r=sfink

Comment 4

[Sebastian Hengst [:aryx] (needinfo me if it's about an intermittent or backout)]

https://hg.mozilla.org/mozilla-central/rev/8fb7879b388f

Comment 5

[Eric Rahm [:erahm]]

(In reply to Steve Fink [:sfink] [:s:] from comment #2)
> Comment on attachment 8923917 [details] [diff] [review]
> bug1413218-limit-malloc-threshold-growth
> 
> Review of attachment 8923917 [details] [diff] [review]:
> -----------------------------------------------------------------
> 
> The thresholds are a little arbitrary, and it got me thinking about the
> ramp-up case where you really are allocating a bunch of memory and we're
> going to do log_1.5(n) steps to get there. But sure, why not give this a try?

Not sure how useful it is, but we have a slightly more convoluted method in nsTArray [1] to help avoid heap churn (exponential up to 8MB), but also have a sane large growth strategy (grow by 1.125). Might be worth a follow up investigation.

[1] http://searchfox.org/mozilla-central/rev/423b2522c48e1d654e30ffc337164d677f934ec3/xpcom/ds/nsTArray-inl.h#148-166