666845 - GC doesn't get triggered for this javascript

Status: RESOLVED FIXED

(Core :: JavaScript Engine, defect)

Description

[Dima Tisnek]

User-Agent:       Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
Build Identifier: Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0

For the given html/javascript gc doesn't seem to catch up with javascript.
Happens on win32 and linux x86.

Reproducible: Always

Steps to Reproduce:
1. start empty firefox
2. open attached html in only page
3. watch memory usage grow

if another page is opened or this page is reloaded GC runs and memory usage drops; same for forceGC() in error console.

Actual Results:  
memory grows, system swaps, firefox gets oom'ed

Expected Results:  
memory stays constant or oscillates within reasonable range

Comment 1

[Dima Tisnek]

Attachment: test case

test case, this html+js causes memory footprint to grow

Comment 2

[Nicholas Nethercote [inactive]]

Dima, thanks for a very nice test case.

Can you try with a Nightly build from nightly.mozilla.org?  Bug 656120 was recently fixed and it sounds like it might solve the problem.  It would be great if you could give specific memory measurements (eg. "private bytes" from the Windows task manager) for Firefox 5.0 and the Nightly after a certain length of time, eg. 30 minutes or an hour or something.

Comment 3

[Dima Tisnek]

Nightly 7.0a1 (2011-06-24)

Memory usage oscillates in a saw pattern with 100s period, at first maxima get bigger and bigger but later appear to stabilize:

started 70M (win xp "Mem Used")
peak 80M, dip 77M
peak 122M, dip 78M
peak 124M, dip 81M
peak 127M, dip 82M
peak 130M, dip 83M
peak 139M, dip 83M
-- 10 minutes later --
peak 154M, dip 99M
peak 151M, dip 97M
peak 151M, dip 95M
peak 150M, dip 96M
peak 149M, dip 96M

So it looks like nightly is more or less fixed !:)
I'll run same and accelerated test on linux later today.

Btw., will fix for bug 656120 land in ff4 or only ff5 or later?

Comment 4

[Andrew McCreight [:mccr8]]

Bug 656120 won't land in either FF4 or FF5, which have already been released.  The soonest it could land would be in FF6, which I think is supposed to be out in August, but maybe not until later.

Comment 5

[Dima Tisnek]

Thanks, I suppose I meant backported rather than land. Anyway my unattended js app can wait for ff6.

Btw, forgot to mention that I disabled spell checker in nightly so that comparison with 5.0 is fair. With spell checker enabled pattern is the same, cpu utilization and memory figures are higher.

About sawtooth pattern, I was able to capture more info with "more verbose" in about:memory, the only large differences between minima and maxima are:

/explicit/heap-unclassified: 29M vs 61M
/explicit/js/string-chars:  4.7M vs 16.5M
/explicit/js/object-slots:  2.4M vs 10.5M
/explicit/js/gc-heap:      19.9M vs 20.9M

Having ran my original, private test case on linux I observed that at first the period of the sawtooth is 10s, but after a while it becomes 100s. I think it's the same on windows, it's just harder to notice startup. It could be that firefox does quite a bit at startup (addon checks, update checks, etc) and triggers gc so that it appears to run every 10s.

Comment 6

[Dima Tisnek]

I was able to work around this on 5.0 with javascript.options.mem.high_water_mark=16

I originally tested 5.0 in virutalboxed linux with only 256MB RAM, that and default high_water_mark of 128 (MB?) were the reasons for OOM.

Would be nice if high water mark took into account system firefox ran on!

Comment 7

[Andrew McCreight [:mccr8]]

We're looking into a similar problem in Bug 655455, though that is more concerned with avoiding thrashing.  The recommended memory for Firefox 4 is 512MB, according to http://www.mozilla.com/en-US/firefox/system-requirements.html

Comment 8

[Gregor Wagner [:gwagner]]

(In reply to comment #6)
> I was able to work around this on 5.0 with
> javascript.options.mem.high_water_mark=16
> 
> I originally tested 5.0 in virutalboxed linux with only 256MB RAM, that and
> default high_water_mark of 128 (MB?) were the reasons for OOM.
> 
> Would be nice if high water mark took into account system firefox ran on!

The high_water mark only counts for additional mallocs and does not limit the actual heap size. So it triggers a GC if N bytes are allocated after the last GC.

javascript.options.mem.max is your friend :)

Comment 9

[Nicholas Nethercote [inactive]]

(In reply to comment #3)
> Nightly 7.0a1 (2011-06-24)
> 
> Memory usage oscillates in a saw pattern with 100s period, at first maxima
> get bigger and bigger but later appear to stabilize:
> 
> started 70M (win xp "Mem Used")
> peak 80M, dip 77M
> peak 122M, dip 78M
> peak 124M, dip 81M
> peak 127M, dip 82M
> peak 130M, dip 83M
> peak 139M, dip 83M
> -- 10 minutes later --
> peak 154M, dip 99M
> peak 151M, dip 97M
> peak 151M, dip 95M
> peak 150M, dip 96M
> peak 149M, dip 96M
> 
> So it looks like nightly is more or less fixed !:)
> I'll run same and accelerated test on linux later today.

You said you got OOMs with FF5 -- what kind of measurements did you see?  I'd like to put a number on how much memory usage dropped.

I'm going to change the resolution of this one to fixed.  Even if there's javascript option buried in about:config that could fix it, that doesn't mean the original report was invalid, it was actually very reasonable.  So bug 656120 comes to the rescue here.

Comment 10

[Dima Tisnek]

FF5.0 memory grew linearily from 60-70M until it got OOMed at 179M RSS on linux
my understanding is that gc never hit first 128M before OOM my case.

FF7.0a1 goes up and down by about 10M, it doesn't have initial linear ramp in my test case, could still be an issue if js allocated really fast though.