673851 - Endurance tests find a regression in 'explicit' memory usage from FF7 to FF8

Status: RESOLVED INVALID

(Core :: General, defect)

Description

[Nicholas Nethercote [inactive]]

Attachment: stress test results

Dave Hunt blogged about some endurance test results:  http://blargon7.com/2011/07/endurance-tests-demonstrate-firefoxs-memory-usage-improvements/

The weird thing is that going from FF7 (current Aurora) to FF8 (current Nightly), the 'explicit' went up a lot (~25% for the average 'explicit') but the 'resident' went down slightly (a couple of percent for average 'resident').

That's an odd result.  It makes it sound like the meaning of 'explicit' changed between FF7 and FF8, but AFAIK it didn't.  The way endurance tests measure 'explicit' did change recently (bug 671859) but in theory the results should be the same -- the computation of 'explicit' just moved from JS code to C++ code.

I couldn't reproduce the difference.  I measured my MemBuster-style stress test (http://valgrind.org/njn/mem.html) and got very little difference in both 'explicit' and 'resident' after opening all the tabs, and after closing them all again.  (If anyone tries this stress test, make sure you wait for the "finished" dialog box to pop up.)  I've attached a file containing all the snapshots.

(I also did a run where I ran it twice in succession, and FF8 had ~8% improvements in both 'resident' and 'explicit'.)

Dave:  are you able to re-run your endurance tests on FF7 and FF8 and then capture about:memory at the end?  That would be very instructive.

And if anyone else is able to demonstrate a significant difference in memory usage between FF7 and FF8, please speak up.

Comment 1

[Justin Lebar (not reading bugmail)]

If there was memory which was mapped but not reported and is now mapped and reported, that would cause an apparent increase in "explicit", right?

Comment 2

[Nicholas Nethercote [inactive]]

Yes, any KIND_NONHEAP reporters added will change the results.  Here's the list of all the NONHEAP reporters:

All platforms:
- explicit/js/stack
- explicit/js/*/mjit-code
- explicit/js/*/tjit-code

Mac-only (because the GC heap is non-heap, due to the lack of jemalloc):
- explicit/js/*/gc-heap/*
- explicit/js/gc-heap-chunk-unused
- explicit/js/gc-heap-chunk-admin

AFAICT all of them are present in FF7.  The fact that some of those reporters are Mac-only makes me wonder, though, since Dave's results are on Mac, and I couldn't reproduce them on Linux.

Hmm, one possibility.  I added multi-reporters in bug 666075.  The JS code I wrote for the endurance tests that computes 'explicit' didn't handle them, because I wrote it before I added multi-reporters.  So if the endurance tests are using my code, they'll under-report on Mac (but only on Mac, due to the JS heap being NONHEAP).  But that'll happen in both FF7 and FF8...  Dave -- for the FF8 measurements, have you switched the endurance tests to use |nsIMemoryReporterManager::explicit| to get the 'explicit' number?  If so, that would explain it -- the FF7 explicit numbers would be artificially low in that case.

Comment 3

[Nicholas Nethercote [inactive]]

Dave Hunt said in today's MemShrink meeting that the explanation in the last paragraph of comment 2 is plausible.  Given that the "resident" numbers in FF8 are lower than FF7, I'm going to close this.

Dave, would it be worth adding an update to your article to say that the FF7 "explicit" numbers are suspect?

Comment 4

[Dave Hunt [:davehunt] [he/him] ⌚BST]

So far I have only run the membuster test on Mac. I can also confirm that I am using the new |nsIMemoryReporterManager::explicit| to get the 'explicit' number. I have updated my blog post as suggested.