Created attachment 636934 [details] AWSY screenshot On June 12 we saw a small spike in the blue, green and pink lines on AWSY. On June 13 we saw a larger spike in the red and purple lines on AWSY. (See the attached screenshot.) The changesets for June 12 are here: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=131961e5e0d1&tochange=733994f12c53 The following three from Bill seem relevant: - 828da55601e7 Bill McCloskey — Bug 761739 - Make timer-triggered GCs be incremental (r=smaug) - 3ecdf0d8b4fa Bill McCloskey — Bug 758034 - Create separate inter-slice GC timer (r=smaug) - 62d7c3276ceb Bill McCloskey — Bug 743396 - Don't unmark non-collected compartments during GC (r=mccr8) The changesets for June 13 are here: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=5581c3b2a95c&tochange=d39501066aa4 Only three patches, and they're almost certainly not relevant. So maybe the changes from June 12 didn't show up on the red and purple lines due to random variation. johns will do some additional AWSY runs with and without those three patches, to see if they really are the cause.
Linux builds for AWSY Base 943402d31225: https://tbpl.mozilla.org/?tree=Try&rev=d68163a77c4e +62d7c3276ceb: https://tbpl.mozilla.org/?tree=Try&rev=2eb4d8e071d0 +3ecdf0d8b4fa: https://tbpl.mozilla.org/?tree=Try&rev=83e97b47a21e +828da55601e7: https://tbpl.mozilla.org/?tree=Try&rev=8bc1983a5713
I ran two tests on each of the above four builds - before the patches, then with 62d7c3276ceb, 3ecdf0d8b4fa, and 3ecdf0d8b4fa https://areweslimyet.com/?series=bug768716&nocondense
Created attachment 637726 [details] AWSY with 2 runs of 4 configurations (In reply to John Schoenick [:johns] from comment #2) > I ran two tests on each of the above four builds - before the patches, then > with 62d7c3276ceb, 3ecdf0d8b4fa, and 3ecdf0d8b4fa > > https://areweslimyet.com/?series=bug768716&nocondense I've attached a screenshot. Note that the first two points are the base configuration, the next two are with the 1st patch applied, etc. There's enough noise that it's hard to tell, but the 1st patch looks innocent, the 2nd is probably guilty of some of the rise, and the 3rd patch might be guilty of some of the rise.
billm said to me on IRC: "I found a bug that causes us to retain too much stuff. I'll see if fixing it affects areweslimyet". He also said "fundamentally, though, incremental GC is expected to use more memory than non-incremental" :/
(In reply to Nicholas Nethercote [:njn] (away until August 8) from comment #4) > billm said to me on IRC: "I found a bug that causes us to retain too much > stuff. I'll see if fixing it affects areweslimyet". This eventually became bug 774104. There's a big gap in the areweslimyet data, but it seems likely that that patch fixed something. Memory usage is now more stable and the big pink spikes have disappeared. The red and purple lines seem to have gone back to their pre-IGC values. The pink line has regressed a little bit--from about 335MB to about 345MB. I don't think there's much I can do about this. Incremental GCs tend to collect more slowly than non-incremental, so memory usage will be a little higher. One other note: the "fresh start" lines seem to be higher than they were before IGC landed. However, I don't think that was my fault. They seem to have regressed at some time in the middle.
(The gap in AWSY data is due to bug 773533)
I think we decided this was mostly fixed.