Closed Bug 702942 Opened 14 years ago Closed 14 years ago

2011-11-15 nightly build memory regression

Categories

(Core :: General, defect)

Product:
Core
Component:
General
Platform:
x86
Windows 7
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED WORKSFORME

People

(Reporter: dindog, Unassigned)

References

(Blocks 1 open bug)

Details

(Whiteboard: [MemShrink])

http://hg.mozilla.org/mozilla-central/rev/9ae1d4f44b8b Notice 20111115 build consume more memory, especially the 'heap-unclassified'. so I did a simple test, open 24 web-page, close them. wait GC work, then mark down heap-unclassified in about:memory after a minute. Comparing 2011-11-14 nightly build, repeat. 20111115: 14.98 (37.99%) 19.68 (44.88%) 22.77 (47.45%) 24.97 (48.83%) 25.70 (50.11%) 27.95 (53.63%) 30.78 (54.70%) 32.47 (57.67%) 34.54 (58.44%) It was getting bigger every time. and the previous build, namely 20111114 12.52 (34.36%) 13.72 (38.87%) 13.86 (39.06%) 14.52 (39.23%) 14.49 (39.60%) 14.09 (38.71%) It seems something go wrong in 20111115 build. And it's more obvious after a few hour. (each test open a bookmark folder contains the follow links 4 times(6*4=24): http://www.nytimes.com http://www.msn.com http://www.aol.com http://www.yahoo.com http://www.bbc.com http://www.washingtonpost.com )
Whiteboard: [MemShrink]
There aren't much landed on 20111115, I try some tinderbox build to narrow it down. it's between: (Normal) http://hg.mozilla.org/mozilla-central/rev/eb84780783ed (heap-unclassified issue) http://hg.mozilla.org/mozilla-central/rev/f694183357ec same test as comment1, the heap-unclassified amount after tabs closed and idle 2 minutes: eb84780783ed: 13.99, 14.90, 14.81, 14.90 (MB) f694183357ec: 13.60, 17.31, 19.57, 21.02 (MB)
Thanks for looking into this! I'd be inclined to blame bug 661746 as it looks like a big scary patch, but it was backed out the same day, so I wouldn't think that it would show up in the nightly. Does this still show up in the 10/16 nightly? Looking at my own heap-unclassified right now, it is 59%, which is pretty high. That's with the 11/15 nightly. Nick, do you see anything in that change set that might cause this?
Blocks: DarkMatter
> Nick, do you see anything in that change set that might cause this? Those two links are particular pushes, how do I see what happened between them, and how do I see patches were part of the inbound-to-central merge?
I think dindog means that eb84780783ed does not show the change, but f694183357ec does. You can see the set of patches in the merge by clicking on "pushlog" which will net you this: http://hg.mozilla.org/mozilla-central/pushloghtml?changeset=f694183357ec Tree-management shows a bunch of memory regressions for around this range, too: http://groups.google.com/group/mozilla.dev.tree-management/browse_thread/thread/37f67d022fbdcf92/4319addc7d19854a?lnk=gst&q=%22Delay+setting+slow+selector+flags+during+selector+matching%22#4319addc7d19854a Ed Morley is firing off some try runs.
I have had a massive increase in memory use in today's build. Heap-unclassified increases to about 400MB after 30 mins of browsing (~ 80% of allocated memory).
(In reply to Andrew McCreight [:mccr8] from comment #2) > Does this still show up in the 11/16 nightly? yes, 11/16 is the same as 11/15..
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:11.0a1) Gecko/20111116 Firefox/11.0a1 Possibly related (stackless) crash: https://crash-stats.mozilla.com/report/index/bp-f1f959d5-a573-4317-a4c5-3eed82111117 Firefox has been open for less than 15 minutes: 733.34 MB (100.0%) -- explicit ├──626.86 MB (85.48%) -- heap-unclassified ├───71.90 MB (09.80%) -- js │ ├──27.94 MB (03.81%) -- gc-heap-chunk-dirty-unused │ ├──27.72 MB (03.78%) -- compartment([System Principal], 0x7764000) │ │ ├──18.57 MB (02.53%) -- gc-heap │ │ │ ├───8.77 MB (01.20%) -- objects │ │ │ │ ├──4.78 MB (00.65%) -- function │ │ │ │ └──3.99 MB (00.54%) -- non-function │ │ │ ├───5.18 MB (00.71%) -- shapes │ │ │ │ └──5.18 MB (00.71%) -- (2 omitted) │ │ │ └───4.62 MB (00.63%) -- (5 omitted) │ │ └───9.15 MB (01.25%) -- (8 omitted) │ ├──11.87 MB (01.62%) -- (13 omitted) │ └───4.36 MB (00.59%) -- compartment(atoms) │ └──4.36 MB (00.59%) -- (2 omitted) ├───17.63 MB (02.40%) -- storage │ ├──16.47 MB (02.25%) -- sqlite │ │ ├───9.52 MB (01.30%) -- places.sqlite │ │ │ ├──9.26 MB (01.26%) -- cache-used [3] │ │ │ └──0.26 MB (00.04%) -- (2 omitted) │ │ └───6.95 MB (00.95%) -- (15 omitted) │ └───1.16 MB (00.16%) -- (1 omitted) ├────9.73 MB (01.33%) -- layout │ ├──5.25 MB (00.72%) -- (8 omitted) │ └──4.48 MB (00.61%) -- shell(about:blank) │ ├──3.80 MB (00.52%) -- styledata [53] │ └──0.68 MB (00.09%) -- (1 omitted) └────7.24 MB (00.99%) -- (9 omitted)
regarding the regression on Talos, this should be a narrower range http://hg.mozilla.org/integration/mozilla-inbound/pushloghtml?fromchange=15cf68a3c027&tochange=736d3db54c41 Note that Bug 685767 has already been identified as a memory offender in bug 703125. There may be more though.
adding a lazy dependency, to track possible relations.
Depends on: 703125
I have anecdotal RAM usage increase from Nov 15th Nightly http://forums.mozillazine.org/viewtopic.php?f=23&t=2364191&p=11471105#p11471105 Nightly has never gone over 500MB here before and right now is reaching 700MB and upwards today. Anything I could provide, just ask.
(In reply to Andrew McCreight [:mccr8] from comment #4) > Ed Morley is firing off some try runs. Could people try using: https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bmo@edmorley.co.uk-3d570cbf42aa/ (built from built from 09ad1943c19a) and: https://ftp.mozilla.org/pub/mozilla.org/firefox/try-builds/bmo@edmorley.co.uk-ca9af4dfae90/ (built from e6705f49eaaa) If bug 685767 is indeed to blame, then the first build should show the issue and the second not. (There are 3 other changesets in that range, but I can spin some more builds once we've narrowed it to that). Thanks :-)
Is it bug 685767 a hardware acceleration related bug? If that is the case, it would be strange because I first noticed this issue on my regular profile which HA never enabled. Test result shows it's related anyway, same test procedure as in comment1: first build heap-unclassified: 14.04 18.21 20.52 20.59 22.87 25.00 26.88(MB) second build: 12.83 13.82 14.40 15.40 14.59 15.05 14.71(MB) Then I did it again, the trend was same, first build became bigger every-time.
Bug 685767 has just been backed out on mozilla-central, so if you try the 2011-11-18 nightly, and that bug was indeed the cause of this too, then the issue should be gone using it.
Status: UNCONFIRMED → NEW
Ever confirmed: true
A commenter named ferongr said this on my blog: > I too had this issue since the Nov 15 Nightly. I downloaded the latest > hourly after the backout and everything seems to be smooth now. > > ─26.84 MB (22.54%) — heap-unclassified after 30 minutes or so of browsing, > where it would have skyrocketed up to 700, 85% of FF’s usage (a hard limit > before everything in Windows freezes due to paging). So that's a good sign. A couple of other people reported the same issue, the comment thread is here: http://blog.mozilla.com/nnethercote/2011/11/16/memshrink-progress-report-week-22/comment-page-1/#comment-4385
(In reply to Nicholas Nethercote [:njn] from comment #14) > A commenter named ferongr said this on my blog: > > > I too had this issue since the Nov 15 Nightly. I downloaded the latest > > hourly after the backout and everything seems to be smooth now. > > > > ─26.84 MB (22.54%) — heap-unclassified after 30 minutes or so of browsing, > > where it would have skyrocketed up to 700, 85% of FF’s usage (a hard limit > > before everything in Windows freezes due to paging). > > So that's a good sign. A couple of other people reported the same issue, > the comment thread is here: > http://blog.mozilla.com/nnethercote/2011/11/16/memshrink-progress-report- > week-22/comment-page-1/#comment-4385 Yup, I was about to head here after I made really sure about it and running some heavy pages (like G+). I did and I can confirm that in my case, the hourly from revision b62e6ee5ba9b does not leak anymore.
(In reply to John Volikas from comment #15) > Yup, I was about to head here after I made really sure about it and running > some heavy pages (like G+). I did and I can confirm that in my case, the > hourly from revision b62e6ee5ba9b does not leak anymore. Thanks for verifying with the hourly John.
(In reply to John Volikas from comment #15) > I made really sure about it and running some heavy pages (like G+). Come to this, should we provide some local as general testcases, so as long as a issue can reproduce with them, we used it, instead of some online website. Oct 1st, www.yahoo.com home page took 40MB memory. Oct 2nd, 50MB. No reliable conclusion can make here.
I mean local file, sorry
dindog: Mozilla does runs some automated tests on consistent pages, which actually did catch this regression (though I think it understated how bad it was for real browsing). The results are posted on mozilla.dev.tree-management, if you are curious. As Jim Jeffery said, this is probably (but not necessarily) a dupe of 703125, so I'm going to close this now. Please reopen if you saw problems with the 10/15 nightly that haven't gone away by today's nightly.
Status: NEW → RESOLVED
Closed: 14 years ago
Resolution: --- → WORKSFORME
You need to log in before you can comment on or make changes to this bug.