Closed
Bug 752887
Opened 12 years ago
Closed 12 years ago
Increased memory usage for endurance tests on Linux Aurora branch
Categories
(Mozilla QA Graveyard :: Mozmill Tests, defect)
Tracking
(Not tracked)
RESOLVED
INVALID
People
(Reporter: mihaelav, Unassigned)
Details
(Whiteboard: [MemShrink])
The memory usage started to slightly increase on Linux builds (x86 and x86_64) since May 5, for Aurora branch. Watching the reports, the tests that show more memory usage are: - /testTabbedBrowsing_OpenNewTab/test1.js testOpenNewTab - /testBookmarks_OpenAllInTabs/test1.js testOpenAllBookmarksInTabs Charts: http://mozmill-ci.blargon7.com/#/endurance/charts?branch=14.0&platform=Linux&from=2012-04-29&to=2012-05-08 Reports: * May 3: - Linux x86: http://mozmill-ci.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c7095a7d56 - Linux x86_64: http://mozmill-ci.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c70959a4cc * May 7: - Linux x86:http://mozmill-ci.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c7098e89e5 - Linux x86_64: http://mozmill-ci.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c7098e64ff
20120505042007 en-US x64: Max memory of 80MB 20120507125428 en-US x64: Max memory of 102MB Note that results are missing for May 6th. Mihaela, can you please track down a hg pushlog of changes between those two dates to see if this is a regression in Firefox Aurora?
Comment 2•12 years ago
|
||
Interesting that the testrun duration dropped at around the same time...
Comment 3•12 years ago
|
||
One difference is that the later test is using Mozmill 1.5.12, which changed the way we reference open windows. Could you run two identical testruns against Aurora where the only difference is the version of Mozmill in use, and report the results here. If there's no/negligible memory difference then we can resume investigation of a Firefox regression. It looks like the difference is more noticeable on 64bit so may be worth attempting to replicate on that platform.
Mihaela, can you please investigate along the lines Dave has pointed out in comment 3? Thanks.
Reporter | ||
Comment 5•12 years ago
|
||
(In reply to Dave Hunt (:davehunt) from comment #3) > Could you run two identical testruns > against Aurora where the only difference is the version of Mozmill in use, > and report the results here. I ran the endurance tests on Ubuntu 11.10 64bit on Aurora Branch with both Mozmill 1.5.11 and 1.5.12 and the results are similar. Moreover, there are no significant differences between May 5 and May 10 when running the tests locally. --- build 20120510042005 --- *Mozmill 1.5.11: http://mozmill-crowd.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c709c61803 * Mozmill 1.5.12: http://mozmill-crowd.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c709c60efa --- build 20120505042007 --- *Mozmill 1.5.11: http://mozmill-crowd.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c709c620bd * Mozmill 1.5.12: http://mozmill-crowd.blargon7.com/#/endurance/report/c2b72632f20450b6d99d14c709c64cfc
Reporter | ||
Comment 6•12 years ago
|
||
(In reply to Mihaela Velimiroviciu [QA] from comment #5) > (In reply to Dave Hunt (:davehunt) from comment #3) > > Could you run two identical testruns > > against Aurora where the only difference is the version of Mozmill in use, > > and report the results here. > > I ran the endurance tests on Ubuntu 11.10 64bit on Aurora Branch with both > Mozmill 1.5.11 and 1.5.12 and the results are similar. ...which is: Max explicit memory: ~100MB Max resident memory: ~130MB
So just for comparison, average resident: May 7 local: 95MB (1.5.11) | 95MB (1.5.12) May 5 local: 95MB (1.5.11) | 96MB (1.5.12) May 7 live: 100MB (1.5.12) May 3 live: 90MB (1.5.11) One important distinction I think we may have missed is the plugins installed... Local: Windows Media Player Plug-in 10 (compatible; Totem) (enabled, compatible) QuickTime Plug-in 7.6.6 (enabled, compatible) VLC Multimedia Plugin (compatible Totem 3.0.1) (enabled, compatible) DivX® Web Player (enabled, compatible) Live: Shockwave Flash (enabled, compatible) iTunes Application Detector (enabled, compatible) IcedTea-Web Plugin (using IcedTea-Web 1.1.3 (1.1.3-1ubuntu1.1)) (enabled, compatible) QuickTime Plug-in 7.6.6 (enabled, compatible) DivX® Web Player (enabled, compatible) Windows Media Player Plug-in 10 (compatible; Totem) (enabled, compatible) VLC Multimedia Plugin (compatible Totem 3.0.1) (enabled, compatible) Mihaela, perhaps you can retest this with the same plugins installed?
Reporter | ||
Comment 9•12 years ago
|
||
Environment: * Ubuntu 11.10 64bit * Plugins: Shockwave Flash (enabled, compatible) IcedTea-Web Plugin (using IcedTea-Web 1.1.3 (1.1.3-1ubuntu1.1)) (enabled, compatible) VLC Multimedia Plugin (compatible Totem 3.0.1) (enabled, compatible) Windows Media Player Plug-in 10 (compatible; Totem) (enabled, compatible) QuickTime Plug-in 7.6.6 (enabled, compatible) DivX® Web Player (enabled, compatible) (could not find the Tunes Application Detector plugin) I've run the tests locally with May 5 and May 10 builds and the results are again similar with both Mozmill 1.5.11 and 1.5.12: * Mozmill 1.5.12 average resident 98-99MB: May 10: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a100b32 May 5: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a10228f * Mozmill 1.5.11 average resident 99-100MB May 10: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a1176c2 May 5: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a110eeb
Comment 10•12 years ago
|
||
And what about May 3rd? Please test using the exact same builds as you reported in comment 0 to eliminate variability.
Reporter | ||
Comment 11•12 years ago
|
||
(In reply to Anthony Hughes, Mozilla QA (irc: ashughes) from comment #10) > And what about May 3rd? Please test using the exact same builds as you > reported in comment 0 to eliminate variability. On May 3 build, the results are comparable with other builds: average resident memory 99-100MB. * Mozmill 1.5.11: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a13690f * Mozmill 1.5.12: http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a13768e
Comment 12•12 years ago
|
||
I have replicated this using our Mozmill CI system with the only difference being the Firefox build, and therefore suspect a regression. The memory increase looks like it is across the entire testsuite, but /testTabbedBrowsing_OpenNewTab/test1.js:testOpenNewTab appears to be the most noticeably worse. Firefox 15.0a1 (15.0a1, en-US, 20120504030509) Average explicit: 55MB Average resident: 84MB http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a13b569 Firefox 15.0a1 (15.0a1, en-US, 20120508030517) Average explicit: 71MB Average resident: 96MB http://mozmill-crowd.blargon7.com/#/endurance/report/f87375a634b1a5ba746e5f763a13c4da I'm concerned that we're not able to replicate this locally. Miheala: Can you confirm that you cannot replicate using the builds mentioned above? If not, I can try replicating it locally here. We need to identify the changeset that caused this potential regression.
Comment 13•12 years ago
|
||
Dave, can you get other tests triggered via mozmill-ci for the builds from May 6th and then 5th or 7th? So we can at least reduce it to a single day.
Comment 14•12 years ago
|
||
Done. It looks like it regressed in 20120505030510. Here's the pushlog: http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=2db9df42823d&tochange=0a48e6561534
Comment 15•12 years ago
|
||
(In reply to Dave Hunt (:davehunt) from comment #14) > Done. It looks like it regressed in 20120505030510. > > Here's the pushlog: > http://hg.mozilla.org/mozilla-central/ > pushloghtml?fromchange=2db9df42823d&tochange=0a48e6561534 Wow, those are too many changes. Looks like we have to try to get it reproduced locally and do a hg bisect on it. Also adding some MemShrink folks here.
Whiteboard: [MemShrink]
Comment 16•12 years ago
|
||
I'm a little confused. Was this regression on Aurora or Nightly? The bug talks about Aurora, but the regression range in comment 14 is from Nightly. That changeset includes the landing of compartment-per-global (CPG), which had an expected memory regression of something like 5-10%. It looks like you are seeing more than that. njn, maybe this is due to memory reporters taking more memory? Does that apply to whatever method the endurance tests may be using or just to the generation of the actual about:memory page?
Comment 17•12 years ago
|
||
You are right, it was reported against Aurora and I have been able to replicate it against Nightly. I can also run the tests against the Aurora nightly builds to see when it regressed there. It's possible that this is two separate regressions.
Comment 18•12 years ago
|
||
I'm unable to replicate a memory regression on Aurora (in fact, the results across builds are surprisingly consistent at the moment). We can either fork this bug for the Nightly regression, or make it specific to the Nightly regression.
Reporter | ||
Comment 19•12 years ago
|
||
(In reply to Dave Hunt (:davehunt) from comment #18) > I'm unable to replicate a memory regression on Aurora (in fact, the results > across builds are surprisingly consistent at the moment). We can either fork > this bug for the Nightly regression, or make it specific to the Nightly > regression. There was a similar bug reported for Nightly, as well: bug #754267
Comment 20•12 years ago
|
||
I knew I'd seen Nightly mentioned somewhere. We should probably reopen that bug and investigate separately, seeing as we have been able to replicate on Nightly but not Aurora. Any objections?
Comment 21•12 years ago
|
||
Closing this and moving investigation to Nightly in bug 754267, where we have been able to replicate the memory regression.
Status: NEW → RESOLVED
Closed: 12 years ago
Resolution: --- → INVALID
Updated•5 years ago
|
Product: Mozilla QA → Mozilla QA Graveyard
You need to log in
before you can comment on or make changes to this bug.
Description
•