687480 - Memory usage spike detected in Mozmill Endurance tests

Status: VERIFIED INCOMPLETE

Description

[Anthony Hughes (:ashughes) [QA]]

There appears to be a memory spike of about 10MB between Sept 6 and Sept 7 on the Mac nightlies. 

In the two weeks previous, we averaged around 160MB consistently.
Since September 7th, we are averaging around 170MB consistently.

Report 2011-09-06 (163MB):
http://mozmill-release.brasstacks.mozilla.com/#/endurance/report/b2f105bf5adf3d9f861770c4a9b4d5d6

Report 2011-09-07 (173MB):
http://mozmill-release.brasstacks.mozilla.com/#/endurance/report/5fdc9d8340b415e9a2e6a934ad122af5

Changeset:
http://hg.mozilla.org/mozilla-central/rev/09935ede3c77

Comment 1

[Anthony Hughes (:ashughes) [QA]]

Pushlog between changesets:
http://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=d2c6783fa715&tochange=09935ede3c77

Comment 2

[Anthony Hughes (:ashughes) [QA]]

On a superficial pass, this looks like it could be a culprit:

Ed Morley — Backout a422b9ff0a9e (bug 591780 part 1) for causing 8% Ts regression on multiple platforms; a=khuey

http://hg.mozilla.org/mozilla-central/rev/db9e99d537f2

Comment 3

[Henrik Skupin (:whimboo)]

Hard to say. I would propose that we do our own builds and find the exact changeset this way. Dave or Anthony, can one of you do this?

Comment 4

[Anthony Hughes (:ashughes) [QA]]

(In reply to Henrik Skupin (:whimboo) from comment #3)
> Hard to say. I would propose that we do our own builds and find the exact
> changeset this way. Dave or Anthony, can one of you do this?

I won't have time for this due to Firefox 7 release testing. Dave, can you investigate?

Comment 5

[Kyle Huey [:khuey] (Exited; not receiving bugmail, email if necessary)]

The TI merge is *much* more likely.

Comment 6

[Henrik Skupin (:whimboo)]

(In reply to Kyle Huey [:khuey] ([e-mail]) from comment #5)
> The TI merge is *much* more likely.

I would agree. Lets add Brian and Nicholas to the bug for further input.

Comment 7

[Brian Hackett (:bhackett)]

The TI merge pulled in one feature patch (bug 663138) and several bug fixes.  The fixes should not have affected anything, and bug 663138 made a few memory-related changes:

- Moved some state from JSScript into TypeScript, which will be a net reduction in memory usage (there are far fewer TypeScripts than JSScripts).

- Allocate a small data structure for function scripts which are nested in each other.  These are unusual on the web, and the memory from these would not account for even remotely close to 6% of explicit memory.

It's possible TI is the culprit, but only if something unintentionally broke with that merge.  Did a similar regression show up on other platforms?

Comment 8

[:Ehsan Akhgari]

Anthony, it would be great if you can bisect the range in comment 1 to see which patch in the range is causing this.  Would you mind doing that please?

Comment 9

[Anthony Hughes (:ashughes) [QA]]

(In reply to Ehsan Akhgari [:ehsan] from comment #8)
> Anthony, it would be great if you can bisect the range in comment 1 to see
> which patch in the range is causing this.  Would you mind doing that please?

How do you "bisect the range"?

Comment 10

[Henrik Skupin (:whimboo)]

I can show it to you, but you will have to have the build system installed on your machine: https://developer.mozilla.org/En/Simple_Firefox_build#Mac

Comment 11

[Anthony Hughes (:ashughes) [QA]]

(In reply to Henrik Skupin (:whimboo) from comment #10)
> I can show it to you, but you will have to have the build system installed
> on your machine: https://developer.mozilla.org/En/Simple_Firefox_build#Mac

I'm going to be very busy over the next week or so with Firefox 7 release work. I'll approach our outsource PM to assign this out. Henrik, I will have whomever get in touch with you once assigned -- if that is okay?

Comment 12

[Dave Hunt [:davehunt] ⌚️UTC+1]

I've tried tracking down the changeset that caused this locally, however as the results of each run can vary I haven't had any success yet. I'll continue to look into this.

Comment 13

[Nicholas Nethercote [:njn]]

Any progress here?

Comment 14

[Dave Hunt [:davehunt] ⌚️UTC+1]

I wasn't able to identify the changeset that caused this. Unfortunately brasstacks is now down so it's not going to be possible to see the original reports. I've added bug 716671 as a dependency.

Comment 15

[Nicholas Nethercote [:njn]]

I suspect this bug will never go anywhere.  Please reopen if you disagree.

Comment 16

[Anthony Hughes (:ashughes) [QA]]

Yeah, I agree, the trail ran dead. We can open a new bug if we find a more concrete regression window to investigate.