Last Comment Bug 672563 - Back out bug 663251 so we run the profiling step only once on Linux
: Back out bug 663251 so we run the profiling step only once on Linux
Status: RESOLVED FIXED
[inbound]
:
Product: Core
Classification: Components
Component: Build Config (show other bugs)
: unspecified
: x86_64 Linux
: -- normal (vote)
: mozilla8
Assigned To: Nobody; OK to take it and work on it
:
: Gregory Szorc [:gps]
Mentors:
Depends on: 663251
Blocks: 653961
  Show dependency treegraph
 
Reported: 2011-07-19 10:44 PDT by Justin Lebar (not reading bugmail)
Modified: 2011-07-20 06:51 PDT (History)
5 users (show)
See Also:
Crash Signature:
(edit)
QA Whiteboard:
Iteration: ---
Points: ---
Has Regression Range: ---
Has STR: ---


Attachments
Backout v1, takes us back to dc8d154f3710 (2.70 KB, patch)
2011-07-19 10:47 PDT, Justin Lebar (not reading bugmail)
ted: review+
Details | Diff | Splinter Review

Description Justin Lebar (not reading bugmail) 2011-07-19 10:44:09 PDT
In bug 663251, we modified profile.py so it accepts a number N and runs the profiling step for the build N times.  Later, we checked in a build-config change setting N to 10 on Linux.

The hope was that this would improve our benchmark scores (in particular, that it would decrease the variance in the scores).  This did not appear to happen when we checked in the build-config change.

Some time after we checked in this change, however, we did see a decrease in scores' variance.  (See bug 653961 comment 87.)

So it appears that one of two things happened.  Either

 * running the profiling step multiple times had no effect on benchmark scores, or
 * the build-config change didn't take effect immediately, for some reason.

I'd like to know, both to make progress on the mystery that is bug 653961 comment 87, and so we don't run the profiled build 10 times on Linux if we don't need to.

As far as I can tell, the only way to know for sure would be to back out bug 663251's changes to profile.py and see how that affects benchmark scores on m-i and m-c.  Tryserver has been shown not to be useful for this investigation, since I saw there performance increases which haven't manifested since, even after the mysterious change in bug 653961 comment 87.

Similarly, I think we should back out specifically the changes to profile.py, rather than the changes to the build-config, because I have more confidence that the profile.py change will take effect immediately.
Comment 1 Justin Lebar (not reading bugmail) 2011-07-19 10:47:10 PDT
Created attachment 546827 [details] [diff] [review]
Backout v1, takes us back to dc8d154f3710
Comment 2 Ted Mielczarek [:ted.mielczarek] 2011-07-19 11:03:03 PDT
Comment on attachment 546827 [details] [diff] [review]
Backout v1, takes us back to dc8d154f3710

Whatever you want to do is fine with me.
Comment 3 Justin Lebar (not reading bugmail) 2011-07-19 11:16:30 PDT
inbound: http://hg.mozilla.org/integration/mozilla-inbound/rev/4b96a9cd9b15
Comment 4 Marco Bonardo [::mak] 2011-07-20 06:51:56 PDT
http://hg.mozilla.org/mozilla-central/rev/4b96a9cd9b15

Note You need to log in before you can comment on or make changes to this bug.