411199 - 3% Ts regression on 2008-01-04

Status: RESOLVED INCOMPLETE

(Core :: Layout, defect, P2)

Description

[[e-mail]]

looks like there was a Ts regression on linux & mac, maybe windows too, sometime between 10am and 1pm on Jan 4th:

http://graphs.mozilla.org/graph.html#spst=range&spstart=1199321290&spend=1199733341&bpst=Cursor&bpstart=1199321290&bpend=1199733341&m1tid=53252&m1bl=0&m1avg=0
http://graphs.mozilla.org/graph.html#spst=range&spstart=1199145600&spend=1199728812&bpst=Cursor&bpstart=1199145600&bpend=1199728812&m1tid=61146&m1bl=0&m1avg=0

windows is less clear:
http://graphs.mozilla.org/graph.html#spst=range&spstart=1199149956&spend=1199732420&bpst=Cursor&bpstart=1199149956&bpend=1199732420&m1tid=61186&m1bl=0&m1avg=0

(you'll have to set the 'range' to 7 days or so manually)

checkins in range:
http://bonsai.mozilla.org/cvsquery.cgi?module=PhoenixTinderbox&date=explicit&mindate=1199468827&maxdate=1199486827

looks like this is most likely mats' checkin for bug 389196. could this be something up with the boxes?

Comment 1

[Mats Palmgren (inactive)]

I'd be surprised if either of my checkins in that span would cause
a perf regression.  It looks to me like bug 410131 affects more than
the download manager though?  If so, this seems the more likely cause.

Comment 2

[Christian :Biesinger (don't email me, ping me on IRC)]

Not really, webbrowserpersist is only used for downloads, and so's nsExternalAppHandler.

Comment 3

[[e-mail]]

alice, could this be something up with the boxes?

Comment 4

[alice nodelman [:alice] [:anode]]

I have a page that collects all the relevant graph links, you can see the results for Ts here:

http://wiki.mozilla.org/Buildbot/Talos/Machines#Ts

It looks to me like there has been a slight elevation on windows starting around 1/4/2008 in the morning.  We try to keep three machines running trunk tests per platform so that we can have tie breaker boxes, ie if only one machine was elevated we would blame the machine - I can see the elevation happening on all three machines at pretty much the same time.

I see a similar jump in the linux graph.  The vista graph is a little too variable to see so slight a change.

I would give it a good chance that there was a regression in that window - I'm sorry that the data we collect isn't so steady that a 3% change is easy to spot, it can pretty easily disappear in the noise.  Way to go in finding this!

Comment 5

[Mike Schroepfer]

When the tree calms down perhaps we should try backouts of everything in the landing window to find the culprit...  Ts is pretty hard to move and we really don't want to take any regressions.

Comment 6

[Robert O'Callahan (:roc) (email my personal email if necessary)]

We should probably just close this.

Comment 7

[Mats Palmgren (inactive)]

Can we back out these changes locally on one of the Tinderboxes that
showed a regression?  I think we should avoid polluting repository
commit history for this test since we suspect it's something in the
test environment rather than code changes that caused it.
I can make the patch if someone is willing to run it locally on a
Tinderbox for a few cycles...

Comment 8

[Justin Wood (:Callek)]

resolving Incomplete, as it appears enough time has passed that tracking down the regression would be problematic at best.