561699 - Reset try server repo, April/May 2010 edition

Status: VERIFIED FIXED

(Release Engineering :: General, defect, P2)

Description

[Phil Ringnalda (:philor)]

Apparently the reason that http://tests.themasta.com/tinderboxpushlog/?tree=MozillaTry is fairly often failing to load ("Parsing pushlog failed :(") is the number of heads that have yet again built up in the tryserver's repo. Fetching the URL that tbpl uses to redirect to the pushlog, for try and then for mozilla-central:

time wget http://tests.themasta.com/tinderboxpushlog/fetchraw.php?site=pushlog\&url=try/pushloghtml%3Fstartdate=12+hours+ago%26enddate=now
...
real	0m57.893s

time wget http://tests.themasta.com/tinderboxpushlog/fetchraw.php?site=pushlog\&url=mozilla-central/pushloghtml%3Fstartdate=12+hours+ago%26enddate=now
...
real	0m3.692s

Sure, there are a few more pushes in try's pushlog than in mozilla-central, but not enough to make 4 seconds versus a minute reasonable. Last reset seems to have been in bug 550256 on March 12th, and the one before was bug 529156 December 2nd. Not sure whether we've been trying like crazy over the last six weeks, or we just lived with it being truly awful over the winter.

Comment 1

[Armen [:armenzg]]

John can you please pull the stats you need from try so we can ask IT to reset the repo?

Comment 2

[Phil Ringnalda (:philor)]

Even on the fastest connection available to me, http://tests.themasta.com/tinderboxpushlog/?tree=MozillaTry is no longer usable at all.

Comment 3

[Chris AtLee [:catlee]]

John, please gather your stats so we can schedule a downtime to reset the try repo.

Comment 5

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #3)
> John, please gather your stats so we can schedule a downtime to reset the try
> repo.

Gathering stats now - sorry for the delay.

Comment 6

[John O'Duinn [:joduinn] (please use "needinfo?" flag)]

(In reply to comment #5)
> (In reply to comment #3)
> > John, please gather your stats so we can schedule a downtime to reset the try
> > repo.
> 
> Gathering stats now - sorry for the delay.

all stats gathered and blogged. Now just need a downtime to reset the try repo.

Comment 7

[Nick Thomas [:nthomas] (UTC+12)]

I'll pass this over to aravind at 7pm tonight.

Comment 8

[Nick Thomas [:nthomas] (UTC+12)]

Aravind, please go ahead and reclone try from mozilla-central (per the doc).

Comment 9

[Nick Thomas [:nthomas] (UTC+12)]

Aravind reports he has finished the work on hg.m.o, back to us for master & pushes.

Comment 10

[Nick Thomas [:nthomas] (UTC+12)]

Pushed 1.9.2 over to try, reconfig'd master, pushed a dummy change and got builds. Just waiting for tinderbox.m.o now.

Comment 11

[Nick Thomas [:nthomas] (UTC+12)]

All done. Bug 564349 for a possible improvement to TBPL for try.

Comment 12

[Jeff Muizelaar [:jrmuizel]]

time wget http://tests.themasta.com/tinderboxpushlog/fetchraw.php?site=pushlog\&url=try/pushloghtml%3Fstartdate=12+hours+ago%26enddate=now
--09:00:24--  http://tests.themasta.com/tinderboxpushlog/fetchraw.php?site=pushlog&url=try/pushloghtml%3Fstartdate=12+hours+ago%26enddate=now
           => `fetchraw.php?site=pushlog&url=try%2Fpushloghtml?startdate=12+hours+ago&enddate=now'
Resolving tests.themasta.com... 85.13.134.173
Connecting to tests.themasta.com|85.13.134.173|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html,charset=utf-8]

    [               <=>                   ] 1,158,850    370.27K/s             

09:01:26 (369.15 KB/s) - `fetchraw.php?site=pushlog&url=try%2Fpushloghtml?startdate=12+hours+ago&enddate=now' saved [1158850]


real	1m1.726s
user	0m0.009s
sys	0m0.055s

This doesn't seem to have gotten any better.

Comment 13

[[e-mail] (:bhearsum)]

I'm not sure why TBPL is taking so long to load, but the pushlog itself is very quick:
foo-ix-blah:~ bhearsum$ time wget http://hg.mozilla.org/try/pushlog
--2010-05-07 09:08:34--  http://hg.mozilla.org/try/pushlog
Resolving hg.mozilla.org... 10.2.74.67, 10.2.74.66
Connecting to hg.mozilla.org|10.2.74.67|:80... connected.
HTTP request sent, awaiting response... 200 Script output follows
Length: unspecified [application/atom+xml]
Saving to: `pushlog.1'

    [    <=>                                ] 377,712      426K/s   in 0.9s    

2010-05-07 09:08:36 (426 KB/s) - `pushlog.1' saved [377712]


real	0m1.696s
user	0m0.005s
sys	0m0.015s
foo-ix-blah:~ bhearsum$

Comment 14

[Phil Ringnalda (:philor)]

Two things: pushlog is a completely different beast, in terms of db access, than pushlog?startdate= (even when the startdate gives you the same results as no startdate param), and, look at the _size_ of what Jeff was fetching, less than 12 hours after the pushes that initially filled the repo. Now that those are out of the 12 hour query, I've gone from 1m28s last night to 0m11s now. So while it would be nice if it was possible to fill the repo with something like a clone of m-c plus a pull of 1.9.2 if that wouldn't put huge pushes in the pushlog, verified fixed and usable again, thanks.