463005 - try server talos boxes seem to skip builds under load

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[Vladimir Vukicevic [:vlad] [:vladv] (needinfo me, slow to respond)]

Not sure what happened, but the talos boxes seem to have skipped my builds -- there's lots of usage on the try server atm, and it looks like it just went straight to the build after mine.

I think http://tinderbox.mozilla.org/showbuilds.cgi?tree=MozillaTry&maxdate=1225775074&legend=0&norules=1 will show roughly the right time range.

Comment 1

[alice nodelman [:alice] [:anode]]

This is a version of the same issue as bug 457885.

Unfortunately, it's been a known state for a while that we skip builds for talos try and production.  We have a pretty good idea how to fix it but it is going to take some time for it to works it way up the priority list.

Comment 2

[Boris Zbarsky [:bzbarsky]]

For what it's worth, this increases the amount of time needed to get try talos results from "many hours" to "many days", since the probability of success is at most the square of the probability of your build being tested, and you need both test and baseline tested.

I tried testing some stuff this morning, and no more than half of my builds actually ended up running in general.  On Linux, none of the test builds ended up being run (and the baseline got run twice!).

Comment 3

[alice nodelman [:alice] [:anode]]

Moving out till next quarter.

Comment 5

[Jonathan Kew [:jfkthame]]

At the same time as skipping the talos run I wanted to see, I notice that it
appeared to have run multiple talos tests on one of the following patches in
the queue.

BTW, it'd also be nice if builds-in-progress would identify themselves on the
tinderbox page, so we could see whether the one we're waiting for has at least
started.

Comment 6

[Chris AtLee [:catlee]]

Attachment: Use sendchange for talos try

This patch should enable sendchange to work between the build try master and the talos try master.

Comment 7

[Chris AtLee [:catlee]]

Attachment: Set uploadpath property after uploading try build

Comment 8

[alice nodelman [:alice] [:anode]]

catlee you are totally my hero for putting together these patches!

Comment 9

[Chris AtLee [:catlee]]

Comment on attachment 368118 [details] [diff] [review]
Set uploadpath property after uploading try build

changeset:   227:e5365edfca3c

Comment 10

[Chris AtLee [:catlee]]

Comment on attachment 368116 [details] [diff] [review]
Use sendchange for talos try

changeset:   1031:47e1c2489591

Comment 11

[Chris AtLee [:catlee]]

changeset:   1032:4f05e8a34fc5

was also checked in to fix some bustage.