585682 - TBPL should take advantage of builds-running.js

Status: RESOLVED FIXED

(Tree Management Graveyard :: TBPL, defect)

Description

[Shawn Wilsher :sdwilsh]

We have a handy little json file that tells what builds are currently running:
http://build.mozilla.org/builds/builds-running.js

TBPL should use it instead of trying to guess what a build is for.

Comment 1

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

There's code within my work on bug 586539 that does this, though it's not especially separable.

Comment 2

[Arpad Borsos [:Swatinem]]

Attachment: part 1: move machine resolution into Data

Comment 3

[Arpad Borsos [:Swatinem]]

Attachment: part 2: use pending/running.json to display pending/running builds

Yay, no more guessedRev :)
But I noticed that the running json does not include Nightly builds.

Comment 4

[Arpad Borsos [:Swatinem]]

catlee, nthomas: re comment 3: why are nightly builds not included in the running json? Also, It would be cool to generate the json files more often, I guess every 2 minutes should work well.

Comment 5

[Arpad Borsos [:Swatinem]]

(In reply to comment #4)
> catlee, nthomas: re comment 3: why are nightly builds not included in the
> running json? Also, It would be cool to generate the json files more often, I
> guess every 2 minutes should work well.

The different refresh intervals can lead to running jobs still being displayed while the tinderbox data already contains the finished results.

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #4)
> catlee, nthomas: re comment 3: why are nightly builds not included in the
> running json? Also, It would be cool to generate the json files more often, I
> guess every 2 minutes should work well.

I don't know why they wouldn't be included....But they won't indicate which revision they're building, since we don't know what revision it's got until the build finishes.

Comment 7

[Arpad Borsos [:Swatinem]]

(In reply to comment #6)
> (In reply to comment #4)
> > catlee, nthomas: re comment 3: why are nightly builds not included in the
> > running json? Also, It would be cool to generate the json files more often, I
> > guess every 2 minutes should work well.
> 
> I don't know why they wouldn't be included....But they won't indicate which
> revision they're building, since we don't know what revision it's got until the
> build finishes.

Then that is the reason. I guess we can live with not displaying running N though.

Comment 8

[Arpad Borsos [:Swatinem]]

Attachment: part 2: use pending/running.json to display pending/running builds

>+++ b/js/Data.js
>+            var key = job.number;

Per discussion with catlee, "number" is not unique and thus not a good key for the running jobs. I was seeing some running jobs not being cleared correctly, and I suspect that to be the cause.
I’m using another key now and added a comment explaining why that is needed.

Comment 9

[Phil Ringnalda (:philor)]

Not the end of the world, but it's a bit odd to have hidden builds like R2D and R3D show while they are running and disappear when they finish.

Comment 10

[Arpad Borsos [:Swatinem]]

(In reply to comment #9)
> Not the end of the world, but it's a bit odd to have hidden builds like R2D and
> R3D show while they are running and disappear when they finish.

What blocks us from just enabling scraping for these builds? D2D and D3D are now enabled by default, we should have the tests show up on tinderbox as well.

Comment 11

[Phil Ringnalda (:philor)]

I think they are hidden because they're pointless: as you say, the things they pref on are also on by default, so they've become the same test as R. They just happened to be the two hidden builds that were still running when I looked at a push. If you prefer, s/R2D/RGL/, or 10.6 xpcshell, or anything else that's hidden because we want it to run, but we don't yet have the permaorange cleared up.

Comment 12

[Phil Ringnalda (:philor)]

The only other problem I noticed after running it for a while is the (relatively) long period when a build is shown as both completed and still running. I presume that's mostly obvious because builds-running.js doesn't refresh often enough, but it's always going to happen unless both running and finished are coming from the same data source. Can we recognize that something which shows up as both finished and running is the same build, and remove the running one?

Comment 13

[Arpad Borsos [:Swatinem]]

We will track builds through all their different states sometime when we completely restructure tbpl to use a builddb-derived db. For now I think this is the best we wan do for the time being without too much hassle.

Comment 14

[Markus Stange [:mstange]]

http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/fb83dd464be0
http://hg.mozilla.org/users/mstange_themasta.com/tinderboxpushlog/rev/20b06fa7105c

Comment 15

[Nick Thomas [:nthomas] (UTC+12)]

Yay!