601711 - buildbot json should have unique IDs for builds that persist across builds-pending/running/4hr/YYYY-MM-DD.js

Status: RESOLVED FIXED

(Release Engineering :: General, defect, P3)

Description

[David Baron :dbaron: (⌚️UTC-4, no longer working on Mozilla)]

The buildbot JSON output at http://build.mozilla.org/builds/ should have unique IDs for each build that persist across all the files.

I think catlee recently added an ID that persists across builds-pending.js and builds-running.js, but the same ID doesn't show up for completed builds, so it doesn't meet the needs of TBPL-on-buildbot.

TBPL-on-buildbot needs these ids for:
 * mapping "stars" to builds (this is the important bit)
 * preferably, dealing with inability to get new copies of the JSON due to corruption.  (That is, if I can't get a current copy of builds-pending.js, I can still use the old one, along with current builds-4hr.js, as long as I overwrite any information in it that's been replaced.)

Comment 1

[[e-mail] (:bhearsum)]

assigning to Catlee for prioritization.

Comment 2

[Chris AtLee [:catlee]]

This is slightly more complicated than it seems.  There isn't a 1:1 mapping between pushes and pending builds, nor a 1:1 mapping between pending and running builds.

What we can do is publish the build request ids for the pending builds, and for the running builds.  Each pending build has a unique build request id.  But a running build may have more than one request id.

Comment 3

[Chris AtLee [:catlee]]

builds-YYYY-MM-DD.js.gz now contain a 'request_ids' member of the build dict.  The ids here refer back to the ids in the pending builds.

I'm working on exporting those ids from the running builds data.

Comment 4

[Chris AtLee [:catlee]]

Attachment: Modify running query to return one result per build, include request ids

Comment 5

[Nick Thomas [:nthomas] (UTC+12)]

Comment on attachment 486074 [details] [diff] [review]
Modify running query to return one result per build, include request ids

>diff --git a/buildapi/model/query.py b/buildapi/model/query.py
>+                else:
>+                    # Use the latest information for the id and revision
>+                    if r.brid > max(this_result['request_ids']):
>+                        this_result['id'] = r.id
>+                        this_result['revision'] = r.revision

Looks good. The only thing I didn't follow was the utility of tracking build.id if brid is way we link the jobs up.

Comment 6

[Chris AtLee [:catlee]]

(In reply to comment #5)
> Comment on attachment 486074 [details] [diff] [review]
> Modify running query to return one result per build, include request ids
> 
> >diff --git a/buildapi/model/query.py b/buildapi/model/query.py
> >+                else:
> >+                    # Use the latest information for the id and revision
> >+                    if r.brid > max(this_result['request_ids']):
> >+                        this_result['id'] = r.id
> >+                        this_result['revision'] = r.revision
> 
> Looks good. The only thing I didn't follow was the utility of tracking build.id
> if brid is way we link the jobs up.

I think be able to refer to builds by id is still useful.  Referring to a build by brid can be ambiguous.