749753 - balrog client needs to set non-locale specific parts of release blobs

Status: RESOLVED FIXED

(Release Engineering :: General, defect)

Description

[[e-mail] (:bhearsum)]

Request matching for OS X builds is currently broken on Balrog, because the dated blobs do not have alias' set. We have manually set-up alias' in the -latest blobs, which is the only reason Mac updates are working at all.

We've got a couple of ways to fix this:
1) Have the Balrog client set-up alias'. We might need additional API methods to do this (eg, /releases/:release/builds/:platform POST)
2) Have the Balrog client submit the same data to all possible platforms, rather than alias.

Comment 1

[Nick Thomas [:nthomas] (UTC+12)]

(In reply to Ben Hearsum [:bhearsum] from comment #0)
> 2) Have the Balrog client submit the same data to all possible platforms,
> rather than alias.

I'd like to try to avoid this as duplicated data will not be easily comparable.

3) Resolve the platform in the partial blog using the aliases in the latest blob (although this feels a bit like backing yourself into a corner)

Comment 2

[[e-mail] (:bhearsum)]

This affects newly created -latest blobs, too, such as when we emptied out the Nightly one for bug 819349.

Comment 3

[[e-mail] (:bhearsum)]

(In reply to Nick Thomas [:nthomas] from comment #1)
> (In reply to Ben Hearsum [:bhearsum] from comment #0)
> > 2) Have the Balrog client submit the same data to all possible platforms,
> > rather than alias.
> 
> I'd like to try to avoid this as duplicated data will not be easily
> comparable.
> 
> 3) Resolve the platform in the partial blog using the aliases in the latest
> blob (although this feels a bit like backing yourself into a corner)

When you say "partial" do you mean dated? (Eg, we find Firefox-mozilla-central-nightly-20130101010101 when matching a request, then somehow decide we should look at Firefox-mozilla-central-nightly-latest to check for alias'.)

Comment 4

[Nick Thomas [:nthomas] (UTC+12)]

That seems likely, although in retrospect 'magic' doesn't seem like a very good idea. 

This affects nigthlies now but will be relevant to releases too, which points towards the API method.

Comment 5

[Rail Aliiev [:rail]]

back to the pool.

Comment 6

[[e-mail] (:bhearsum)]

Nick and I chatted about this and decided that it's not a blocker for having nightly builds in production. In the current state of the world we need them for two reasons:
1) To match incoming requests to blobs (requires alias' in the dated blobs)
2) To serve updates at all (requires alias' in the latest and release blobs)

Nick proposed a better solution to matching incoming requests to blobs, which is filed in bug 852738, so we won't absolutely need them in dated blobs after that's fixed.

The latest blobs can be managed by hand for now. We'll only need to update alias' in them if we need new ones, or have to delete or reset a blob for some reason.

The release blobs will definitely need to get updated by some sort of API. We figured that this can happen in whatever replaces the "updates" builder in a Balrog world. (It would need to set alias, release notes URLs, and a bunch of other information.) Let's use this bug to track that.

Comment 7

[[e-mail] (:bhearsum)]

The API side of this is done - we support updating overall release blobs in https://github.com/mozilla/balrog/blob/master/auslib/admin/views/releases.py#L215 (that's /releases/:release POST). The Balrog submitter still needs to deal with setting alias' and possibly other things.

Comment 8

[[e-mail] (:bhearsum)]

This happened at some point: https://github.com/mozilla/build-tools/blob/master/lib/python/balrog/submitter/cli.py#L94