consider matching incoming releases by buildid directly, not through their blob

RESOLVED WONTFIX

Status

Release Engineering
Applications: Balrog (backend)
RESOLVED WONTFIX
4 years ago
4 years ago

People

(Reporter: bhearsum, Unassigned)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

4 years ago
[12:12:20] <bhearsum> nthomas: random thing i just remembered that i don't want to forget...catlee and i were talking a couple of weeks ago and started wondering why we use release name to identify incoming releases instead of buildid. the latter would be much cheaper because it doesn't require extra db queries. and it looks like the only thing we compare against in queryMatchesRelease is buildid: https://github.com/mozilla/balrog/blob/master/auslib/AUS.py#L58

Not sure if we want to do this or not, just filing to avoid forgetting about it.
(Reporter)

Updated

4 years ago
Blocks: 797033
One reason is that the buildID may differ between platforms; in theory anyway, even though that's rare these days. Using a release blob also (kinda) asserts that the apps are the same.

We could add a fast lookup to avoid loading json blobs if we need it, eg a table with product, version, buildID, release blob name, and index it all up the wazoo. Or just cache results for a while.
(Reporter)

Comment 2

4 years ago
(In reply to Nick Thomas [:nthomas] from comment #1)
> One reason is that the buildID may differ between platforms; in theory
> anyway, even though that's rare these days. Using a release blob also
> (kinda) asserts that the apps are the same.

Good point. It's certainly not guaranteed that the buildids are the same...eg, if an en-US build fails and we retrigger it, it may have a different buildid.

> We could add a fast lookup to avoid loading json blobs if we need it, eg a
> table with product, version, buildID, release blob name, and index it all up
> the wazoo. Or just cache results for a while.

If we add buildID to the releases table it further locks us release-global buildids. This could get ugly if we have a nasty release with some rebuilds. If we notice a buildid difference late in the game, we may have to rebuild or live without partials (or add a nasty hack to Balrog).

I also realized while thinking about this that we've had cases in the past (though extremely rare) where we've shipped build1 of some platforms, and build2 of others. I'm not sure how we'd handle that in Balrog right now. Manually create a new blob, I guess?
(Reporter)

Comment 3

4 years ago
Nick and I talked about this last night. We decided that locking ourselves into buildids being the same across platforms wasn't something we want to. There are cases in release builds where this could happen (if we "force build" a failed en-US build instead of "rebuild"ing it), and they could cause several hour delays in releasing if we don't catch it earlier enough.

Additionally, I realized this morning that this will break partials for nightlies, because buildids are not consistent across the -latest blobs between the time the first en-US build submits, and the time that all repacks have completed.
Status: NEW → RESOLVED
Last Resolved: 4 years ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.