1282898 - allow multifile updates that point directly to blobs instead of product names

Status: RESOLVED FIXED

(Release Engineering Graveyard :: Applications: Balrog (backend), defect)

Description

[[e-mail] (:bhearsum)]

The current multifile update system was designed for GMP with the assumption that it would also work for System Addons. Unfortunately, now that System Addons want to have a revision number that is unique (and increasing) for each set of things we ship, it won't work very well. Because the revision is a property of the set, it must live on the superblob. However, the current superblob only points at product names, and Balrog re-parses the rules table for each of those before ending up with blobs. This means that the current superblob doesn't have enough information to effectively increment the revision number.

Rail and I did some brainstorming about this, and realized that if we had a superblob that pointed at complete release blobs instead of product names, we'd be able to have revision numbers while still putting addon-specific information in their own blobs.

Building on the examples from bug 1281347, things would look something like this:
Releases:
* Loop-1.0
* Loop-2.0
* Pocket-1.0
* SystemAddons-Rev5
** Points at Loop-1.0 and Pocket-1.0
* SystemAddons-Rev6
** Points at Loop-2.0 and Pocket-1.0
* SystemAddons-Rev7
** Points at Loop-1.0 and Pocket-1.0
* SystemAddonsRev8
** Points at Loop-1.0 and Pocket-2.0

We'd just need one rule per version to manage the System Addons. When we ship Loop 1.0 and Pocket 1.0, we'd use:
product: SystemAddons, version: 45.0, channel: release, mapping: SystemAddons-Rev5, rate: 100%

When we want to start rolling out Loop 2.0, we'd use:
product: SystemAddons, version: 45.0, channel: release, mapping: SystemAddons-Rev6, rate: 10%

If we discover a problem and want to revert back to Loop 1.0:
product: SystemAddons, version: 45.0, channel: release, mapping: SystemAddons-Rev7, rate: 100%

And if we want to fully ship Pocket-2.0 afterwards:
product: SystemAddons, version: 45.0, channel: release, mapping: SystemAddons-Rev8, rate: 100%

Comment 1

[[e-mail] (:bhearsum)]

Ninad, I'm assigning this to you per our IRC conversation. I should have some time tomorrow to chat a bit more about the implementation.

Comment 2

[[e-mail] (:bhearsum)]

Robert, did we decide where in the XML the revision number would be?

Comment 3

[Robert Helmer [:rhelmer]]

(In reply to Ben Hearsum (:bhearsum) from comment #2)
> Robert, did we decide where in the XML the revision number would be?

What I have been coding against is revision as an attribute on the addons element:

<updates>
    <addons revision="123">
    </addons>
</updates>

Comment 4

[[e-mail] (:bhearsum)]

(In reply to Robert Helmer [:rhelmer] from comment #3)
> (In reply to Ben Hearsum (:bhearsum) from comment #2)
> > Robert, did we decide where in the XML the revision number would be?
> 
> What I have been coding against is revision as an attribute on the addons
> element:
> 
> <updates>
>     <addons revision="123">
>     </addons>
> </updates>

Hm, while reviewing Ninad's initial work on this I realized that things would be a lot simpler on the server side if we made revision an attribute of the updates elment (eg: <updates revision="123">). Would that still work for the client?

Comment 5

[Robert Helmer [:rhelmer]]

(In reply to Ben Hearsum (:bhearsum) from comment #4)
> (In reply to Robert Helmer [:rhelmer] from comment #3)
> > (In reply to Ben Hearsum (:bhearsum) from comment #2)
> > > Robert, did we decide where in the XML the revision number would be?
> > 
> > What I have been coding against is revision as an attribute on the addons
> > element:
> > 
> > <updates>
> >     <addons revision="123">
> >     </addons>
> > </updates>
> 
> Hm, while reviewing Ninad's initial work on this I realized that things
> would be a lot simpler on the server side if we made revision an attribute
> of the updates elment (eg: <updates revision="123">). Would that still work
> for the client?

WFM

Comment 6

[[e-mail] (:bhearsum)]

(In reply to Robert Helmer [:rhelmer] from comment #5)
> (In reply to Ben Hearsum (:bhearsum) from comment #4)
> > (In reply to Robert Helmer [:rhelmer] from comment #3)
> > > (In reply to Ben Hearsum (:bhearsum) from comment #2)
> > > > Robert, did we decide where in the XML the revision number would be?
> > > 
> > > What I have been coding against is revision as an attribute on the addons
> > > element:
> > > 
> > > <updates>
> > >     <addons revision="123">
> > >     </addons>
> > > </updates>
> > 
> > Hm, while reviewing Ninad's initial work on this I realized that things
> > would be a lot simpler on the server side if we made revision an attribute
> > of the updates elment (eg: <updates revision="123">). Would that still work
> > for the client?
> 
> WFM

Sorry for the flipflop...can we forget I asked about this though? :). Ninad found a way to make it work nicely with <addons revision="123"> as it turns out.

Comment 7

[[github robot]]

Commit pushed to master at https://github.com/mozilla/balrog

https://github.com/mozilla/balrog/commit/3ee593b89385ea9fa30f649b1f10b5a3277f4799
Bug 1282898: allow multifile updates that point directly to blobs instead of product names (#145). r=bhearsum

Comment 8

[[e-mail] (:bhearsum)]

This is in production, thanks Ninad!