Multiple rates for rules

RESOLVED WONTFIX

Status

P3
normal
RESOLVED WONTFIX
a year ago
9 months ago

People

(Reporter: rehan, Assigned: peterbe)

Tracking

Firefox Tracking Flags

(Not tracked)

Details

(Reporter)

Description

a year ago
As more groups choose to ship system addons through Go Faster we are hitting an issue where multiple groups want to stagger roll out of their addons to the same version/channel.

Currently we can only choose one rate per channel per version and have a fallback for the rest of the users. Meaning only one addon can do a staggered roll out or all the addons have to sync their roll out timeline.

The only other workaround has been shipping an addon to 100% of users and having the addon handle sampling itself.

It would be great to have a way to set the rate individually for each addon.
As stated, we'd have to put throttle rates into either the SystemAddon Superblob, or individual SystemAddon Releases. This has the unfortunate side effect of making the backgroundRate of the SystemAddon Rules more or less ignored (we'd always set it to 100%, and if we set to less, we'd end up having two levels of backgroundRate considered -- very confusing).

One possible alternative would be to switch to GMP-style multifile updates. The way these work is by specifying a list of products in the Superblob, and querying the Rules table for each of them to find a list of Releases to return. Because the Rules table is queried for each Product, it means we can apply a separate backgroundRate to each. This would have a couple of implications:
- It would mean we'd have a separate set of Rules per SystemAddon, which could be good and bad -- it makes it easier to think about the state of one particular SystemAddon, but it makes it more difficult to understand the overall state at a glance.
- We'd probably need to find a way to map the SystemAddon required signoffs to each new product as well - otherwise we'd have to set them up again each time we add a new SystemAddon.


Either way we go here I think we need consider what type of response the client requires. When we talked about this a long time ago, I think we said that the client won't cope well with a SystemAddon missing from the response, and that we should always return either the new version being rolled out, or the previous version that we rolled out in the response. Rob, is that still true?
Flags: needinfo?(rhelmer)
(In reply to Ben Hearsum (:bhearsum) from comment #1)
> As stated, we'd have to put throttle rates into either the SystemAddon
> Superblob, or individual SystemAddon Releases. This has the unfortunate side
> effect of making the backgroundRate of the SystemAddon Rules more or less
> ignored (we'd always set it to 100%, and if we set to less, we'd end up
> having two levels of backgroundRate considered -- very confusing).
> 
> One possible alternative would be to switch to GMP-style multifile updates.
> The way these work is by specifying a list of products in the Superblob, and
> querying the Rules table for each of them to find a list of Releases to
> return. Because the Rules table is queried for each Product, it means we can
> apply a separate backgroundRate to each. This would have a couple of
> implications:
> - It would mean we'd have a separate set of Rules per SystemAddon, which
> could be good and bad -- it makes it easier to think about the state of one
> particular SystemAddon, but it makes it more difficult to understand the
> overall state at a glance.
> - We'd probably need to find a way to map the SystemAddon required signoffs
> to each new product as well - otherwise we'd have to set them up again each
> time we add a new SystemAddon.
> 
> 
> Either way we go here I think we need consider what type of response the
> client requires. When we talked about this a long time ago, I think we said
> that the client won't cope well with a SystemAddon missing from the
> response, and that we should always return either the new version being
> rolled out, or the previous version that we rolled out in the response. Rob,
> is that still true?

Yes this is still true... whatever Balrog returns after an update check is
considered to be the latest update set, and the client will switch from the "old"
to the "new" set.

Rehan, for simplicity and ease of testing etc. we've enforced the above in the client,
to date... I am open to changing it but we'll need to modify Firefox also if we do so.

This would actually allow us to simplify things in AddonManager since it'd be more like
the way regular add-on updates work. 

I'd expect QA and relman to have concerns here so it's probably worth chatting with them
before we do any serious work.
Flags: needinfo?(rhelmer) → needinfo?(rdalal)
(Assignee)

Comment 3

10 months ago
Chatted about what Rehan needs to Balrog for this all to work. 
https://docs.google.com/document/d/1seFGmoYz6pUWzeemnN3dcoEDI87d4l2fRjMEr0F978s/edit#

"Simple" conclusion is that we need the ability to match multiple Rules and merge their releases into one in realtime.
This requires a fundamental change to how rule matching works. And also instead of yielding the matched rule's release, we need to merge releases from potentially multiple rules. 

The Superblob concept/hack is an antipattern that needs to (slowly) be phased out in favor of the solution of merging multiple rules. Let's not do that in one sweep (db migrations etc are hard!) but shift that responsibility to up to Rehan etc to slowly move from using Superblobs to instead using multiple rules with Priority unset. 

We (Ben, Ethan, myself) need to percolate this idea and figure out how disruptive the change is. Risks and magnitude and feasibility. 

In terms of timing, we're loosely aiming to extend the current UI with a new "simulation tool" to better visualize which Rule(s) match which client(s). This is going to greatly help to have around when making a big change to the engine. Perhaps we should build that first so we can more confidently change the engine to support matching multiple rules. This'll come extra-in-handy when we add the ability to match version *ranges*.
Assignee: nobody → peterbe
Flags: needinfo?(rdalal)

Comment 4

10 months ago
I mentioned this in a meeting we had yesterday, but I want to echo it here as well: I don't know if we can support multiple Rules matching a single request the way it's described in comment #3. If I've understood correctly, it would mean that Firefox update requests have the potential to match multiple Rules, which would make it impossible to construct a reasonable response (you'd have multiple Releases, and we can stack them for Firefox the way we can for SystemAddons).

I definitely want to find a way to improve the SystemAddons situation, we just have to be sure that we don't break Firefox.

Updated

10 months ago
Priority: -- → P3
Last week we made the decision to move System Addons out of Balrog. Since this bug is focused specifically on System Addon use cases, I'm going to close it. There may be a reason we may want something like this for Firefox in the future, but we can file a new bug with that new use case in mind if it comes up.
Status: NEW → RESOLVED
Last Resolved: 9 months ago
Resolution: --- → WONTFIX
You need to log in before you can comment on or make changes to this bug.