601976 - should be able to set product/version/channel throttling easily

Status: RESOLVED FIXED

(AUS Graveyard :: Administration, task)

Description

[[e-mail] (:bhearsum)]

We wanted to gradually unthrottle the 3.5.14 beta channel when we pushed updates to it, but were thwarted by the fact that we couldn't unthrottle beta without also unthrottling the release channel -- which has a MU on it that we want to remain throttled. AUS should make this possible/easy.

Comment 1

[Robert Helmer [:rhelmer]]

I am reading the code here since it's unclear to me from the comments in config-dist.php, and I think this might actually be possible already.

It's currently possible to disable throttling for channel/version combinations:
http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/inc/aus.class.php#68

Here's where it's called and how it's compared to productThrottling array:
http://mxr.mozilla.org/mozilla/source/webtools/aus/xml/index.php#109

So, I think you could do:

# Throttle Firefox 3.6.14, all channels
$productThrottling = array(
  'Firefox' => array(
    '3.6.14' => 15
  ),
)

# Disable throttling for Firefox 3.6.14 release channel only
$throttleExceptions = array(
  '3.5.14' => array (
    'release',
  ),
)
Would be nicer to read if productThrottling just let us specify channels. Also, I guess there's no way to specify "product" in throttleExceptions? Maybe morgamic knows better.

In any case, I'd like to be able to test this on a real staging server (bug 601861) but we should be able to scrape up a dev environment if you want to try it.

Comment 2

[Robert Helmer [:rhelmer]]

(In reply to comment #1)
> # Throttle Firefox 3.6.14, all channels
> $productThrottling = array(
>   'Firefox' => array(
>     '3.6.14' => 15
>   ),
> )
> 
> # Disable throttling for Firefox 3.6.14 release channel only
> $throttleExceptions = array(
>   '3.5.14' => array (

Typo there, should be '3.6.14' for both.

Comment 3

[Lukas Blakk [:lsblakk] use ?needinfo]

> So, I think you could do:
> 
> # Throttle Firefox 3.6.14, all channels
> $productThrottling = array(
>   'Firefox' => array(
>     '3.6.14' => 15
>   ),
> )
> 
> # Disable throttling for Firefox 3.6.14 release channel only
> $throttleExceptions = array(
>   '3.5.14' => array (
>     'release',
>   ),
> )

Maybe I'm wrong, but wouldn't this set betatest and releasetest to also be at 15% throttling?

Comment 4

[Lukas Blakk [:lsblakk] use ?needinfo]

It would be great to be able to set it like:

$throttleExceptions = array(
   '3.5.14' => array (
     'release' => 0,
     'beta' => 15,
     'betatest' => 100,
     'releasetest' => 100
   ),

Comment 5

[[e-mail] (:bhearsum)]

I don't think your example is what we want, though. In this scenario, we wanted:
3.5.14 beta channel: 15% of requests get updates
3.5.14 betatest/releasetest channels: 100% of requests get updates
3.5.14 all other channels (including release): 0% of requests get updates


Did I misunderstand your example?

Comment 6

[Robert Helmer [:rhelmer]]

(In reply to comment #5)
> I don't think your example is what we want, though. In this scenario, we
> wanted:
> 3.5.14 beta channel: 15% of requests get updates
> 3.5.14 betatest/releasetest channels: 100% of requests get updates
> 3.5.14 all other channels (including release): 0% of requests get updates
> 
> 
> Did I misunderstand your example?

(In reply to comment #3)
> > So, I think you could do:
> > 
> > # Throttle Firefox 3.6.14, all channels
> > $productThrottling = array(
> >   'Firefox' => array(
> >     '3.6.14' => 15
> >   ),
> > )
> > 
> > # Disable throttling for Firefox 3.6.14 release channel only
> > $throttleExceptions = array(
> >   '3.5.14' => array (
> >     'release',
> >   ),
> > )
> 
> Maybe I'm wrong, but wouldn't this set betatest and releasetest to also be at
> 15% throttling?

No misunderstanding, I was just trying to keep it simple, sorry :)


(In reply to comment #4)
> It would be great to be able to set it like:
> 
> $throttleExceptions = array(
>    '3.5.14' => array (
>      'release' => 0,
>      'beta' => 15,
>      'betatest' => 100,
>      'releasetest' => 100
>    ),

Yes I think this would do what you want.

Comment 7

[Robert Helmer [:rhelmer]]

Here I'll just attach a realistic example as a patch.

Comment 8

[Robert Helmer [:rhelmer]]

Attachment: throttle beta only for 3.5.14

Comment 9

[[e-mail] (:bhearsum)]

Comment on attachment 481045 [details] [diff] [review]
throttle beta only for 3.5.14

This would work, but it completely unthrottles the MU offer on the 3.5.14 release channel -- which we aren't necessarily going to want.

Comment 10

[Robert Helmer [:rhelmer]]

(In reply to comment #9)
> Comment on attachment 481045 [details] [diff] [review]
> throttle beta only for 3.5.14
> 
> This would work, but it completely unthrottles the MU offer on the 3.5.14
> release channel -- which we aren't necessarily going to want.

To make sure I'm understanding correctly: the choice you have right now is to throttle some set of channels (f.e. release, beta) at a particular number (f.e. 15) and leave the rest unthrottled, right?

And what you want is to be able to throttle release and beta at different levels, and not in lockstep like this?

Comment 11

[[e-mail] (:bhearsum)]

(In reply to comment #10)
> (In reply to comment #9)
> > Comment on attachment 481045 [details] [diff] [review] [details]
> > throttle beta only for 3.5.14
> > 
> > This would work, but it completely unthrottles the MU offer on the 3.5.14
> > release channel -- which we aren't necessarily going to want.
> 
> To make sure I'm understanding correctly: the choice you have right now is to
> throttle some set of channels (f.e. release, beta) at a particular number (f.e.
> 15) and leave the rest unthrottled, right?
> 

Correct.

> And what you want is to be able to throttle release and beta at different
> levels, and not in lockstep like this?

Exactly.

Comment 12

[Robert Helmer [:rhelmer]]

(In reply to comment #4)
> It would be great to be able to set it like:
> 
> $throttleExceptions = array(
>    '3.5.14' => array (
>      'release' => 0,
>      'beta' => 15,
>      'betatest' => 100,
>      'releasetest' => 100
>    ),

I think it would be more readable to do it more directly:

# default is 0
$productThrottling = array(
  'Firefox' => array(
    '3.5.14' => array(
      'release' => 50,
      'beta' => 15,
    ),
    '4.07b4' => array(
      'beta' => 15,
    ),  ),
);

Couldn't we then do away with $throttleExceptions entirely? I don't like how throttleExceptions doesn't look at the $product, anyway...

Comment 13

[[e-mail] (:bhearsum)]

Yeah, that is _much_ more readable. I'm assuming the 4.07b4 (sic) block means "15% of requests on its beta channel will receive an update" as well as 100% of requests from any other channel?

One downside of explicitly needing to list channels that need throttling is that if we ever had to, say, mass-throttle all the partner channels, it would be a PITA. I don't think that outweighs the benefits, though.

Comment 14

[craig.comperatore]

A Pivotal Tracker story has been created for this Bug: https://www.pivotaltracker.com/story/show/98494596

Comment 15

[craig.comperatore]

Craig Comperatore deleted the linked story in Pivotal Tracker

Comment 16

[[e-mail] (:bhearsum)]

This is much better in Balrog.